They provide relaxed-ordered atomic access semantic. Due to the
FreeBSD memory model, the operations are syntaxical wrappers around
the volatile accesses. The volatile qualifier is used to ensure that
the access not optimized out and in turn depends on the volatile
semantic as implemented by supported compilers.
The motivation for adding the operation is to help people coming from
other systems or knowing the C11/C++ standards where atomics have
special type and require use of the special access operations. It is
still the case that FreeBSD requires plain load and stores of aligned
integer types to be atomic.
Suggested by: jhb
Reviewed by: alc, jhb
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D13534
of low physical memory:
Update the comment about leaving the permanent mapping in place. This
also improves the wording of the comment. PTD 0 is still left alone
because it is fairly important that it was unmapped earlier, and the
comment now describes the unmapping of the other low PTDs that the code
actually does.
Reviewed by: kib
it by a transient double mapping for the one instruction in ACPI wakeup
where it is needed (and for many surrounding instructions in ACPI resume).
Invalidate the TLB as soon as convenient after undoing the transient
mapping. ACPI resume already has the strict ordering needed for this.
This fixes the non-trapping of null pointers and other garbage pointers
below NBPDR (except transiently). NBPDR is quite large (4MB, or 2MB for
PAE).
This fixes spurious traps at the first instruction in VM86 bioscalls.
The traps are for transiently missing read permission in the first
VM86 page (physical page 0) which was just written to at KERNBASE in
the kernel. The mechanism is unknown (it is not simply PG_G).
locore uses a similar but larger transient double mapping and needs
it for 2 instructions instead of 1. Unmap the first PDE in it after
the 2 instructions to detect most garbage pointers while bootstrapping.
pmap_bootstrap() finishes the unmapping.
Remove the avoidance of the double mapping for a recently fixed special
case. ACPI resume could use this avoidance (made non-special) to avoid
any problems with the transient double mapping, but no such problems
are known.
Update comments in locore. Many were for old versions of FreeBSD which
tried to map low memory r/o except for special cases, or might have
allowed access to low memory via physical offsets. Now all kernel
maps are r/w, and removal of of the double map disallows use of physical
offsets again.
when KERNLOAD is smaller than NBPDR (not the default) and PG_G is
enabled (the default if the CPU supports it). This case has relatively
minor problems with coherency of the permanent double mapping, but the
fix in r167869 to improve coherency creates page tables with 3 different
errors so never worked.
The permanent double mapping is fundamentally broken and will be removed
soon. It fundamentally breaks trapping for null pointers and requires
complications to avoid cache coherency bugs. It is currently used for
only a single instruction in ACPI resume,
Many fixes VM86 and/or ACPI and/or the double map were attempted near
r1200000. r167869 attempted to fix cache coherency bugs in an unusual
case, but the bugs were unreachable because older errors in page tables
caused a crash first.
This commit just makes r167869 work as intended. Part 1 of these fixes
fixed the other errors, but also stopped mapping the PDE for KERNBASE
as a large page, so double mapping of this PDE only causes the same
problems as when KERNLOAD is the default. Except for the problem of
trapping null pointers, r167869 could be used to fix these problems,
but it is inactive in usual cases. The only known other problem is
that incoherent permissions for page 0 cause spurious traps in VM86
BIOS calls.
Reviewed by: kib
when KERNLOAD is not a multiple of NBPDR (not the default) and PSE is
enabled (the default if the CPU supports it). Addresses in PDEs must
be a multiple of NBPDR in the PSE case, but were not so in the crashing
case.
KERNLOAD defaults to NBPDR. NBPDR is 4 MB for !PAE and 2 MB for PAE.
The default can be changed by editing i386/include/vmparam.h or using
makeoptions. It can be changed to less than NBPDR to save real and
virtual memory at a small cost in time, or to more than NBPDR to waste
real and virtual memory. It must be larger than 1 MB and a multiple of
PAGE_SIZE. When it is less than NBPDR, it is necessarily not a multiple
of NBPDR. This case has much larger bugs which will be fixed in part 2.
The fix is to only use PSE for physical addresses above <KERNLOAD
rounded _up_ to an NBPDR boundary>. When the rounding is non-null,
this leaves part of the kernel not using large pages. Rounding down
would avoid this pessimization, but would break setting of PAT bits
on i/o pages if it goes below 1MB. Since rounding down always goes
below 1MB when KERNLOAD < NBPDR and the KERNLOAD > NBPDR case is not
useful, never round down.
Fix related style bugs (e.g., wrong literal values for NBPDR in comments).
Reviewed by: kib
memory:
Load the kernel eflags less magically, as in locore. The magic increased
when I removed eflags from the pcb in r305899.
Remove a jump to low memory that became garbage when the i386 version was
mostly replaced by the amd64 version in r235622.
The amd64 version is very similar. It still loads the flags magically,
but is not missing comments about using the special page table.
Reviewed by: kib
The DTrace fasttrap entry points expect a struct reg containing the
register values of the calling thread. Perform the conversion in
fasttrap rather than in the trap handler: this reduces the number of
ifdefs and avoids wasting stack space for traps that don't involve
DTrace.
MFC after: 2 weeks
Logically, extend r286288 to cover all threads, by default.
The world has largely moved on from i386. Most FreeBSD users and developers
test on amd64 hardware. For better or worse, we have written a non-trivial
amount of kernel code that relies on stacks larger than 8 kB, and it "just
works" on amd64, so there has been little incentive to shrink it.
amd64 had its KSTACK_PAGES bumped to 4 back in Peter's initial AMD64 commit,
r114349, in 2003. Since that time, i386 has limped along on a stack half
the size. We've even observed the stack overflows years ago, but neglected
to fix the issue; see the 20121223 and 20150728 entries in UPDATING.
If anyone is concerned with this change, I suggest they configure their
AMD64 kernels with KSTACK_PAGES 2 and fix the fallout there first. Eugene
has identified a list of high stack usage functions in the first PR below.
PR: 219476, 224218
Reported by: eugen@, Shreesh Holla <hshreesh AT yahoo.com>
Relnotes: maybe
Sponsored by: Dell EMC Isilon
This variable should be pure MI except possibly for reading it in MD
dump routines. Its initialization was pure MD in 4.4BSD, but FreeBSD
changed this in r36441 in 1998. There were many imperfections in
r36441. This commit fixes only a small one, to simplify fixing the
others 1 arch at a time. (r47678 added support for
special/early/multiple message buffer initialization which I want in
a more general form, but this was too fragile to use because hacking
on the msgbufp global corrupted it, and was only used for 5 hours in
-current...)
In the linux ENOADATA is frequently #defined as ENOATTR.
The change is required for an xattrs support implementation.
MFC after: 1 week
Discussed with: netchild
Approved by: pfg
Differential Revision: https://reviews.freebsd.org/D13221
Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone - task.
The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.
For FreeBSD/arm64's cloudabi32 support, I'm going to need a TO_PTR() in
this place. Also use it for all of the other source files, so that the
difference remains as minimal as possible.
MFC after: 2 weeks
Mainly focus on files that use BSD 3-Clause license.
The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.
Special thanks to Wind River for providing access to "The Duke of
Highlander" tool: an older (2014) run over FreeBSD tree was useful as a
starting point.
The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.
Special thanks to Wind River for providing access to "The Duke of
Highlander" tool: an older (2014) run over FreeBSD tree was useful as a
starting point.
Initially, only tag files that use BSD 4-Clause "Original" license.
RelNotes: yes
Differential Revision: https://reviews.freebsd.org/D13133
It is for console presented at 2001 and featuring Pentium III
processor. Even if any of them are still alive and run FreeBSD, we do
not have any sign of life from their users. While removing another
dozens of #ifdefs from the i386 sources reduces the aversion from
looking at the code and improves the platform vitality.
Reviewed by: cem, pfg, rink (XBOX support author)
Sponsored by: The FreeBSD Foundation
Differential revision: https://reviews.freebsd.org/D13016
Some callers of fpusetregs()/npxsetregs(), most importantly
set_fpcontext(), clear reserved bits. But some did not. Do the
clearing in fpusetregs() and remove now redundand operation from
set_fpcontext().
Reported by: Maxime Villard <max@m00nbsd.net>
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
HEAD. Enable VIMAGE in GENERIC kernels and some others (where GENERIC does
not exist) on HEAD.
Disable building LINT-VIMAGE with VIMAGE being default.
This should give it a lot more exposure in the run-up to 12 to help
us evaluate whether to keep it on by default or not.
We are also hoping to get better performance testing.
The feature can be disabled using nooptions.
Requested by: many
Reviewed by: kristof, emaste, hiren
X-MFC after: never
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D12639
All of the kernel dump implementations keep track of the current offset
("dumplo") within the dump device. However, except for textdumps, they
all write the dump sequentially, so we can reduce code duplication by
having the MI code keep track of the current offset. The new
dump_append() API can be used to write at the current offset.
This is needed to implement support for kernel dump compression in the
MI kernel dump code.
Also simplify dump_encrypted_write() somewhat: use dump_write() instead
of duplicating its bounds checks, and get rid of the redundant offset
tracking.
Reviewed by: cem
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D11722
descriptors does not fit into currently allocated LDT, or trim the
return if the range fits partially. Before, the function returned
EINVAL.
Fix two bugs in r324366: use capped num counter for malloc size, and
do not leak allocated buffer on EINVAL (by handling EINVAL case as
normal, see above).
Reviewed by: bde
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Remove mtx_owned() checks from set_user_ldt(). Split the function
into _locked() version which requires the dt_lock spinlock owned, and
make set_user_ldt() a wrapper. Add a comment in swtch.s noting that
the call to the new set_user_ldt() cannot recurse on dt_lock.
Remove #ifdef SMP block, the addend is always zero on UP.
Fix type of set_user_ldt_rv(), making it match the type used for
smb_rendezvous() callback, and remove the cast. Use curproc.
Reviewed by: bde
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
The values from the old address space do not make sense for the new
program. In particular, gsbase might be the TLS base for the old
program but the new program has no TLS now.
amd64 already handles this correctly.
Reported and reviewed by: bde
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Provide consistent snapshot of the requested descriptors by preventing
other threads from modifying LDT while we fetch the data, lock dt_lock
around the read. Copy the data into intermediate buffer, which is
copied out after the lock is dropped.
Comparing with the amd64 version, the read is done byte by byte, since
there is no atomic 64bit read (cmpxchg8b method is too heavy comparing
with the avoided issues).
Improve overflow checking for the descriptors range calculations and
remove unneeded casts. Use unsigned types for sizes.
Allow zero num argument to i386_get_ldt() and i386_set_ldt(). This
case is handled naturally by the code flow.
Reviewed by: bde
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Split the handlers for pop of invalid selectors from the trap frame
into usermode and kernel variants. Usermode handler is kept as is, it
restores the already loaded parts of the trap frame and jumps to set
up a signal delivery to the user process.
New kernel part of the handler emulates IRET treatment of the segments
which would violate access right. It loads NUL selector in the
segment register which load causes the fault, and then continues the
return to interrupted kernel code. Since invalid selectors in the
segment registers in the kernel mode can only exist while kernel still
enters or exits from userspace, we only zero invalid userspace
selectors. If userspace tries to use the segment register, it gets a
signal, as if the processor segment descriptor cache was reloaded.
Reported by: Maxime Villard <max@m00nbsd.net>
Suggested and reviewed by: bde
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Do not return from interrupt using the POP_FRAME;iret instruction
sequence, always jump to doreti.
The user segments selectors saved on the stack might become invalid
because userspace manipulated LDT in a parallel thread. trap() is
aware of such issue, but it is only prepared to handle it at iret and
segment registers load operations in doreti path.
Also remove POP_FRAME macro because it is no longer used.
Reviewed by: bde, jhb (as part of r323722)
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
some still useful bits of the reverted revision.
The problem with the committed fix is that there are still issues with
returning from NMI, when NMI interrupted kernel in a moment where the
kernel segments selectors were still not loaded into registers. If
this happens, the NMI return would loose the userspace selectors
because r323722 does not reload segment registers on return to kernel
mode.
Fixing the problem is complicated. Since an alternative approach to
handle the original bug exists, it makes sence to stop adding more
complexity.
Discussed with: bde
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Care must be taken when updating the active LDT, since parallel
threads might try to load a segment descriptor which is currently
updated. Since the results are undefined, this cannot be ignored by
claiming to be an application race.
Reviewed by: jhb
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
Differential revision: https://reviews.freebsd.org/D12413
Suppose that userspace is executing with the non-standard segment
descriptors. Then, until exception or interrupt handler executed
SET_KERNEL_SEGS, kernel is still executing with user %ds, %es and %fs.
If an interrupt occurs in this window, the interrupt handler is
executed unsafely, relying on usability of the usermode registers. If
the interrupt results in the context switch on return, the
contamination of the kernel state spreads to the thread we switched
to. As result, kernel data accesses might fault or, if only the base
is changed, completely messed up.
More, if the user segment was allocated in LDT, another thread might
mark the descriptor as invalid before doreti code tried to reload
them. In this case kernel panics.
The issue exists for all exception entry points which use trap gate,
and thus do not automatically disable interrupts on entry, and for
lcall_handler.
Fix is two-fold: first, we need to disable interrupts for all kernel
entries, changing the IDT descriptor types from trap gate to interrupt
gate. Interrupts are re-enabled not earlier than the kernel segments
are loaded into the segment registers. Second, we only load the
segment registers from the trap frame when returning to usermode. For
the later, all interrupt return paths must happen through the doreti
common code.
There is no way to disable interrupts on call gate, which is the
supposed mode of servicing for lcall $7,$0 syscalls. Change the LDT
descriptor 0 into a code segment type and point it to the userspace
trampoline which redirects the syscall to int $0x80.
All the measures make the segment register handling similar to that of
amd64. We do not apply amd64 optimizations of not reloading segment
registers on return from the syscall.
Reported by: Maxime Villard <max@m00nbsd.net>
Tested by: pho (the non-lcall part)
Reviewed by: jhb
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
Differential revision: https://reviews.freebsd.org/D12402
