distributeworld relies on "foo" sorting directly before "foo type=...",
but with glibc both en_US and en_GB have "fooa" sort between "foo" and
"foo z", resulting in some files (in particular, id due to "ident"
sorting before "id type=" but after "id") not being included in the meta
files and thus not included in the dist tarballs. Forcing use of the C
locale ensures this does not occur.
Reviewed by: brooks
Obtained from: CheriBSD
Differential Revision: https://reviews.freebsd.org/D35812
FreeBSD and macOS have a test that treats == as an alias for =, but
Linux tends to use GNU coreutils (when not a builtin) which does not.
Use the standard syntax instead for compatibility.
Obtained from: CheriBSD
Differential Revision: https://reviews.freebsd.org/D35811
Currently the host's database files are used, but on non-FreeBSD these
are not necessarily sufficient; in particular, Linux does not have a
wheel group. Instead, use -N to use the in-tree database files when
creating the METALOG entries, as is done for the recursive makes via
IMAKE_MTREE.
Reviewed by: brooks
Obtained from: CheriBSD
Differential Revision: https://reviews.freebsd.org/D35810
With clang 15, the following -Werror warning is produced:
stand/i386/boot2/boot2.c:358:6: error: a function declaration without a prototype is deprecated in all versions of C [-Werror,-Wstrict-prototypes]
This is because parse() is declared with a (void) argument list, and
defined with an empty argument list. Make the definition match the
declaration.
MFC after: 3 days
Merge commit 1737d8397a0 by Brooks Davis:
time() is declared in time.h
This fixes a -Werror warning from clang 15:
tests/sys/cddl/zfs/bin/readmmap.c:97:9: error: call to undeclared function 'time'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
Obtained from: https://github.com/CTSRD-CHERI/cheribsd/commit/1737d8397a0
MFC after: 3 days
Since https://github.com/llvm/llvm-project/commit/ca75ac5f04f2, clang 15
has a new warning about _Generic selection expressions, such as used in
math.h:
lib/libc/gdtoa/_ldtoa.c:82:10: error: due to lvalue conversion of the controlling expression, association of type 'volatile float' will never be selected because it is qualified [-Werror,-Wunreachable-code-generic-assoc]
switch (fpclassify(u.e)) {
^
lib/msun/src/math.h:109:2: note: expanded from macro 'fpclassify'
__fp_type_select(x, __fpclassifyf, __fpclassifyd, __fpclassifyl)
^
lib/msun/src/math.h:85:14: note: expanded from macro '__fp_type_select'
volatile float: f(x), \
^
This is because the controlling expression always undergoes lvalue
conversion first, dropping any cv-qualifiers. The 'const', 'volatile',
and 'volatile const' associations will therefore never be used.
MFC after: 1 week
Reviewed by: theraven
Differential Revision: https://reviews.freebsd.org/D35815
Create a wrapper for the mount system call. To ensure a sane early boot
environment and to gather data we need for kexec, we may need to mount
some special filesystems.
Sponsored by: Netflix
Early in boot, we need to create the normal stdin/out/err env for the
boot loader to run in. To do that, we need to open the console and
duplicate the file descriptors which requires dup(2). Implement a
wrapper as host_dup.
Sponsored by: Netflix
Linux's /dev/fd is implemented inside of /proc/self/fd, so we may need
to create a symlink to it early in boot. "/dev/fd" and "/dev/std*" might
not be strictly required for the boot loader, but should be present for
maximum flexibility.
Sponsored by: Netflix
Add host_getpid() so we can know if we're running as init(8) or not. If
we are, we may chose to do early system setup / sanity operations.
Sponsored by: Netflix
Implement stat(2) and fstat(2) in terms of newfstatat and newfstat
system calls respectively (assume we have a compat #define when
there's no newfstat and just a regular fstat and do so for ppc).
Snag struct kstat (the Linux kernel stat(2), et al interface) from musl
and attribute properly.
Sponsored by: Netflix
Add the common O_ constants for the open, fcntl, etc system calls. They
are different than FreeBSD's. While they can differ based on
architecture, they are constant for architectures we care about, and
those architectures use the 'generic' version so future architectures
will also work.
Sponsored by: Netflix
Fallback to currdev when NULL is passed in when 'rootdev' is NULL. Other
getdevs do this. Additional features are needed here still, though.
Sponsored by: Netflix
Split _start into _start and _start_c (inspired by musl and the powerpc
impl is copied from there). This allows us to actually get the command
line arguments on all the platforms. We have a very simplified startup
that supports only static linking.
Sponsored by: Netflix
If an error occurs while processing a TCP segment with some data and the FIN
flag, the back out of the sequence number advance does not take into account the
increase by 1 due to the FIN flag.
Reviewed By: jch, gnn, #transport, tuexen
Sponsored by: NetApp, Inc.
Differential Revision: https://reviews.freebsd.org/D2970
Silently ignore invalid set ' ' and invalid help help commands.
This is the same fix applied by NetBSD in hg commit 1019940:4f248823eaff.
PR: 265097
Reported by: Simon Josefsson <simon@josefsson.org>
Obtained from: NetBSD hg commit 1019940:4f248823eaff
NetBSD PR/56918
MFC after: 1 week
In ec81497cc7 crunchgen was updated to remove -dc from the linker
invocations in its generated makefile output, as this flag is no longer
necessary, and is going to be an error with lld 15.
Update the BOOTSTRAPPING conditions for copying the crunchgen binary
from the host, or actually bootstrapping it when necessary. Since
ec81497cc7 did not bump __FreeBSD_version, I have chosen the nearest
values.
Fixes: ec81497cc7
MFC after: 3 days
Before this patch CAM periph drivers called both disk_alloc() and
disk_create() same time on periph creation. But then prevented disks
from opening until the periph probe completion with cam_periph_hold().
As result, especially if disk misbehaves during the probe, GEOM event
thread, triggered to taste the disk, got blocked on open attempt,
potentially for a long time, unable to process other events.
This patch moves disk_create() call from periph creation to the end of
the probe. To allow disk_create() calls from non-sleepable CAM contexts
some of its duties requiring memory allocations are moved either back
to disk_alloc() or forward to g_disk_create(), so now disk_alloc() and
disk_add_alias() are the only disk methods that require sleeping. If
disk fails during the probe disk_create() may just be skipped, going
directly to disk_destroy(). Other method calls during that time are
just ignored. Since GEOM may now see the disks after CAM bus scan is
already completed, introduce per-periph boot hold functions. Enclosure
driver already had such mechanism, so just generalize it.
Reviewed by: imp
MFC after: 1 month
Sponsored by: iXsystems, Inc.
Differential Revision: https://reviews.freebsd.org/D35784
Leave one band of ithread priorities available below PI_SOFT for
demoted ithreads but reclaim additional ithread priorities for use by
user time-sharing threads. This is an ABI change in that PZERO moves
up so old ps and top binaries will not format priorities correctly on
newer kernels, but that is a cosmetic rather than functional change.
Reviewed by: kib, markj
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D35647
Allow high priority hardware interrupts to run at PI_REALTIME via
INTR_TYPE_CLK, but collapse all other hardware interrupt threads to
the next priority level (PI_INTR). Collapse all SWI priorities to
the same priority level (PI_SOFT) just below PI_INTR.
Reviewed by: kib, markj
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D35646
If an interrupt thread runs for a full quantum without yielding the
CPU, demote its priority and schedule a preemption to give other
ithreads a turn.
Reviewed by: kib
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D35645
If an interrupt thread runs for a full quantum without yielding the
CPU, demote its priority and schedule a preemption to give other
ithreads a turn.
Reviewed by: kib, markj
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D35644
Use sched_wakeup instead of sched_add when marking an ithread
runnable. This allows schedulers to reset their internal time slice
tracking state and restore the base ithread priority when an ithread
resumes from idle.
Reviewed by: markj
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D35643
Use it instead of sched_prio when setting the priority of an interrupt
thread.
Reviewed by: kib, markj
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D35642
bmake will not think that object files such as read.o are out of date
due to common.h changing since the dependency is only recorded in
.depend.kqtest.read.o in an old object directory.
Reviewed by: markj
Fixes: 68fe988a40 kqueue tests: Simplify the test runner
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D35820
Different fields in the tdq have different synchronization protocols.
Some are constant, some are accessed only while holding the tdq lock,
some are modified with the lock held but accessed without the lock, some
are accessed only on the tdq's CPU, and some are not synchronized by the
lock at all.
Convert ULE to stop using volatile and instead use atomic_load_* and
atomic_store_* to provide the desired semantics for lockless accesses.
This makes the intent of the code more explicit, gives more freedom to
the compiler when accesses do not need to be qualified, and lets KCSAN
intercept unlocked accesses.
Thus:
- Introduce macros to provide unlocked accessors for certain fields.
- Use atomic_load/store for all accesses of tdq_cpu_idle, which is not
synchronized by the mutex.
- Use atomic_load/store for accesses of the switch count, which is
updated by sched_clock().
- Add some comments to fields of struct tdq describing how accesses are
synchronized.
No functional change intended.
Reviewed by: mav, kib
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D35737
ULE's tdq_notify() tries to avoid delivering IPIs to the idle thread.
In particular, it tries to detect whether the idle thread is running.
There are two mechanisms for this:
- tdq_cpu_idle, an MI flag which is set prior to calling cpu_idle(). If
tdq_cpu_idle == 0, then no IPI is needed;
- idle_state, an x86-specific state flag which is updated after
cpu_idleclock() is called.
The implementation of the second mechanism is racy; the race can cause a
CPU to go to sleep with pending work. Specifically, cpu_idle_*() set
idle_state = STATE_SLEEPING, then check for pending work by loading the
tdq_load field of the CPU's runqueue. These operations can be reordered
so that the idle thread observes tdq_load == 0, and tdq_notify()
observes idle_state == STATE_RUNNING.
Some counters indicate that the idle_state check in tdq_notify()
frequently elides an IPI. So, fix the problem by inserting a fence
after the store to idle_state, immediately before idling the CPU.
PR: 264867
Reviewed by: mav, kib, jhb
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D35777
The load balancer executes from statclock and periodically tries to move
threads among CPUs in order to balance load. It may move a thread to
the current CPU (the loader balancer always runs on CPU 0). When it
does so, it may need to schedule preemption of the interrupted thread.
Use sched_setpreempt() to do so, same as sched_add().
PR: 264867
Reviewed by: mav, kib, jhb
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D35744
Thread switching used to be atomic with respect to the current CPU's
tdq lock. Since commit 686bcb5c14 that is no longer the case. Now
sched_switch() does this:
1. lock tdq (might already be locked)
2. maybe put the current thread in the tdq, choose a new thread to run
2a. update tdq_lowpri
3. unlock tdq
4. switch CPU context, update curthread
Some code paths in ULE will load pc_curthread from a remote CPU with
that CPU's tdq lock held, usually to inspect its priority. But, as of
the aforementioned commit this is racy.
The problem I noticed is in tdq_notify(), which optionally sends an IPI
to a remote CPU when a new thread is added to its runqueue. If the new
thread's priority is higher (lower) than the currently running thread's
priority, then we deliver an IPI. But inspecting
pc_curthread->td_priority doesn't work, since pc_curthread might be
between steps 3 and 4 above. If pc_curthread's priority is higher than
that of the newly added thread, but pc_curthread is switching to a
lower-priority thread, then tdq_notify() might fail to deliever an IPI,
leaving a high priority thread stuck on the runqueue for longer than it
should. This can cause multi-millisecond stalls in
interactive/ithread/realtime threads.
Fix this problem by modifying tdq_add() and tdq_move() to return the
value of tdq_lowpri before the addition of the new thread. This ensures
that tdq_notify() has the correct priority value to compare against.
The other two uses of pc_curthread are susceptible to the same race. To
fix the one in sched_rem()->tdq_setlowpri() we need to have an exact
value for curthread. Thus, introduce a new tdq_curthread field to the
tdq which gets updated any time a new thread is selected to run on the
CPU. Because this field is synchronized by the thread lock, its
priority reflects the correct lowpri value for the tdq.
PR: 264867
Fixes: 686bcb5c14 ("schedlock 4/4")
Reviewed by: mav, kib, jhb
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D35736
Older sysctls with constant OID macros were identified with those
in inet.4, tcp.4, and udp.4; newer sysctls with automatic numbering
were identified by sysctl names. No one remembers the OID macros,
or knows what they are; sysctls are always done by name now, usually
via sysctl(8).
Replace the OID macro names with sysctl names so that there is one
uniform identifier type; sysctl names were previously in parens.
Make the formatting a little more consistent in this area. In inet.4
and udp.4, move the "ip." or "udp." prefix from each entry into the
top-level name at the start of the section, as they are all the same.
Reviewed by: rpokala
Differential Revision: https://reviews.freebsd.org/D35806
with grab_faults, we can try to print out the trace of function calls.
Without symbol table, we can not translate addresses to function names,
but even addresses can help to track the bugs.
For loader functions, print out absolute address, so it could be
searched from objdump -d output.
Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D35433
On x86 systems, the debug.late_console tunable makes it possible to set
up the console before we call pmap_bootstrap. (The tunable is turned
on by default; setting late_console=0 results in consoles being probed
early.)
Unfortunately this is not compatible with using the ACPI SPCR table to
find the console, since consulting ACPI tables requires mapping memory
addresses. As such, we skip the call to uart_cpu_acpi_spcr from
uart_cpu_x86 in the !late_console case.
Reviewed by: imp
Sponsored by: https://www.patreon.com/cperciva
Differential Revision: https://reviews.freebsd.org/D35794
Currently for distributeworld we pass DESTDIR to certctl.sh as an
environment variable, which sets the default value in the script.
However, for -DNO_ROOT builds, CERTCTLFLAGS has METALOG_INSTALLFLAGS
which includes -D ${DESTDIR}, overriding the custom DESTDIR pointing at
the base dist directory.
Moreover, in order to ensure that the METALOG includes the base/ prefix
for all the files, we need to have certctl call install with -D set to
DESTDIR/DISTDIR without the /base suffix but also ensure the files get
installed to DESTDIR/DISTDIR/base.
Fix these by passing the custom DESTDIR to certctl via -D rather than in
the environment and to pass the /base suffix in the distributeworld case
via the newly-added -d option.
We also need to run certctl rehash before we generate the .meta files
from the METALOG, not after, otherwise they won't include the METALOG
additions, so move the certctl rehash call.
Finally, add a missing semicolon that results in no message being
printed in the missing openssl case. By not including the semicolon,
else echo "..." is treated as extra arguments to certctl, which is lax
in its argument parsing and ignores additional arguments, and the
semicolon and fi after the intended echo terminate the if statement as
normal so there's no syntax error at the shell level. This is harmless
as we weren't trying to do anything other than echo anyway, all that
happens is the echo doesn't actually get run.
Reported by: markj (missing semicolon)
Reviewed by: brooks, kevans
Obtained from: CheriBSD
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D35809
This will be used by Makefile.inc1 to fix -DNO_ROOT distributeworld,
which needs to split out DESTDIR from DISTBASE so the METALOG file
includes the base/ prefix.
Reviewed by: kevans
Obtained from: CheriBSD
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D35808
I mis-read the RFC w.r.t. handling of the sequenceid
when a CreateSession is done after the initial one
that confirms the ClientID. Fortunately this does
not affect most extant NFSv4.1/4.2 clients, since
they only acquire a single session for TCP for a
ClientID (Solaris might be an exception?).
This patch fixes the server to handle this case,
where the RFC requires the sequenceid be incremented
for each CreateSession and is required to reply to
a retried CreateSession with a cached reply.
It adds a field to nfsclient called lc_prevsess,
which caches the sessionid, which is the only field
in a CreateSession reply that will change for a
retry, to implement this reply cache.
The recent commits up to d4a11b3e3b that mark
session slots bad when "intr" and/or "soft" mounts
are used by the client needs this server patch.
Without this patch, the client will do a full
recovery, including a new ClientID, losing all
byte range locks. However, prior to the recent
client commits, the client would hang when all
session slots were bad, so even without this
patch it is not a regression.
PR: 260011
MFC after: 2 weeks
