Turns out there's two hidden a.out dependencies. pxeldr.S assumes it has
access to the a.out header from /boot/loader and cdboot.S assumes that
/boot/loader is also a.out and doesn't use boot2.
So, go back to making a.out files for these and adjust the size checks
to use ls, but we only need to check loader.bin. Trim the size we check
against by 2,000. The difference in size between loader and loader.bin
is about 3000 bytes, but clang15 produces binaries that are a smidge
bigger so we need to relax the check just a little and accept some
additional risk for the moment.
Add some comments to loader's Makefile about this.
Sponsored by: Netflix
Reviewed by: emaste
Differential Revision: https://reviews.freebsd.org/D36142
It's sometimes desirable to override the size limit: It's a soft limit
and there are times we exceed the limit by just a little bit and don't
want the build to fail (or we are hitting runtime failures below the
510,000 byte limit).
Sponsored by: Netflix
devformat produces the same output as i386_fmtdev, so just use it to
reduce on the dependencies.
Sponsored by: Netflix
Reviewed by: tsoome
Differential Revision: https://reviews.freebsd.org/D35927
devformat produces the same output as uboot_fmtdev, so just use it to
reduce on the dependencies.
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D35926
devformat produces the same output as userboot_fmtdev, so just use it to
reduce on the dependencies. In addition, we don't need to use the
incomplete struct userboot_devdesc type, we can use struct devdesc
instead (in fact, there's no userboot_devdesc defined anywhere).
Sponsored by: Netflix
Reviewed by: jhb (prior version)
Differential Revision: https://reviews.freebsd.org/D35925
devformat produces the same output as efi_fmtdev, so just use it to
reduce on the dependencies.
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D35924
Add a generic way to get the string representation of a zfs device / mount.
Sponsored by: Netflix
Reviewed by: tsoome (prior version)
Differential Revision: https://reviews.freebsd.org/D35923
Use devformat instead of disk_devfmt. This allows us to avoid knowing
the details of the device that's underneath us. Remove disk.h include
and the -I${LDRSRC} from the build of ufs.c since they are no longer
needed.
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D35922
Fix layering violation and use devformat to get the string
representation of the device to see if we're mounted yet or not. Remove
added include to pickup disk.h.
Sponsored by: Netflix
Reviewed by: tsoome (prior version)
Differential Revision: https://reviews.freebsd.org/D35919
All of the archsw fmtdev functions treat DEVT_DISK as a call to
disk_fmtdev. Set all disks' dv_fmtdev to disk_fmtdev so devformat
will return the same thing.
Sponsored by: Netflix
Reviewed by: tsoome (prior version)
Differential Revision: https://reviews.freebsd.org/D35917
Use dv_fmtdev to return a formatted string for a device. If this is a
null pointer, return the device name and unit followed by a colon (eg
disk3:).
Sponsored by: Netflix
Reviewed by: tsoome (prior version)
Differential Revision: https://reviews.freebsd.org/D35916
Add a new pointer, dv_devfmt, to allow devices to format themselves. We
will use this to simplify many of the fmtdev functions in the tree as
they are all almost the same, or all are isomorphic to each other.
Sponsored by: Netflix
Reviewed by: tsoome (prior version)
Differential Revision: https://reviews.freebsd.org/D35915
We do a number of games with ploymorphism for different types struct
*devdesc. Adjust one place that this affects to take the address of the
base class (most others have void * at the moment). This is more type
safe than a bare void *.
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D35914
Rather than have the magic, hand-crafted fields that have to align with
fields in other structures at the end of i386_devdesc, make it into
anonymous union and adjust the code accordingly. This is safer and
similar to what CAM does.
Sponsored by: Netflix
Reviewed by: kevans, tsoome (prior version)
Differential Revision: https://reviews.freebsd.org/D35965
The BIOS method of booting imposes an absolute limit of 640k for the
size of the program being run due to btx. In practice, this means that
programs larger than about 500kiB will fail in odd ways as the stack /
heap will overflow.
Pick 510,000 as the cutoff line semi-arbitrarily. loader_lua is now
almost too big and we want to break the build when it crosses this
threshold. In my experience, below 500,000 always works, above 520,000
always seems to fail with things getting bad somewhere between 512,000
to 515,000. 510,000 is as close to the line as I think we can go, though
experience may dictate we need to lower this in the future.
This is at-best a stop-breakage until we have a better way to subset the
boot loader for BIOS booting to allow better, more fined-tuned
/boot/loaders for the many different environments they have to run
in. This likely means we'll have a graphical loader than understands a
few filesystmes for installation, and a non-graphical loader that
understands the most filesystems possible for everything else in the
future. Our build infrastructure needs some work before we can do that,
however.
At this late date, it likely isn't worth the efforts to move parts of
the loader into high memory. There's a number of assumptions about where
the stack is, where buffers reside, etc that are fulfilled when it lives
in the first 640k that would need bounce buffers and/or other counter
measures if we were to split it up. All BIOS calls are done in 16-bit
mode with SEG:OFF addresses, requiring them to be in the first 640k of
RAM. And nearly all machines in the last decade can boot with UEFI
(though there's some exceptions, so it isn't worth killing outright
yet).
Sponsored by: Netflix
Reviewed by: kevans
Differential Revision: https://reviews.freebsd.org/D36129
The first level boot blocks have understood how to load ELF code since
1999. Switch /boot/loader and /boot/pxeldr over to being ELF format so
that in-tree tools can examine them more closely. In addition, one
could, in theory, now have a 'lo-mem' and a 'hi-mem' segment (though a
lot of work would need to be done with bounce buffers, btx, code segment
marking, etc for an arrangement like that to work).
As far as I can tell, this is the last a.out binary in the tree. There
are several raw binaries left, but everything else is ELF.
Reviewed by: emaste, kevans
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D36130
geliboot.c and geliboot_crypto.c don't need anything from stand/common,
so remove them from the list of things to add it.
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D35921
bootstrap.h isn't needed at all by geil, so remove it.
Sponsored by: Netflix
Reviewed by: tsoome (earlier version)
Differential Revision: https://reviews.freebsd.org/D35920
This code was written prior to snprintf being in the then libstand (now
libsa). Since we have it, use it for extra safety. The code already
tries to be safe, but since we have snprintf as well, the added layer of
protection will suffice. The current code reserves 16 bytes (plus a NUL)
at the end for worst case of inet_ntoa, which is still a little
pessimal, but safe from overflow.
Sponsored by: Netflix
Reviewed by: tsoome
Differential Revision: https://reviews.freebsd.org/D35102
Rather than trying to shoehorn flags into the requested superblock
address, create a separate flags parameter to the ffs_sbget()
function in sys/ufs/ffs/ffs_subr.c. The ffs_sbget() function is
used both in the kernel and in user-level utilities through export
to the sbget() function in the libufs(3) library (see sbget(3)
for details). The kernel uses ffs_sbget() when mounting UFS
filesystems, in the glabel(8) and gjournal(8) GEOM utilities,
and in the standalone library used when booting the system
from a UFS root filesystem.
The ffs_sbget() function reads the superblock located at the byte
offset specified by its sblockloc parameter. The value UFS_STDSB
may be specified for sblockloc to request that the standard
location for the superblock be read.
The two existing options are now flags:
UFS_NOHASHFAIL will note if the check hash is wrong but will still
return the superblock. This is used by the bootstrap code to
give the system a chance to come up so that fsck can be run to
correct the problem.
UFS_NOMSG indicates that superblock inconsistency error messages
should not be printed. It is used by programs like fsck that
want to print their own error message and programs like glabel(8)
that just want to know if a UFS filesystem exists on a partition.
One additional flag is added:
UFS_NOCSUM causes only the superblock itself to be returned, but does
not read in any auxiliary data structures like the cylinder group
summary information. It is used by clients like glabel(8) that
just want to check for possible filesystem types. Using UFS_NOCSUM
skips the superblock checks for csum data which allows superblocks
that have corrupted csum data to be read and used.
The validate_sblock() function checks that the superblock has not
been corrupted in a way that can crash or hang the system. Unless
the UFS_NOMSG flag is specified, it will print out any errors that
it finds. Prior to this commit, validate_sblock() returned as soon
as it found an inconsistency so would print at most one message.
It now does all its checks so when UFS_NOMSG has not been specified
will print out everything that it finds inconsistent.
Sponsored by: The FreeBSD Foundation
Add a shortcut for invokging ${SRCTOP}/tools/boot/universe.sh by
creating a 'universe' target in src/stand. This will make it easier to
test out all the different combinations of boot loaders that we build.
Sponsored by: Netflix
Since both EFI and the future kboot will benefit from a 'quit' command,
move it from efi/loader/main.c to common/commands.c. In EFI this command
exits back to the boot loader (which will cause the next BootXXXX in the
BootOrder list to be attempted). In kboot, this will exit back to
whatever called loader.kboot. In uboot this will cause a reset (which
will restart uboot, not quite a simple exit, but will look similar)
and in OFW it will execute OF_exit which should return to the
openfirmware prompt.
Sponsored by: Netflix
This is similar to 5d48fb3b16.
With LLVM14 the .data.rel.ro ELF section appears after .data,
making loader behave erractly and kernel is not loaded.
This patch makes ensures the correct order.
Based on discussion at:
https://github.com/llvm/llvm-project/issues/56306
MFC after: 1 day
Sponsored by: Instituto de Pesquisas Eldorado (eldorado.org.br)
Put the console into raw mode on startup. This allows the menus to work
as expected. Boot is now interruptable.
Note: Likely should restore the terminal settings on most exists. It's
not clear the best way to do this, and most shells have an auto stty
sane anyway, so note it for future improvement.
Sponsored by: Netflix
Implement a stripped down termios, obtained from various files in musl
and HOST_ or host_ prepended to most things and a few unavoidable style
tweaks. Only implements the bits of termios we need for the boot loader:
put the terminal into raw mode, restore terminal settings and speed
stuff.
Sponsored by: Netflix
Clients of libsa are expected to implement exit(). The current exit just
loops forever. It is better to really exit: when running as init that
will reboot the system. When not running as init, other programs can
recover (not that we support running as init, but when we do in the
future, this is still the rigtht thing).
Sponsored by: Netflix
Add support for aarch64. exec.c and ldscript are copied from the EFI
version with #ifdefs for the differences. Once complete, I'll refactor
them. host_syscall.S implements a generic system call. tramp.S is a
first attempt to create a tramoline that we can use to jump to the
aarch64 kernel. Add aarch64-specific startup and stat files as well.
exec.c tweaked slightly to avoid bringing in bi_load(), which will come
in later. Includes tweaks to stat due to name differences between names
on different Linux architectures.
Sponsored by: Netflix
conf.c is the same now between powerpc64 and amd64, so move it up to
kboot. Move powerpc file formats defines to ppc64_elf_freebsd.c
Sponsored by: Netflix
This was copied from powerpc/ofw and has never been used. We also don't
care about -DAIM. It's only relevant for in-kernel structures, which we
don't use in this userland program.
Sponsored by: Netflix
Linux 2.4 introduced getdents64. Switch to using it because aarch64
doesn't have getdents as that syscall was obsoleted before that port was
created.
Sponsored by: Netflix
tftp.c includes bootinfo.h, but doesn't need it. Remove it, and remove
the -Istand/common from CFALGS since that's the only reason we had it.
Sponsored by: Netflix
Reviewed by: tsoome
Differential Revision: https://reviews.freebsd.org/D35928
dv_cleanup is specified almost everywhere. Use nullsys instead of NULL
to indicate 'do nothing'. Also, be consistent in trailing commas that
were missing before.
Sponsored by: Netflix
Reviewed by: tsoome
Differential Revision: https://reviews.freebsd.org/D35913
While in theory, once upon a time, dv_type was arch specific, that's
never been the case in FreeBSD (and certaintly isn't in the surviving
drivers). Remove that notation.
Sponsored by: Netflix
Reviewed by: tsoome
Differential Revision: https://reviews.freebsd.org/D35911
Rather than hard coding 8 for the device name length, create a #define for it.
Sponsored by: Netflix
Reviewed by: tsoome
Differential Revision: https://reviews.freebsd.org/D35910
Fix a typo by reworking the comment for blake3_impl_hack.c and amplify
the nature of the hack and its temporary reason for existing.
Sponsored by: Netflix
Reviewed by: delphij
Differential Revision: https://reviews.freebsd.org/D35895
Now that the minor issues preventing zfs.c from using CFLAGS_EARLY have
been fixed, use that mechanism like everything else that needs the
OpenZFS spl headers. This simplifies things somewhat. Update comments to
document why zfs.c is still special, though in different ways.
Note: We also use the fact that NEED_SOLARIS_BOOLEAN is only defined in
an environment where the solaris compat boolean stuff will be defined
prior to this point (eg, when we're building zfs.c in libsa), but not in
other environments (like when we're building mkimage and stand-alone
boot loaders that don't use libsa). These latter uses should be changed
to use the same ZFS compile env, but aren't as part of this commit.
This has to be done in the same change as the ZFS_EARLY change to not
break zfs.c building for one commit affecting bisectabiltiy.
Sponsored by: Netflix
Reviewed by: tsoome, delphij
Differential Revision: https://reviews.freebsd.org/D35894
Constrain CFLAGS for ZFS: don't add anything globally. Add the includes
to only the files that need them. Add -DHAS_ZSTD_ZFS to zfs.c (which
includes zfsimpl.c which includes zfssubr.c both of which need this
defined). Also add it to efi/boot1/Makefile since zfs_module.c also
includes zfsimple.c.
Sponsored by: Netflix
Reviewed by: tsoome
Differential Revision: https://reviews.freebsd.org/D35887
Various GELI sources need bootstrap.h and disk.h. In theory they
shouldn't need anything outside of libsa, but disk.h and bootstrap.h are
currently required.
This fixes the build with MK_LOADER_ZFS=no.
Obtained from: CheriBSD
Fixes: eaf7aabddc stand: geli CFLAGS tightening
Sponsored by: DARPA
Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D35861
In theory they shouldn't need anything outside of libsa, but disk.h and
bootstrap.h are currently required. Future work wil address this issue.
This fixes the build with MK_LOADER_ZFS=no. ZFS's Makefile.inc adds
these flags globally to CFLAGS when it should not. This masked the
problem because the tools/boot/universe.sh didn't build MK_LOADER_ZFS=no
as part of its regressions. Future work will also fix this.
Obtained from: CheriBSD
Fixes: 84bf2bbbec stand: constrain zlib/gzip CFLAGS better
Sponsored by: DARPA
Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D35860
With clang 15, the following -Werror warning is produced:
stand/i386/boot2/boot2.c:358:6: error: a function declaration without a prototype is deprecated in all versions of C [-Werror,-Wstrict-prototypes]
This is because parse() is declared with a (void) argument list, and
defined with an empty argument list. Make the definition match the
declaration.
MFC after: 3 days
Create a wrapper for the mount system call. To ensure a sane early boot
environment and to gather data we need for kexec, we may need to mount
some special filesystems.
Sponsored by: Netflix
Early in boot, we need to create the normal stdin/out/err env for the
boot loader to run in. To do that, we need to open the console and
duplicate the file descriptors which requires dup(2). Implement a
wrapper as host_dup.
Sponsored by: Netflix
Linux's /dev/fd is implemented inside of /proc/self/fd, so we may need
to create a symlink to it early in boot. "/dev/fd" and "/dev/std*" might
not be strictly required for the boot loader, but should be present for
maximum flexibility.
Sponsored by: Netflix
Add host_getpid() so we can know if we're running as init(8) or not. If
we are, we may chose to do early system setup / sanity operations.
Sponsored by: Netflix
Implement stat(2) and fstat(2) in terms of newfstatat and newfstat
system calls respectively (assume we have a compat #define when
there's no newfstat and just a regular fstat and do so for ppc).
Snag struct kstat (the Linux kernel stat(2), et al interface) from musl
and attribute properly.
Sponsored by: Netflix
Add the common O_ constants for the open, fcntl, etc system calls. They
are different than FreeBSD's. While they can differ based on
architecture, they are constant for architectures we care about, and
those architectures use the 'generic' version so future architectures
will also work.
Sponsored by: Netflix
Fallback to currdev when NULL is passed in when 'rootdev' is NULL. Other
getdevs do this. Additional features are needed here still, though.
Sponsored by: Netflix
Split _start into _start and _start_c (inspired by musl and the powerpc
impl is copied from there). This allows us to actually get the command
line arguments on all the platforms. We have a very simplified startup
that supports only static linking.
Sponsored by: Netflix
with grab_faults, we can try to print out the trace of function calls.
Without symbol table, we can not translate addresses to function names,
but even addresses can help to track the bugs.
For loader functions, print out absolute address, so it could be
searched from objdump -d output.
Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D35433
Add comments describing the weird nesting things we have to do to live
inside the ZFS world. Also fix a stale comment by moving nvlist.c to an
early user.
Sponsored by: Netflix
Add the necessary glue to get blake3 building for the boot loaded as
well as connected to the ZFS system so it is useful.
On some platforms, we create references to blake3_sse2_impl and
blake3_sse41_impl ops structs to utilize SIMD. These aren't present on
x86 (since we dind't ask for them), but are on aarch64 with no
implementation. Since we don't want SIMD in the boot loader, have these
all return 'unsupported' always. This should be fixed upstream to allow
more flexibility in this selection, but for now we use this hack to not
modify the sys/contrib/openzfs with difficult to maintain hacks while
an upstreamable solution is found.
tsoome@ did the implementation bits in sys/cddl/boot, and I did the
Makefile work and the aweful blake3_impl_hack.c.
Co-author: tsoome@freebsd.org
Sponsored by: Netflix
Reviewed by: kevans (earlier version)
Differential Revision: https://reviews.freebsd.org/D35750
We no longer need to use this to get zfs_zstd.c compiling, so delete it.
Sponsored by: Netflix
Reviewed by: kevans
Differential Revision: https://reviews.freebsd.org/D35750
Now that we have the ability to work around all the issues that
zstd_stub.c worked around, compile zfs_std.c directly.
Sponsored by: Netflix
Reviewed by: kevans
Differential Revision: https://reviews.freebsd.org/D35750
sys/blake3.h is not safe to include in a standalone environment. It
assumes, unwisely, that there's only kernel or userland. The userland
choice is bad due to imperfections in how we handle stdlib.h in this
environment (we wind up including the host's stdlib.h, it expects a
standard FreeBSD setup which the gymnastics we've done to create a
OpenSolaris/Illumos-like environment interfere with). Thankfully, in
this case we can just pretend to be the kernel for a little bit by
defining _KERNEL and the undefing it after we exit.
Sponsored by: Netflix
Reviewed by: kevans
Differential Revision: https://reviews.freebsd.org/D35750
There's a number of issues with including zfs_context.h from a
standalone environment. First, sys/uio_imp.h isn't at all safe for this
environment, so define its guard #defines so that its contents are
skipped. Next, there's a problem including string.h to get the mem*
routines, so just define them here. ZFS_MODULE_PARAM_ARGS isn't defined
properly. I had wanted to define it when I was upstreaming changes to
include/os/freebsd/spl/sys/zfs_context.h, but they ran into resistance
so I'm defining that here now (it is also defined in zstd_shim.c, but
that will disappear once the issues it works around are
cleared). Finally, sys/sysmacros.h has to be included now before
sys/atomic.h, but upstream includes it after so include it here so that
the guards make the out-of-order includes in upstream irrelevant.
Sponsored by: Netflix
Reviewed by: kevans
Differential Revision: https://reviews.freebsd.org/D35750
OpenZFS is based on having a number of layers of include files that
define things and include the next layer. Insert a later at the start
for files in libsa/zfs/spl so we can override issues with upstream
OpenZFS while we work to get the issues corrected upstream.
Sponsored by: Netflix
Reviewed by: kevans
Differential Revision: https://reviews.freebsd.org/D35750
Add a note that veriexec / bearssl builds will pollute the CFLAGS in a
way that's somewhat hard to fix, so I'm just noting it for now.
Sponsored by: Netflix
Only add -DWEAK_REFS to sha256.c and sha512.c instead of
everything. Remove redundant include that's not needed.
Minor formatting tweak.
Sponsored by: Netflix
CFLAGS+= here affects *ALL* libsa files being built. However, this is
only needed for zfs.c, so define it only for this. Also, use the defines
from defs.mk. Move all the zfs.c include hacks together. Also, move the
-Wformat -Wall warnings that were added to CFLAGS+= to the individual
files instead for the same reason.
Sponsored by: Netflix
ZFSSRC is the top level directory where the ZFS sources come from. Don't
reuse it for a list of ZFS sources. Instead, use ZFS_SRC
Sponsored by: Netflix
Given that hints set this way don't work when a static kenv is compiled
into the kernel. acpi.rsdp has been set for this for the past 6 years,
and all kernels in that time have used it in preference to the hints. As
such, we no longer hints.*, so remove them.
Sponsored by: Netflix
Reviewed by: jhb
Differential Revision: https://reviews.freebsd.org/D35634
Several years ago, x86 moved from using hints to communicate this
information to using the simpler acpi.rsdp variables. If one compiles
static hints into the kernel, then these hints are ignored. We can
remove this when we branch FreeBSD 15. Thought about BURN_BRIDGES
here, but it's too messy.
Sponsored by: Netflix
Reviewed by: andrew, jhb
Differential Revision: https://reviews.freebsd.org/D35632
With the introduction of llvm14, the powerpc* loader used on ofw/pseries
for 32 and 64 bit architectures puts the .data.rel.ro section after
.data section. This caused a crash kernel didn't boot. Bisect pointed to
change https://reviews.llvm.org/D111717 but problem could be fixed by
adding a section description to make it appear in the expected order.
This patch is based on discussion at:
https://github.com/llvm/llvm-project/issues/56306
MFC after: 1 day
Sponsored by: Instituto de Pesquisas Eldorado (eldorado.org.br)
In some cases ubldr would try to mount a disk device before
the archsw struct was filled with functions pointers.
This would result in a NULL pointer derefrence of the arch_getdev field.
Fix that filling the archsw functions earlier.
Note that this matches the EFI behavoiur.
Reviewed by: imp, mw
Sponsored by: Stormshield
Obtained from: Semihalf
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D35670
geli_probe_and_attach() does pick geli_devsw structure for
encrypted disks, the implementation depends on device
name "disk" when device type is DEVT_DISK, but geli_devsw is
setting name field "gelidisk".
PR: 264282
Submitted by: yamagi@yamagi.org
Reported by: yamagi@yamagi.org
MFC after: 2 weeks
Detect the end of the disk condition. This may happpen when
disk image is truncated and the reads are addressing blocks past
image end.
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D35432
panic() is using multiple services - attempting to read
keyboard, accessing time functions and finally, exiting the loader.
Protect all the accessed listed above. Note, when BS are off,
we really can not just exit the loader, we only can reboot.
MFC after: 1 week
The check fails in kernels compiled with KASAN because AddressSanitizer
inserts redzones around global variables, so the size of the "kernphys"
symbol is 32 rather than 8. Thus we fall back to copying even though
it's not necessary.
Simply remove the size check. I didn't want to extend the symbol size
check since there's no guarantee that AddressSanitizer will always emit
32 bytes for "kernphys".
Reviewed by: kib
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D35448
Notable upstream pull request merges:
#12321 Fix inflated quiesce time caused by lwb_tx during zil_commit()
#13244 zstd early abort
#13360 Verify BPs as part of spa_load_verify_cb()
#13452 More speculative prefetcher improvements
#13466 Expose zpool guids through kstats
#13476 Refactor Log Size Limit
#13484 FreeBSD: libspl: Add locking around statfs globals
#13498 Cancel in-progress rebuilds when we finish removal
#13499 zed: Take no action on scrub/resilver checksum errors
#13513 Remove wrong assertion in log spacemap
Obtained from: OpenZFS
OpenZFS commit: b9d98453f9
When reading the UEFI memory map we pass in a pointer to the memory to
hold the map. Unfortunately it wasn't initialised before the first use
so clang decided it was undefined behaviour so the entire loop was
removed. This leads to everything in bi_load after this to also be
removed as dead code.
The next function after bi_load in the binary is efi_copy_init. The
above caused us to enter efi_copy_init with a return address of the
start of the function. Because of this it would enter an infinite
loop of calling the function, allocating memory, then returning to
the start of the function.
PR: 264021
head_errlog is new format for errlog, but we do not really
use errlog, so we can just mark it supported, to enable reading
from pool.
MFC after: 1 week
Move declarations into a new nvlist.h rather than putting everything in
libzfs.h. This makes this nvlist code easier to reuse elsewhere. In
particular, the nvlist implementation in sys/contrib/libnv does not
provide XDR encoding, but this is needed when reading from or writing to
ZFS pools.
Also:
- Remove references to boolean_t. It has to be a 32-bit int here, so
just reference the underlying type.
- Add includes needed when compiling the nvlist code outside of stand/.
No functional change intended.
Reviewed by: tsoome
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D35255
When adding an entry to an nvlist, the data buffer might need to be
resized. When this happens, the XDR encoder's notion of the buffer size
also needs to be updated, otherwise the operation may erroneously fail.
Reviewed by: tsoome, imp
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D35254
The loader.efi man page describes UEFI-specific loader details, but not
general loader information.
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
Get amd64 compiling. However, the current kboot supports an old way of
enumerating memory and the new way needs to be incorporated as well. The
powerpc folks could use either, it seems and newer powerpc platforms
need some changes for kboot to work anyway.
This commit includes the linker script, trampoline code to start the new
kernel, Linux system calls and the necessary configuration glue needed
to build the binaries.
This includes a quick hack to get multiboot support, but we need to
really share these defines. The multiiboot2.h is the minimum needed to
build. We have multiboot information in three places now, so a
refactoring is in order.
This should be considered, at best, preliminary and experimental for
anybody wishing to try it out.
Sponsored by: Netflix
Reviewed by: tsoome
Differential Revision: https://reviews.freebsd.org/D35100
Turns out there is a libsa.3. It's a bit out of date, but we reference
it in a number of places so we should install it. We need to do the DO32
dance because this Makefile is included twice and we don't want it
installing twice.
Sponsored by: Netflix
This mirrors dmu_read_impl(), zeroing out the tail end of the buffer and
clipping the read to what's contained by the block that exists.
This fixes an issue that arose during the 13.1 release process; in
13.1-RC1 and later, setting up GELI+ZFS will result in a failure to
boot. The culprit is this, which causes us to fail to load geom_eli.ko
as there's a residual portion after the single datablk that should be
zeroed out.
PR: 263407
Reviewed by: tsoome
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D35019
We don't need it for userboot, but it avoids issues with BIND_NOW, so
just provide it. time(3) isn't defined but ends up being provided by
libc linked into the host process, which is generally fine.
PR: 262920
Reviewed by: imp, jhb
MFC after: 3 days
Diferential Revision: https://reviews.freebsd.org/D34758
Printing device followed by interface matches, e.g., edk2. Note that
this is only a fallback, many firmware implementations will provide the
protocol that we'll use to format device paths.
Reviewed by: imp, tsoome
Sponsored by: Ampere Computing
Submitted by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D34539
- Add SPDX tag
- Add a files section describing different locations related to the
loader
- Add an example explaining how to install a new loader.efi to ESP
- Reference uefi(8)
Reviewed by: tsoome
Reviewed by: Pau Amma <pauamma@gundo.com>
Reviewed by: Jose Luis Duran <jlduran@gmail.com>
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D34592
The console escape code standard (ECMA-48) specifies color #3 (escape
code 33) as yellow. A brown console color is an artifact of the VGA
palette, which replaces dim (but not bright) yellow with brown.
Reviewed by: adrian, imp
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D34531
For historical reasons console color number 3 may be either yellow (most
consoles) or brown (VGA palette). The console escape code standard
uses "yellow", but teken color name constants appear to be based on the
VGA scheme and use TC_BROWN for color 3. Even so, the palette table
used 50,50,0 as the RGB percentage tuple, resulting in a dim yellow for
framebuffer consoles at the time teken was introduced.
Amusingly, in 19e2ce2d83 the comment on the palette entry was changed
from "brown" to "dark yellow" but the colour itself was changed from
a pure yellow to being somewhat brown.
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Accept "bright" or "light" prefix for named colors.
For numeric colors, update error message to specify that values 0 to 15
are allowed, and verify that values are in that range.
Reviewed by: imp, tsoome (both earlier version)
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D34512
Notable upstream pull request merges:
#9078: log xattr=sa create/remove/update to ZIL
#11919: Cross-platform xattr user namespace compatibility
#13014: Report dnodes with faulty bonuslen
#13016: FreeBSD: Fix zvol_cdev_open locking
#13019: spl: Don't check FreeBSD rwlocks for double initialization
#13027: Fix clearing set-uid and set-gid bits on a file when
replying a write
#13031: Add enumerated vdev names to 'zpool iostat -v' and
'zpool list -v'
#13074: Enable encrypted raw sending to pools with greater ashift
#13076: Receive checks should allow unencrypted child datasets
#13098: Avoid dirtying the final TXGs when exporting a pool
#13172: Fix ENOSPC when unlinking multiple files from full pool
Obtained from: OpenZFS
OpenZFS commit: a86e089415
If module_verbose is set to a value below MODULE_VERBOSE_TWIDDLE
call twiddle_divisor(UINT_MAX).
This makes more sense here than when we are loading the kernel.
Sponsored by: Juniper Networks, Inc.
UEFI provides a protocol for accessing randomness. This is a good way
to gather early entropy, especially when there's no driver for the RNG
on the platform (as is the case on the Marvell Armada8k (MACCHIATObin)
for now).
If the entropy_efi_seed option is enabled in loader.conf (default: YES)
obtain 2048 bytes of entropy from UEFI and pass is to the kernel as a
"module" of name "efi_rng_seed" and type "boot_entropy_platform"; if
present, ingest it into the kernel RNG.
Submitted by: Greg V
Reviewed by: markm, kevans
Approved by: csprng (markm)
MFC after: 3 weeks
Differential Revision: https://reviews.freebsd.org/D20780
Set module_verbose to control the printing of information
about loaded modules and kernel:
0 MODULE_VERBOSE_SILENT None
1 MODULE_VERBOSE_SIZE Pathname and size
2 MODULE_VERBOSE_TWIDDLE as for 1 but also twiddle for progress
3 MODULE_VERBOSE_FULL extra detail
When the loader is verifying modules we already have a
running indication of progress and module_verbose=0 makes sense.
Reviewed by: rpokala
Differential Revision: https://reviews.freebsd.org/D34245
ld.bfd in binutils 2.34+ now reports an error in more cases for custom
ldscripts that do not place PHDRs in a LOAD segment. However, EFI
binaries are not dynamic binaries which need PHDRs, so pass
--no-dynamic-linker to disable this check.
Reviewed by: imp, emaste
Differential Revision: https://reviews.freebsd.org/D34124
The text after .error et al is emitted verbatim.
Reviewed by: sjg
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D33904
- Use AES_XTS_IV_LEN instead of the key length as the IV length.
- Use G_ELI_IVKEYLEN as the size of the zeroed iv[] array in
g_eli_crypto_cipher() to match geli_io().
PR: 261172
Reported by: Malcolm Matalka <mmatalka@gmail.com>, mikael
Reviewed by: markj
Sponsored by: FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D33884
This helps mesuring what's happening when we load the kernel/modules/mfsroot.
This also adds TSENTER2 which uses the third argument of TSRAW, same
as in the kernel.
MFC after: 2 weeks
Sponsored by: Beckhoff Automation GmbH & Co. KG
Differential Revision: https://reviews.freebsd.org/D33699
The saf1761 OTG support was only for mips targets (BERI?). Retire it.
Sponsored by: Netflix
Reviewed by: brooks
Differential Revision: https://reviews.freebsd.org/D33706
Shortlinks occupy the space of both di_db and di_ib when used. However,
everywhere that wants to read or write a shortlink takes a pointer do
di_db and promptly runs off the end of it into di_ib. This is fine on
most architectures, if a little dodgy. However, on CHERI, the compiler
can optionally restrict the bounds on pointers to subobjects to just
that subobject, in order to mitigate intra-object buffer overflows, and
this is enabled in CheriBSD's pure-capability kernels.
Instead, clean this up by inserting a union such that a new di_shortlink
can be added with the right size and element type, avoiding the need to
cast and allowing the use of the DIP macro to access the field. This
also mirrors how the ext2fs code implements extents support, with the
exact same structure other than having a uint32_t i_data[] instead of a
char di_shortlink[].
Reviewed by: mckusick, jhb
Differential Revision: https://reviews.freebsd.org/D33650
Just have the MD code provide syscall and have generic code for the
rest.
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D33515
Split the powerpc specific parts into kboot/arch/powerpc64. More may be
needed here.
Sponsored by: Netflix
Reviewed by: jhibbits
Differential Revision: https://reviews.freebsd.org/D33514
As the first step at making this more generic, move kboot to top level.
Sponsored by: Netflix
Reviewed by: luporl, tsoome
Differential Revision: https://reviews.freebsd.org/D33513
tftp_open reads the first block so copy it in the cached data.
If we have more than one block (i.e. we called tftp_read before
tftp_preload) simply just reset the transfer.
Reported by: mmel
Reviewed by: mmel, tsoome
MFC after: 2 weeks
Sponsored by: Beckhoff Automation GmbH & Co. KG
Differential Revision: https://reviews.freebsd.org/D33697
shadow FB size could be rather large and depends on resolution,
instead of using heap, allocate dedicated space outside of heap.
Reviewed by: manu
Differential Revision: https://reviews.freebsd.org/D33600
MFC after: 2 weeks
On boot we don't need to perform any CPU cache management when the IDC
and DIC fields in the ctr_el0 register are set. Add a command to tell
loader to ignore these fields. This could be useful, for example, if the
hardware is misreporting the values and we are missing a quirk to enable
it.
It is not expected this will be needed, but is only intended as a
workaround to ensure the kernel can still boot.
Sponsored by: The FreeBSD Foundation
In the arm64 loader we need to syncronise the I and D caches. On some
newer CPUs the I and D caches are coherent so we don't need to perform
these operations.
While here remove the arguments to cpu_inval_icache as they are unneeded.
Reported by: cperciva
Tested by: cperciva
Sponsored by: Innovate UK
Kernel needs physical frame buffer address and size, which Block
Transfer-only Graphics Output Protocol instances do not have.
Some recent ASUS boards like PRIME Z690M-PLUS D4 and PRIME H570-Plus
report two GOPs, out of which the second one support ConOut protocol,
that made it preferable, but is BLT-only, that made console unusable.
Discussed with: tsoome (previous version)
MFC after: 1 week
Otherwise on fs like tftp where no directory listing is possible we fail
on the .dir method.
Reviewed by: imp, kevans
MFC after: 2 weeks
Sponsored by: Beckhoff Automation GmbH & Co. KG
Differential Revision: https://reviews.freebsd.org/D33414
Use %j to it works on 64 and 32 bits system.
Reviewed by: imp, tsoome
MFC after: 2 weeks
Sponsored by: Beckhoff Automation GmbH & Co. KG
Differential Revision: https://reviews.freebsd.org/D33430
The preload method will transfer the whole file in a buffer and cache it
so read/lseek operations are faster.
Reviewed by: imp, tsoome
MFC after: 2 weeks
Sponsored by: Beckhoff Automation GmbH & Co. KG
Differential Revision: https://reviews.freebsd.org/D33410
When we load an ELF file (kernel or module) we do seek(2) a lot to
parse/load the different sections of the ELF file.
Protocol like TFTP suffers a lot from this as there is no resume or
a way to start the tranfer from a specified offset in the file.
fs_preload is added to help those protocol.
Call preload just after opening the ELF file that we need to load so
the underlying method can cache the hole file and then read/lseek operations
are faster.
Reviewed by: imp
MFC after: 2 weeks
Sponsored by: Beckhoff Automation GmbH & Co. KG
Differential Revision: https://reviews.freebsd.org/D33409
getsecs is very costly, reuse the values we got before.
Fetching a ~30MB kernel with the tftp command use to take ~26 seconds
and now it's ~18 seconds.
Reviewed by: imp, tsoome
MFC after: 2 weeks
Sponsored by: Beckhoff Automation GmbH & Co. KG
Differential Revision: https://reviews.freebsd.org/D33408
When we load a kernel or module we open/close it a few times.
Since we're using the same port number each time and that we requested
the same file the ACK that we send are valid on the server side and the
server send us the file multiple times.
This makes tftp loading time very inconsistant due to the UDP "flood" that
we have to process.
Reviewed by: imp, tsoome
MFC after: 2 weeks
Sponsored by: Beckhoff Automation GmbH & Co. KG
Differential Revision: https://reviews.freebsd.org/D33407
readtest will simply load the file in memory, useful for timing
loading on some filesystems.
Reviewed by: tsoome
MFC after: 2 weeks
Sponsored by: Beckhoff Automation GmbH & Co. KG
Differential Revision: https://reviews.freebsd.org/D33411
tftp-hpa sends NAK with tftp error set to 0 when trying to get
a directory and this is the first thing that loader tries to do
and this make it hangs.
Reviewed by: imp, tsoome
MFC after: 2 weeks
Sponsored by: Beckhoff Automation GmbH & Co. KG
Differential Revision: https://reviews.freebsd.org/D33406
We never use 'dev' after fetching it from the varargs list. Skip the
whole bother of fetching it, or setting up the meachinery to fetch it.
Sponsored by: Netflix
Build uboot ubldr and friends like we build efi binaries
o move everything to be under stand/uboot
o md code goes in arch/$ARCH
o move everything over from the library
- Had to rename console.c, disk.c and module.c due to conflicts
o update version to 1.5 to reflect the new way of building
This results in a more consistent build system and should represent no
functional change, apart from powerpc version getting new help
file. Also, moved to exlcuding uboot on powerpc64le by using
BROKEN_OPTION instead of the incidental exclusion we had before due to
Makefile reorgs.
Sponsored by: Netflix
Feedback by: stevek, jrtc27
Differential Revision: https://reviews.freebsd.org/D33362
All these files are the same, modulo one comment. Move them all into
common/gfx_fb_stub.c and adjust Makefiles accordingly.
Sponsored by: Netflix
Reviewed by: manu
Differential Revision: https://reviews.freebsd.org/D33428
Store whether or not we found a vbefb module (eg, a tg supported kernel)
in the preloaded_file structure. This automatically resets on reload and
eliminates load_elf knowing about any gfx_* interface. Restrict this to
i386, which is the only place it's used. Update libi386 to check in the
preloaded_file struct. Eliminate this from the teken_gfx
structure. Rewrite the parsing code to be more inline. Check this from
the same place we check for a relocatable amd64 kernel.
Sponsored by: Netflix
Reviewed by: manu, tsoome
Differential Revision: https://reviews.freebsd.org/D33427
Reading from Write Combining memory can be very-very slow. Try to use
shadow buffer to avoid such reads.
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D33305
As part of decommissioning mips support, remove the boot loader
support. Do this in advance of other boot loader work to limit the
amount of work that will be thrown away.
Sponsored by: Netflix
Reviewed by: jrtc27
Differential Revision: https://reviews.freebsd.org/D33377
The elf loader needs to know how to reach into the gfx_fb code, so
simplify how we include files to find that stuff.
Sponsored by: Netflix
Reviewed by: manu, tsoome
Differential Revision: https://reviews.freebsd.org/D33376
Move direct call of ExitBootServices to efi_exit_boot_services. This
function sets boot_services_active to false so callers don't have to do
it everywhere (though currently only loader/bootinfo.c is affected).
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D32226
This makes GOP not probed on some situation (AMD Card on PCIe slot
with EDK2 as we have a SERIAL_IO_PROTOCOL compatible uart).
Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D32992
Sponsored by: Beckhoff Automation GmbH & Co. KG
Printing the EFI_HANDLE pointer isn't very useful.
If the handle have a IMAGE_DEVICE_PATH or a DEVICE_PATH protocol print it.
This makes it easier to see which devices are present and what protocol they
expose.
Reviewed by: imp, tsoome
Differential Revision: https://reviews.freebsd.org/D32991
Sponsored by: Beckhoff Automation GmbH & Co. KG
The STDSB macro is passed to the ffs_sbget() routine to fetch a
UFS/FFS superblock "from the stadard place". It was identically defined
in lib/libufs/libufs.h, stand/libsa/ufs.c, sys/ufs/ffs/ffs_extern.h,
and sys/ufs/ffs/ffs_subr.c. Delete it from these four files and
define it instead in sys/ufs/ffs/fs.h. All existing uses of this macro
already include sys/ufs/ffs/fs.h so no include changes need to be made.
No functional change intended.
Sponsored by: Netflix
Add ofwbus, iicbus and spibus to pnpautoload so modules under those
buses will be loaded.
On my rockpro64 now :
OK pnpautoload -v
Autoloading modules for simplebus
Using DTB provided by EFI at 0x8100000.
Autoloading modules for ofwbus
/boot/kernel/rk_spi.ko text=0x14b2 text=0xd4c data=0x4d0+0x8 syms=[0x8+0xa98+0x8+0x807]
/boot/kernel/dwwdt.ko text=0x12e2 text=0x78c data=0x4c8+0x10 syms=[0x8+0x990+0x8+0x6e1]
Autoloading modules for iicbus
Autoloading modules for spibus
/boot/kernel/mx25l.ko text=0x1613 text=0x114c data=0x6e8+0x8 syms=[0x8+0xa08+0x8+0x665]
loading required module 'fdt_slicer'
/boot/kernel/fdt_slicer.ko text=0x95e text=0x340 data=0x290 syms=[0x8+0x6c0+0x8+0x4a0]
Turn the presence or absence of boot services into a positive bool (and
change its type to bool). Move declaration to efi.h in the global
variables section.
Sponsored by: Netflix
Reviewed by: tsoome, kib
Differential Revision: https://reviews.freebsd.org/D31814
When the timer drops from double to single digits, a spare 'e' is left
on the end of the line as we don't overwrite it. Include an extra space
at the end to account for this and overwrite the leftover character.
PR: 259429
MFC after: 3 days
Reviewed by: emaste
If CAS detects that radix is not supported, set radix_mmu to 0
to avoid the kernel trying to use it and panic.
MFC after: 2 weeks
Sponsored by: Instituto de Pesquisas Eldorado (eldorado.org.br)
Fix boot1 and loader on PowerPC64 little-endian (LE).
Due to endian issues, boot1 couldn't find the UFS boot partition
and loader wasn't able to load the kernel. Most of the issues
happened because boot1 and loader were BE binaries trying to access
LE UFS partitions and because loader expects the kernel ELF image
to use the same endian as itself.
To fix these issues, boot1 and loader are now built as LE binaries
on PPC64LE. To support this, the functions that call OpenFirmware
were enhanced to correctly perform endian conversion on its input
and output arguments and to change the CPU into BE mode before
making the calls, as OpenFirmware always runs in BE. Besides that,
some other small fixes were needed.
Submitted by: bdragon (initial version)
Reviewed by: alfredo, jhibbits
Sponsored by: Instituto de Pesquisas Eldorado (eldorado.org.br)
Differential Revision: https://reviews.freebsd.org/D32160
net_open() does replace f_devdata with pointer to netdev_sock,
this will cause memory leak when device is closed, but also does
alter the devopen() logic.
We should store &netdev_sock to dev->d_opendata instead, this
would preserve and follow the devopen() logic.
Fixes network boot on aarch64 (tested by bz).
Reviewed-by: imp
MFC After: 2 weeks
Differential Revision: https://reviews.freebsd.org/D32227
As Radix MMU with superpages enabled is now stable, make it the
default choice on supported hardware (POWER9 and above), since its
performance is greater than that of HPT MMU.
Reviewed by: alfredo, jhibbits
Sponsored by: Instituto de Pesquisas Eldorado (eldorado.org.br)
Differential Revision: https://reviews.freebsd.org/D30797
Pass the ivlen along through, and just drop this KASSERT() if we're
building _STANDALONE for the time being.
Fixes: 1833d6042c ("crypto: Permit variable-sized IVs ...")
Prior to this commit, the loader would perform readaheads of up to
128 kB; when booting on a UFS filesystem this resulted in a series
of 160 kB reads (32 kB request + 128 kB readahead).
This commit allows readaheads to be longer, subject to a total I/O
size limit of 256 kB; i.e. 32 kB read requests will have added
readaheads of up to 224 kB.
In my testing on an EC2 c5.xlarge instance, this change reduces the
boot time by roughly 80 ms.
Reviewed by: tsoome
MFC after: 1 week
Sponsored by: https://www.patreon.com/cperciva
Differential Revision: https://reviews.freebsd.org/D32251
The loader bcache attempts to determine whether readahead is useful,
increasing or decreasing its readahead length based on whether a
read could be serviced out of the cache. This resulted in two
unfortunate behaviours:
1. A series of consecutive 32 kB reads are requested and bcache
performs 16 kB readaheads. For each read, bcache determines that,
since only the first 16 kB is already in the cache, the readahead
was not useful, and keeps the readahead at the minimum (16 kB) level.
2. A series of consecutive 32 kB reads are requested and bcache
starts with a 32 kB readahead resulting in a 64 kB being read on
the first request. The second 32 kB request can be serviced out of
the cache, and bcache responds by doubling its readahead length to
64 kB. The third 32 kB request cannot be serviced out of the cache,
and bcache reduces its readahead length back down to 32 kB.
The first syndrome converts a series of 32 kB reads into a series of
(misaligned) 32 kB reads, while the second syndrome converts a series
of 32 kB reads into a series of 64 kB reads; in both cases we do not
increase the readahead length to its limit (currently 128 kB) no
matter how many consecutive read requests are made.
This change avoids this problem by tracking the "unconsumed
readahead" length; readahead is deemed to be useful (and the
read-ahead length is potentially increased) not only if a request was
completely serviced out of the cache, but also if *any* of the request
was serviced out of the cache and that length matches the amount of
unconsumed readahead. Conversely, we now only reduce the readahead
length in cases where there was unconsumed readahead data.
In my testing on an EC2 c5.xlarge instance, this change reduces the
boot time by roughly 120 ms.
Reviewed by: imp, tsoome
MFC after: 1 week
Sponsored by: https://patreon.com/cperciva
Differential Revision: https://reviews.freebsd.org/D32250
Booting on an EC2 c5.xlarge instance, this reduces the number of I/Os
performed from 609 to 432, reduces the total number of blocks read
from 61963 to 60797, and reduces the time spent in the loader by 39 ms.
Note that b4cb3fe0e3 allowed the bcache to be retained for most of
the boot process, but relies on mounting filesystems; this commit
allows the bcache to be retained at the start of the boot process,
before the root filesystem has been located.
Reviewed by: imp, tsoome
MFC after: 1 week
Sponsored by: https://www.patreon.com/cperciva
Differential Revision: https://reviews.freebsd.org/D32239
The definition for 'ST' is in efilib.h, so we don't need extern ST here.
Sponsored by: Netflix
Reviewed by: tsoome, kevans
Differential Revision: https://reviews.freebsd.org/D32225
Lua bindings appeared in FreeBSD 12.0. Delete the authors section of the
man page, since it's unclear who wrote different parts of the man
page.
Noted by: Trond Endrestol
Sponsored by: Netflix
Create a man page per loader. Loader(8) will have information common to
all of them, while loader_${INTERP}(8) will have information relevant to
that specific loader. Rewrite loader(8) to give an overview and point to
the appropriate man page. Rewrite each of the loader_${INTER}(8) man
pages to contain only the relevant information to that loader. Put all
the common commands, environment variables, etc in loader_simp(8) and
refernce that from the loader_lua or loader_4th man pages. The
loader_lua(8) could use more details about the Lua
integration. Additional organization may be benefitial.
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D31340
Booting FreeBSD on an EC2 c5.xlarge instance, the loader "twiddles"
810 times over the course of 510 ms, a rate of 1.59 kHz. Even accepting
that many systems are slower than this particular VM and will take
longer to boot (especially if using spinning-rust disks), this seems
like an unhelpfully large amount of twiddling when compared to the
~60 Hz frame rate of many displays; printing the twiddles also consumes
roughly 10% of the boot time on the aforementioned VM.
Setting the default globaldiv to 16 dramatically reduces the time spent
printing twiddles to the console while still twiddling at roughly 100
Hz; this should be ample even for systems which take longer to boot and
consequently twiddle slower.
Note that this can adjusted via the twiddle_divisor variable in
loader.conf, but that file is not processed until nearly halfway
through the loader's runtime.
Reviewed by: allanjude, jrtc27, kevans
MFC after: 1 week
Sponsored by: https://www.patreon.com/cperciva
Differential Revision: <https://reviews.freebsd.org/D32163>
Now that the loader tslog code doesn't call printf, we can profile
printf using TSLOG. On an EC2 c5.xlarge instance, we spend roughly
45 ms here (out of roughly 500 ms), presumably due to the time spent
writing output to the console.
MFC after: 1 week
Sponsored by: https://www.patreon.com/cperciva
After b4cb3fe0e3, loader started crashing on PowerPC64, with a
Program Exception (700) error. The problem was that archsw was
used before being initialized, with the new mount feature. This
change fixes the issue by initializing archsw earlier, before
setting currdev, that triggers the mount.
Reviewed by: tsoome
MFC after: 1 month
X-MFC-With: b4cb3fe0e3
Sponsored by: Instituto de Pesquisas Eldorado (eldorado.org.br)
Differential Revision: https://reviews.freebsd.org/D32027
Use radix_mmu environment variable to select between Hash or Radix
MMU, when performing the CAS method call. This matches kernel's
behavior, by selecting Hash MMU by default and Radix if radix_mmu
is not zero, to make sure that both loader and kernel always select
the same MMU.
The device tree is queried to detect Radix/GTSE support and to
find out if CAS is supported, making the old CPU version and HV
bit checks unnecessary now.
Reviewed by: jhibbits
MFC after: 2 weeks
Sponsored by: Instituto de Pesquisas Eldorado (eldorado.org.br)
Differential Revision: https://reviews.freebsd.org/D31951
The behavior remains the same, but lualoader now uses the more concise
verbiage that forthloader used. This is particularly important because
the previous line would exceed the right boundary of the menu and run
straight into space that would typically be allowed for the logo.
This makes it slightly easier to port logos from forthloader to
lualoader.
We want to keep our root file system open to preserve bcache segment
between file accesses, thus reducing physical disk IO.
Reviewed by: imp, allanjude, kevans (previous version)
Differential Revision: https://reviews.freebsd.org/D30848
MFC after: 1 month
Because lld 13 and higher default to garbage collecting start/stop
symbols when using --gc-sections, the linker sets used in the i386 boot
loaders will disappear. This leads to the loaders not recognizing any
commands, and failure to boot.
Until we have a good set of linker scripts for the loaders, work around
it by disabling the start-stop-gc feature.
MFC after: 1 week
When Boot Services (BS) are switched off, we can not use BS
functions any more. Since drawn console does implement our own
Blt(), we can use it to draw the console.
However, SimpleTextOutput protocol based console output must be
blocked.
Tested by inserting printf() after ExitBootServices() call.
MFC after: 1 week
We need to round it up to 2M, for instance. Having staging area too small
might cause the first resize to use negative size for memmove()/memcpy(),
which kills loader.
Tested by: Harry Schmalzbauer <freebsd@omnilan.de>
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
The definition can be overridden by users, and before f75caed644 it
was in MBs. Make the symbol' unit MB, to be compatible with users
customizations.
Reported and tested by: Harry Schmalzbauer <freebsd@omnilan.de>
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
This pushes the bulk of the rx servicing into a single loop that's only
slightly convoluted, and it addresses a problem with rx handling in the
process. If we hit a tx interrupt while we're processing, we'd
previously drop the frame on the floor completely and ultimately
timeout, increasing boot time on particularly busy hosts as we keep
having to backoff and resend.
After this patch, we don't seem to hit timeouts at all on zoo anymore
though loading a 27M kernel is still relatively slow (~1m20s).
Reviewed by: tsoome
Triage by: Ash Gokhale <ashfixit gmail com>
Sponsored By: National Bureau of Economic Research
Sponsored by: Klara, Inc.
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D31512
When we quit pager, the return value 1 is returned and command_more()
interprets it as error.
when lua loader gets error from command, it will try to
interpret it once more, so we get the same file shown once more.
There is no reason why we should return error from command_more().
MFC after: 1 week
Scrolling screen will leave "trail" of chars from first column.
Apparently caused by cursor location mismanagement.
Make sure we do not [attempt to] set cursor out of the screen.
MFC after: 1 week
The change makes block caching algorithm to work better for remote
media on low-BW/high-delay links.
This cuts boot time over IP KVMs noticeably, since the initialization
stage reads bunch of small 4th (and now lua) files that are not in
the same cache stripe (usually), thus wasting lot of bandwidth and
increasing latency even further.
The original regression came in 2017 with revision 87ed2b7f5. We've
seen increase of time it takes for the loader to get to the kernel
loading from under a minute to 10-15 minutes in many cases.
Reviewed by: tsoome
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D31623
The Xen kernel has no symbol tables, so calling lookup_symbol against
it triggers the following Divide by Zero fault:
Loading Xen kernel...
/boot/xen data=0x2809c8+0x149638 |
!!!! X64 Exception Type - 00(#DE - Divide Error) CPU Apic ID - 00000000 !!!!
Fix lookup_symbol to prevent the #DE fault from happening if the
symbol table is not loaded and also fix loadfile_raw to mark multiboot
kernels as relocatable, since the only multiboot kernel supported is
Xen and was already unconditionally booted as relocatable.
Fixes: f75caed644 ('amd64 UEFI loader: stop copying staging area to 2M physical')
Reviewed by: imp, kib
Differential Revision: https://reviews.freebsd.org/D31507
Summary:
Open file list is currently created as statically allocated array (64 items).
Once this array is filled up, loader will not be able to operate with files.
In most cases, this mechanism is good enough, but the problem appears, when
we have many disks with zfs pool(s). In current loader implementation, all
discovered zfs pool configurations are kept in memory and disk devices open -
consuming the open file array. Rewrite the open file mechanism to use
dynamically allocated list.
Reviewed by: imp
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D31364
There's no need to build both pie and non-pie .o's for stand. There's
some other build thing with MK_BEAR_SSL=yes and/or MK_LOADER_VERIEXEC=yes
that causes the pie build to fail that the 'ar' stage now. Since we don't
need the PIE stuff and the non-PIE stuff, disable PIE for the boot loader.
Reviewed by: emaste
Sponsored by: Netflix
On amd64, add a possibility to activate kernel with staging area in place.
Add 'copy_staging' command to control this. For now, by default the
old mode of copying kernel to 2M phys is retained. It is going to be
changed in several weeks.
On amd64, add some slop to the staging area to satisfy both requirements
of the kernel startup allocator, and to have space for minor staging data
increase after the final size is calculated. Add a new command
'staging_slop' to control its size.
Improve staging area resizing, in particular, reallocate it anew if
we cannot grow it neither down nor up.
Reviewed by: kevans, markj
Discussed with: emaste (the delivery plan)
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D31121
It is more idiomatic. CFLAGS is only augmented with $SSP_CFLAGS when
$MK_SSP != "no".
Reviewed by: imp
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D31401
The userboot/test program links against the default userspace libraries
(e.g. shared libgcc_s.so) that will be instrumented if WITH_ASAN is set.
All other programs link against libsa instead of libc and therefore can't
use the sanitizer runtime library. To fix the stand/ build with
sanitizers, we disable MK_ASAN/MK_UBSAN if -nostdlib is found in the
LDFLAGS (i.e. we are using libsa instead of libc).
Reviewed By: imp, tsoome
Differential Revision: https://reviews.freebsd.org/D31047
servip is set from bootp bp_siaddr (if present) and rootip is
set immediately from servip in tha sane bootp code.
However, the common/dev_net.c does only set rootip (based on
url processing etc). Therefore, we should also use rootip in tftp
reader.
Fixes hung tftp based boot when bp_siaddr is not provided.
MFC after: 1 week
disable-device fooX will set hint.foo.X.disabled=1 as a way to easily
disable a device attaching during boot.
Reviewed by: tsoome
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D31297
Large nextboot.conf files (over 80 bytes) are not read correctly by the
Forth loader, causing file parsing to abort, and nextboot configuration
fails to apply.
Simple repro:
nextboot -e foo=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
shutdown -r now
That will cause the bug to cause a parse failure but shouldn't otherwise
affect the boot. Depending on your loader configuration, you may also
have to set beastie_disable and/or reduce the number of modules loaded
to see the error on a small console screen. 12.0 or CURRENT users will
also have to explicitly use the Forth loader instead of the Lua loader.
The error will look something like:
Warning: syntax error on file /boot/loader.conf.local
foo="xxxxxxxxxxxxxxnextboot_enable="YES"
^
/boot/support.4th has crude file I/O buffering, which uses a buffer
'read_buffer', defined to be 80 bytes by the 'read_buffer_size'
constant. The loader first tastes nextboot.conf, reading and parsing
the first line in it for nextboot_enable="YES". If this is true, then
it reopens the file and parses it like other loader .conf files.
Unfortunately, the file I/O buffering code does not fully reset the
buffer state in the reset_line_reading word. If the last file was read
to the end, that doesn't matter; the file buffer is treated as empty
anyway. But in the nextboot.conf case, the loader will not read to the
end of file if it is over 80 bytes, and the file buffer may be reused
when reading the next file. When the file is reread, the corrupt text
may cause file parsing to abort on bad syntax (if the corrupt line has
<>2 quotes in it), the wrong variable to be set, no variable to be set
at all, or (if the splice happens to land at a line ending) something
approximating normal operation.
The bug is very old, dating back to at least 2000 if not before, and is
still present in 12.0 and CURRENT r345863 (though it is now hidden by
the Lua loader by default).
Suggested one-line attached. This does change the behavior of the
reset_line_reading word, which is exported in the line-reading
dictionary (though the export is not documented in loader man pages).
But repo history shows it was probably exported for the PNP support
code, which was never included in the loader build, and was removed 5
months ago.
One thing that puzzles me: how has this bug gone unnoticed/unfixed for
nearly 2 decades? I find it hard to believe that nobody's tried to do
something interesting with nextboot, like load a kernel and filesystem,
which is what I'm doing.
Tested by: Gary Jennejohn
PR: 239315
MFC After: 3 weeks
Reviewed by: imp (and correctly applied this time)
Differential Revision: https://reviews.freebsd.org/D31328
CPUTYPE?=native causes -march=native to be added to the command
line. When the host machine is haswell, this causes some versions of
clang to generate code that can't execute in the efi boot loader
environment. Set _CPUCFLAGS= to undo what's done bsd.cpu.mk. bsd.cpu.mk
is included too early to control with NO_CPU_CFLAGS here. The only other
option is to put that in all the Makefiles, and this is less tedious and
error prone.
PR: 194641
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D31187
MFC After: 1 week
loader_simp is a much simplified version of loader that will process a
linear sequence of commands from loader.rc. It has neither Forth nor Lua
built in and is much smaller. Document it. This is largely copied from
loader.8 since it implements those built-in commands. Future revisions
will fix this duplication.
Sponsored by: Netflix
boot.4th was a thing for only a few months around FreeBSD 3.1. The
correct name has been loader.4th for a long time.
MFC After: 2 days
Sponsored by: Netflix
