Fix a typo by reworking the comment for blake3_impl_hack.c and amplify
the nature of the hack and its temporary reason for existing.
Sponsored by: Netflix
Reviewed by: delphij
Differential Revision: https://reviews.freebsd.org/D35895
Now that the minor issues preventing zfs.c from using CFLAGS_EARLY have
been fixed, use that mechanism like everything else that needs the
OpenZFS spl headers. This simplifies things somewhat. Update comments to
document why zfs.c is still special, though in different ways.
Note: We also use the fact that NEED_SOLARIS_BOOLEAN is only defined in
an environment where the solaris compat boolean stuff will be defined
prior to this point (eg, when we're building zfs.c in libsa), but not in
other environments (like when we're building mkimage and stand-alone
boot loaders that don't use libsa). These latter uses should be changed
to use the same ZFS compile env, but aren't as part of this commit.
This has to be done in the same change as the ZFS_EARLY change to not
break zfs.c building for one commit affecting bisectabiltiy.
Sponsored by: Netflix
Reviewed by: tsoome, delphij
Differential Revision: https://reviews.freebsd.org/D35894
Constrain CFLAGS for ZFS: don't add anything globally. Add the includes
to only the files that need them. Add -DHAS_ZSTD_ZFS to zfs.c (which
includes zfsimpl.c which includes zfssubr.c both of which need this
defined). Also add it to efi/boot1/Makefile since zfs_module.c also
includes zfsimple.c.
Sponsored by: Netflix
Reviewed by: tsoome
Differential Revision: https://reviews.freebsd.org/D35887
Various GELI sources need bootstrap.h and disk.h. In theory they
shouldn't need anything outside of libsa, but disk.h and bootstrap.h are
currently required.
This fixes the build with MK_LOADER_ZFS=no.
Obtained from: CheriBSD
Fixes: eaf7aabddc stand: geli CFLAGS tightening
Sponsored by: DARPA
Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D35861
In theory they shouldn't need anything outside of libsa, but disk.h and
bootstrap.h are currently required. Future work wil address this issue.
This fixes the build with MK_LOADER_ZFS=no. ZFS's Makefile.inc adds
these flags globally to CFLAGS when it should not. This masked the
problem because the tools/boot/universe.sh didn't build MK_LOADER_ZFS=no
as part of its regressions. Future work will also fix this.
Obtained from: CheriBSD
Fixes: 84bf2bbbec stand: constrain zlib/gzip CFLAGS better
Sponsored by: DARPA
Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D35860
With clang 15, the following -Werror warning is produced:
stand/i386/boot2/boot2.c:358:6: error: a function declaration without a prototype is deprecated in all versions of C [-Werror,-Wstrict-prototypes]
This is because parse() is declared with a (void) argument list, and
defined with an empty argument list. Make the definition match the
declaration.
MFC after: 3 days
Create a wrapper for the mount system call. To ensure a sane early boot
environment and to gather data we need for kexec, we may need to mount
some special filesystems.
Sponsored by: Netflix
Early in boot, we need to create the normal stdin/out/err env for the
boot loader to run in. To do that, we need to open the console and
duplicate the file descriptors which requires dup(2). Implement a
wrapper as host_dup.
Sponsored by: Netflix
Linux's /dev/fd is implemented inside of /proc/self/fd, so we may need
to create a symlink to it early in boot. "/dev/fd" and "/dev/std*" might
not be strictly required for the boot loader, but should be present for
maximum flexibility.
Sponsored by: Netflix
Add host_getpid() so we can know if we're running as init(8) or not. If
we are, we may chose to do early system setup / sanity operations.
Sponsored by: Netflix
Implement stat(2) and fstat(2) in terms of newfstatat and newfstat
system calls respectively (assume we have a compat #define when
there's no newfstat and just a regular fstat and do so for ppc).
Snag struct kstat (the Linux kernel stat(2), et al interface) from musl
and attribute properly.
Sponsored by: Netflix
Add the common O_ constants for the open, fcntl, etc system calls. They
are different than FreeBSD's. While they can differ based on
architecture, they are constant for architectures we care about, and
those architectures use the 'generic' version so future architectures
will also work.
Sponsored by: Netflix
Fallback to currdev when NULL is passed in when 'rootdev' is NULL. Other
getdevs do this. Additional features are needed here still, though.
Sponsored by: Netflix
Split _start into _start and _start_c (inspired by musl and the powerpc
impl is copied from there). This allows us to actually get the command
line arguments on all the platforms. We have a very simplified startup
that supports only static linking.
Sponsored by: Netflix
with grab_faults, we can try to print out the trace of function calls.
Without symbol table, we can not translate addresses to function names,
but even addresses can help to track the bugs.
For loader functions, print out absolute address, so it could be
searched from objdump -d output.
Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D35433
Add comments describing the weird nesting things we have to do to live
inside the ZFS world. Also fix a stale comment by moving nvlist.c to an
early user.
Sponsored by: Netflix
Add the necessary glue to get blake3 building for the boot loaded as
well as connected to the ZFS system so it is useful.
On some platforms, we create references to blake3_sse2_impl and
blake3_sse41_impl ops structs to utilize SIMD. These aren't present on
x86 (since we dind't ask for them), but are on aarch64 with no
implementation. Since we don't want SIMD in the boot loader, have these
all return 'unsupported' always. This should be fixed upstream to allow
more flexibility in this selection, but for now we use this hack to not
modify the sys/contrib/openzfs with difficult to maintain hacks while
an upstreamable solution is found.
tsoome@ did the implementation bits in sys/cddl/boot, and I did the
Makefile work and the aweful blake3_impl_hack.c.
Co-author: tsoome@freebsd.org
Sponsored by: Netflix
Reviewed by: kevans (earlier version)
Differential Revision: https://reviews.freebsd.org/D35750
We no longer need to use this to get zfs_zstd.c compiling, so delete it.
Sponsored by: Netflix
Reviewed by: kevans
Differential Revision: https://reviews.freebsd.org/D35750
Now that we have the ability to work around all the issues that
zstd_stub.c worked around, compile zfs_std.c directly.
Sponsored by: Netflix
Reviewed by: kevans
Differential Revision: https://reviews.freebsd.org/D35750
sys/blake3.h is not safe to include in a standalone environment. It
assumes, unwisely, that there's only kernel or userland. The userland
choice is bad due to imperfections in how we handle stdlib.h in this
environment (we wind up including the host's stdlib.h, it expects a
standard FreeBSD setup which the gymnastics we've done to create a
OpenSolaris/Illumos-like environment interfere with). Thankfully, in
this case we can just pretend to be the kernel for a little bit by
defining _KERNEL and the undefing it after we exit.
Sponsored by: Netflix
Reviewed by: kevans
Differential Revision: https://reviews.freebsd.org/D35750
There's a number of issues with including zfs_context.h from a
standalone environment. First, sys/uio_imp.h isn't at all safe for this
environment, so define its guard #defines so that its contents are
skipped. Next, there's a problem including string.h to get the mem*
routines, so just define them here. ZFS_MODULE_PARAM_ARGS isn't defined
properly. I had wanted to define it when I was upstreaming changes to
include/os/freebsd/spl/sys/zfs_context.h, but they ran into resistance
so I'm defining that here now (it is also defined in zstd_shim.c, but
that will disappear once the issues it works around are
cleared). Finally, sys/sysmacros.h has to be included now before
sys/atomic.h, but upstream includes it after so include it here so that
the guards make the out-of-order includes in upstream irrelevant.
Sponsored by: Netflix
Reviewed by: kevans
Differential Revision: https://reviews.freebsd.org/D35750
OpenZFS is based on having a number of layers of include files that
define things and include the next layer. Insert a later at the start
for files in libsa/zfs/spl so we can override issues with upstream
OpenZFS while we work to get the issues corrected upstream.
Sponsored by: Netflix
Reviewed by: kevans
Differential Revision: https://reviews.freebsd.org/D35750
Add a note that veriexec / bearssl builds will pollute the CFLAGS in a
way that's somewhat hard to fix, so I'm just noting it for now.
Sponsored by: Netflix
Only add -DWEAK_REFS to sha256.c and sha512.c instead of
everything. Remove redundant include that's not needed.
Minor formatting tweak.
Sponsored by: Netflix
CFLAGS+= here affects *ALL* libsa files being built. However, this is
only needed for zfs.c, so define it only for this. Also, use the defines
from defs.mk. Move all the zfs.c include hacks together. Also, move the
-Wformat -Wall warnings that were added to CFLAGS+= to the individual
files instead for the same reason.
Sponsored by: Netflix
ZFSSRC is the top level directory where the ZFS sources come from. Don't
reuse it for a list of ZFS sources. Instead, use ZFS_SRC
Sponsored by: Netflix
Given that hints set this way don't work when a static kenv is compiled
into the kernel. acpi.rsdp has been set for this for the past 6 years,
and all kernels in that time have used it in preference to the hints. As
such, we no longer hints.*, so remove them.
Sponsored by: Netflix
Reviewed by: jhb
Differential Revision: https://reviews.freebsd.org/D35634
Several years ago, x86 moved from using hints to communicate this
information to using the simpler acpi.rsdp variables. If one compiles
static hints into the kernel, then these hints are ignored. We can
remove this when we branch FreeBSD 15. Thought about BURN_BRIDGES
here, but it's too messy.
Sponsored by: Netflix
Reviewed by: andrew, jhb
Differential Revision: https://reviews.freebsd.org/D35632
With the introduction of llvm14, the powerpc* loader used on ofw/pseries
for 32 and 64 bit architectures puts the .data.rel.ro section after
.data section. This caused a crash kernel didn't boot. Bisect pointed to
change https://reviews.llvm.org/D111717 but problem could be fixed by
adding a section description to make it appear in the expected order.
This patch is based on discussion at:
https://github.com/llvm/llvm-project/issues/56306
MFC after: 1 day
Sponsored by: Instituto de Pesquisas Eldorado (eldorado.org.br)
In some cases ubldr would try to mount a disk device before
the archsw struct was filled with functions pointers.
This would result in a NULL pointer derefrence of the arch_getdev field.
Fix that filling the archsw functions earlier.
Note that this matches the EFI behavoiur.
Reviewed by: imp, mw
Sponsored by: Stormshield
Obtained from: Semihalf
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D35670
geli_probe_and_attach() does pick geli_devsw structure for
encrypted disks, the implementation depends on device
name "disk" when device type is DEVT_DISK, but geli_devsw is
setting name field "gelidisk".
PR: 264282
Submitted by: yamagi@yamagi.org
Reported by: yamagi@yamagi.org
MFC after: 2 weeks
Detect the end of the disk condition. This may happpen when
disk image is truncated and the reads are addressing blocks past
image end.
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D35432
panic() is using multiple services - attempting to read
keyboard, accessing time functions and finally, exiting the loader.
Protect all the accessed listed above. Note, when BS are off,
we really can not just exit the loader, we only can reboot.
MFC after: 1 week
The check fails in kernels compiled with KASAN because AddressSanitizer
inserts redzones around global variables, so the size of the "kernphys"
symbol is 32 rather than 8. Thus we fall back to copying even though
it's not necessary.
Simply remove the size check. I didn't want to extend the symbol size
check since there's no guarantee that AddressSanitizer will always emit
32 bytes for "kernphys".
Reviewed by: kib
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D35448
Notable upstream pull request merges:
#12321 Fix inflated quiesce time caused by lwb_tx during zil_commit()
#13244 zstd early abort
#13360 Verify BPs as part of spa_load_verify_cb()
#13452 More speculative prefetcher improvements
#13466 Expose zpool guids through kstats
#13476 Refactor Log Size Limit
#13484 FreeBSD: libspl: Add locking around statfs globals
#13498 Cancel in-progress rebuilds when we finish removal
#13499 zed: Take no action on scrub/resilver checksum errors
#13513 Remove wrong assertion in log spacemap
Obtained from: OpenZFS
OpenZFS commit: b9d98453f9
When reading the UEFI memory map we pass in a pointer to the memory to
hold the map. Unfortunately it wasn't initialised before the first use
so clang decided it was undefined behaviour so the entire loop was
removed. This leads to everything in bi_load after this to also be
removed as dead code.
The next function after bi_load in the binary is efi_copy_init. The
above caused us to enter efi_copy_init with a return address of the
start of the function. Because of this it would enter an infinite
loop of calling the function, allocating memory, then returning to
the start of the function.
PR: 264021
head_errlog is new format for errlog, but we do not really
use errlog, so we can just mark it supported, to enable reading
from pool.
MFC after: 1 week
Move declarations into a new nvlist.h rather than putting everything in
libzfs.h. This makes this nvlist code easier to reuse elsewhere. In
particular, the nvlist implementation in sys/contrib/libnv does not
provide XDR encoding, but this is needed when reading from or writing to
ZFS pools.
Also:
- Remove references to boolean_t. It has to be a 32-bit int here, so
just reference the underlying type.
- Add includes needed when compiling the nvlist code outside of stand/.
No functional change intended.
Reviewed by: tsoome
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D35255
When adding an entry to an nvlist, the data buffer might need to be
resized. When this happens, the XDR encoder's notion of the buffer size
also needs to be updated, otherwise the operation may erroneously fail.
Reviewed by: tsoome, imp
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D35254
The loader.efi man page describes UEFI-specific loader details, but not
general loader information.
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
Get amd64 compiling. However, the current kboot supports an old way of
enumerating memory and the new way needs to be incorporated as well. The
powerpc folks could use either, it seems and newer powerpc platforms
need some changes for kboot to work anyway.
This commit includes the linker script, trampoline code to start the new
kernel, Linux system calls and the necessary configuration glue needed
to build the binaries.
This includes a quick hack to get multiboot support, but we need to
really share these defines. The multiiboot2.h is the minimum needed to
build. We have multiboot information in three places now, so a
refactoring is in order.
This should be considered, at best, preliminary and experimental for
anybody wishing to try it out.
Sponsored by: Netflix
Reviewed by: tsoome
Differential Revision: https://reviews.freebsd.org/D35100
Turns out there is a libsa.3. It's a bit out of date, but we reference
it in a number of places so we should install it. We need to do the DO32
dance because this Makefile is included twice and we don't want it
installing twice.
Sponsored by: Netflix
This mirrors dmu_read_impl(), zeroing out the tail end of the buffer and
clipping the read to what's contained by the block that exists.
This fixes an issue that arose during the 13.1 release process; in
13.1-RC1 and later, setting up GELI+ZFS will result in a failure to
boot. The culprit is this, which causes us to fail to load geom_eli.ko
as there's a residual portion after the single datablk that should be
zeroed out.
PR: 263407
Reviewed by: tsoome
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D35019
We don't need it for userboot, but it avoids issues with BIND_NOW, so
just provide it. time(3) isn't defined but ends up being provided by
libc linked into the host process, which is generally fine.
PR: 262920
Reviewed by: imp, jhb
MFC after: 3 days
Diferential Revision: https://reviews.freebsd.org/D34758
Printing device followed by interface matches, e.g., edk2. Note that
this is only a fallback, many firmware implementations will provide the
protocol that we'll use to format device paths.
Reviewed by: imp, tsoome
Sponsored by: Ampere Computing
Submitted by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D34539
- Add SPDX tag
- Add a files section describing different locations related to the
loader
- Add an example explaining how to install a new loader.efi to ESP
- Reference uefi(8)
Reviewed by: tsoome
Reviewed by: Pau Amma <pauamma@gundo.com>
Reviewed by: Jose Luis Duran <jlduran@gmail.com>
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D34592
The console escape code standard (ECMA-48) specifies color #3 (escape
code 33) as yellow. A brown console color is an artifact of the VGA
palette, which replaces dim (but not bright) yellow with brown.
Reviewed by: adrian, imp
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D34531
For historical reasons console color number 3 may be either yellow (most
consoles) or brown (VGA palette). The console escape code standard
uses "yellow", but teken color name constants appear to be based on the
VGA scheme and use TC_BROWN for color 3. Even so, the palette table
used 50,50,0 as the RGB percentage tuple, resulting in a dim yellow for
framebuffer consoles at the time teken was introduced.
Amusingly, in 19e2ce2d83 the comment on the palette entry was changed
from "brown" to "dark yellow" but the colour itself was changed from
a pure yellow to being somewhat brown.
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Accept "bright" or "light" prefix for named colors.
For numeric colors, update error message to specify that values 0 to 15
are allowed, and verify that values are in that range.
Reviewed by: imp, tsoome (both earlier version)
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D34512
Notable upstream pull request merges:
#9078: log xattr=sa create/remove/update to ZIL
#11919: Cross-platform xattr user namespace compatibility
#13014: Report dnodes with faulty bonuslen
#13016: FreeBSD: Fix zvol_cdev_open locking
#13019: spl: Don't check FreeBSD rwlocks for double initialization
#13027: Fix clearing set-uid and set-gid bits on a file when
replying a write
#13031: Add enumerated vdev names to 'zpool iostat -v' and
'zpool list -v'
#13074: Enable encrypted raw sending to pools with greater ashift
#13076: Receive checks should allow unencrypted child datasets
#13098: Avoid dirtying the final TXGs when exporting a pool
#13172: Fix ENOSPC when unlinking multiple files from full pool
Obtained from: OpenZFS
OpenZFS commit: a86e089415
If module_verbose is set to a value below MODULE_VERBOSE_TWIDDLE
call twiddle_divisor(UINT_MAX).
This makes more sense here than when we are loading the kernel.
Sponsored by: Juniper Networks, Inc.
UEFI provides a protocol for accessing randomness. This is a good way
to gather early entropy, especially when there's no driver for the RNG
on the platform (as is the case on the Marvell Armada8k (MACCHIATObin)
for now).
If the entropy_efi_seed option is enabled in loader.conf (default: YES)
obtain 2048 bytes of entropy from UEFI and pass is to the kernel as a
"module" of name "efi_rng_seed" and type "boot_entropy_platform"; if
present, ingest it into the kernel RNG.
Submitted by: Greg V
Reviewed by: markm, kevans
Approved by: csprng (markm)
MFC after: 3 weeks
Differential Revision: https://reviews.freebsd.org/D20780