Currently, our overlay blob will grow to include the size of the complete
overlay blob we're applying. This doesn't scale very well with a lot of
overlays- they tend to include a lot of overhead, and they will generally
only add a fraction of their total size to the blob they're being applied
to.
To combat this, pack the blob as we apply new overlays and keep track of how
many overlays we've applied. Only ubldr has any fixups to be applied after
overlays, so we only need to re-pad the blob in ubldr. Presumably the
allocation won't fail since we just did a lot worse in trying to apply
overlays and succeeded.
I have no intention of removing the padding in make_dtb.sh. There might be
an argument to be had over whether it should be configurable, since ubldr
*is* the only loader that actually has fixups to be applied and we can do
this at runtime, but I'm not too concerned about this.
This diff has been sitting in Phabricator for a year and a half, but I've
decided to flush it as it does make sure that we're scaling the blob
appropriately and leave room at the end for fixups in case of some freak
circumstance where applying overlays leaves us with a blob of insufficient
size.
Reviewed by: gonzo (a long time ago)
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D14133
This adds some new commands to loader :
- pnpmatch
This takes a pnpinfo string as argument and tries to find a kernel module
associated with it. -v and -d option are available and are the same as in
devmatch (v is verbose, d dumps the hints).
- pnpload
This takes a pnpinfo string as argument and tries to load a kernel module
associated with it.
- pnpautoload
This will attempt to load every kernel module for each buses. Each buses are
probed, the probe function will generate pnpinfo string and load kernel module
associated with it if it exists.
Only simplebus for FDT system is implemented for now.
Since we need the dtb and overlays to be applied before searching the tree
fdt_devmatch_next will load and apply the dtb + overlays.
All the pnp parsing code comes from devmatch and is the same at 99%.
Reviewed by: imp, kevans
Differential Revision: https://reviews.freebsd.org/D19498
When efi_autoload is called it will call fdt_setup_fdtp which setup the
dtb and overlays. If a user already loaded at dtb or overlays or just
printed the efi provided dtb, this will re-setup everything and also
re-applying the overlays.
Test that everything is setup before doing it again.
Reviewed by: kevans
Differential Revision: https://reviews.freebsd.org/D20059
It was pointed out that manually loading a .dtb to be used rather than
relying on platform-specific method for loading .dtb will result in overlays
not being applied. This was true because overlay loading was hacked into
fdt_platform_load_dtb, rather than done in a way more independent from how
the .dtb is loaded.
Instead, push overlay loading (for now) out into an
fdt_platform_load_overlays. This method easily allows ubldr to pull in any
fdt_overlays specified in the ub env, and omits overlay-checking on
platforms where they're not tested and/or not desired (e.g. powerpc). If we
eventually stop caring about fdt_overlays from ubenv (if we ever cared),
this method should get chopped out in favor of just calling
fdt_load_dtb_overlays() directly.
Reported by: Manuel Stühn (freebsdnewbie freenet de)
Generally straightforward enough; a copy of argv[1] was being made in
command_fdt_internal, solely used for a comparison within the
handler-search, then promptly leaked.
Reported by: ports gcc and clang's static analyzer
As indicated by the comment, any fixups applied (which might include
overlays) can invalidate the previously located node by adding nodes or
setting/adding properties. The later fdt_setprop of fixup-applied property
would then fail because of the bad/wrong node offset.
This would have generally been harmless, but potentially caused multiple
applications of fixups and caused a little bit of bloat.
MFC after: 1 week
OK. We don't really need a bsd.stand.mk, and it was causing a -fPIC
for the toolchain to be added (bogusly) when building on amd64. Pull
all relevant defs back into defs.mk and delete bsd.stand.mk.
This saves about 15-20k on i386 loader and zfsloader which when
combined with Lua give us a lot more stack space in those constrained
environments.
MK_CTF, MK_SSP, MK_PROFILE, NO_PIC, and INTERNALLIB are always the
same, so set them in defs.mk. MAN= is common, so set it here too.
This removes a lot of boring repetition from the Makefiles that added
almost no value.
Whether we should be overwriting the loaded FDT module with the 'fixed up'
version or not was questionable when this was added, and now that overlays
are possible this is downright wrong.
Overlays can increase the size of the blob, so writing it back to the
original VA will generally write past the end of the block and start
clobbering other things in memory.
Rip it out- it was questionable to begin with, it's doing bad things now,
and it serves no purpose since the modified blob will be copied into place
rather than relying on this to reflect the changes.
Reviewed by: gonzo
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D14130
Example overlays seen in other places use a compatible property on root node
of an overlay to specify SOC compatibility. These don't get merged into base
FDT as they're not part of a fragment, but it's expected that consumers of
the overlay may want to check it.
If /compatible on the overlay is missing, just apply it. This is the "I know
what I'm doing" mode for those wanting to whip up a quick overlay and apply
it. An overlay intended for distribution should include /compatible so as
not to break a user's system.
If /compatible on the overlay exists, honor it and cross-check it with
/compatible on the base FDT. If /compatible on the base FDT is missing in
this case, don't apply the overlay rather than risk breaking the system.
Move the COPYOUT of overlay material to before we allocate space for
next_fdtp so that we can avoid the allocation and copy into next_fdtp if we
already know that the overlay can't apply.
This gives way to the possibility of autoloading overlays found in
/boot/overlays, since this provides a means of filtering out overlays not
applicable to the current board.
Reviewed by: gonzo
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D13969
This should have been done as part of r327350, but due to lack of foresight
it came later. In the different places we apply overlays, we duplicate the
bits that check for fdt_overlays in the environment and supplement that with
any other places we need to check for overlays to load. These "other places"
will be loader specific and are not candidates for consolidation.
Provide an fdt_load_dtb_overlays to capture the common logic, allow passing
in an additional list of overlays to be loaded. This additional list of
overlays is used in practice for ubldr to pull in any fdt_overlays passed to
it from U-Boot environment, but it can be used for any other source of
overlays.
These additional overlays supplement loader.conf(5) fdt_overlays, rather
than replace, so that we're not restricted to specifying overlays in only
one place. This is a change from previous behavior where loader.conf(5)
supplied fdt_overlays would cause us to ignore U-Boot environment, and this
seems nonsensical- user should have sufficient control over both of these
aspects, or lack of control for good reasons.
A knob could be considered in the future to ignore U-Boot supplied overlays,
but the supplemental treatment seems like a good start.
Reviewed by: imp (earlier version), gonzo (earlier version)
Differential Revision: https://reviews.freebsd.org/D13993
This behavior also matches a Linux-ism by allowing fdt_overlays to specify
names of overlays without an extension, e.g. fdt-overlays="sunxi-h3-h5-emac"
If we fail to load the file given by a name in fdt_overlays, try again with
".dtbo" appended to it. This still allows overlays to lack .dtbo extension
if user prefers it and just adds a fallback cushion.
Future work could move this from a hard-coded ".dtbo" to a loader.conf(5)
configuration option.
Reviewed by: gonzo
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D13968
libfdt highlights since 1.4.3:
- fdt_property_placeholder added to create a property without specifying its
value at creation time
- stringlist helper functions added to libfdt
- Improved overlay support
- Various internal cleanup
Also switch stand/fdt over to using libfdt for overlay support with this
update. Our current overlay implementation works only for limited use cases
with overlays generated only by some specific versions of our dtc(1). Swap
it out for the libfdt implementation, which supports any properly generated
overlay being applied to a properly generated base.
This will be followed up fairly soon with an update to dtc(1) in tree to
properly generate overlays.
MFC note: the <stdlib.h> include this update introduces in libfdt_env.h is
apparently not necessary in the context we use this in. It's not immediately
clear to me the motivation for it being introduced, but it came in with
overlay support. I've left it in for the sake of accuracy and because it's
not harmful here on HEAD, but MFC'ing this to stable/11 will require
wrapping the #include in an `#ifndef _STANDALONE` block or else it will
cause build failures.
Tested on: Banana Pi-M3 (ARMv7)
Tested on: Pine64 (aarch64)
Tested on: PowerPC [nwhitehorn]
Reviewed by: manu, nwhitehorn
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D13893
If fdt_overlay_apply fails at some stage to apply the overlay to the base,
both the base and overlay may be in an inconsistent state (some fixups
applied, some phandles adjusted, some symbols merged). These can be bad for
a number of reasons, to include user frustration if some fixups applied and
not others. Fail a little safer by making a clean copy of the base FDT for
every overlay that we can simply discard if things go awry.
This also allows us the luxury of simply discarding overlays if we hit some
kind of memory limit or if they're malformed and extremely large for some
reason. We'll now leave a nice error message indicating that some overlays
could not be applied due to size restrictions and we apply what we can.
I note that our overlay implementation has some flaws that might still leave
your system in an unbootable state even if an overlay applies correctly;
please exercise caution in using overlays until we can swap it out for
libfdt's implementation.
Tested on: BananaPi-M3 (armv7)
Tested on: Pine64 (aarch64)
Differential Revision: https://reviews.freebsd.org/D13709
libfdt will assume a writable fdt overlay blob has been passed in, so make
ours compatible to allow easier review when we try to drop libfdt into
place. overlay from the calling context is writable, making it safe to
simply rip out everything related to copying the overlay blob in
fdt_overlay_apply.
I note here that we still have problems: fdt_overlay_apply, both our version
and libfdt's, may fail and have already clobbered the base fdt to some
extent. Future work will make sure we don't apply a potentially bogus fdt,
instead discarding the base fdt if we had an error.
Reviewed by: gonzo
Differential Revision: https://reviews.freebsd.org/D13695
libfdt.h should be included before fdt.h, as hinted at by all of libfdt/;
standard include order being libfdt.h, libfdt_env.h, fdt.h.
The current include order also causes problems when libfdt gets updated, as
fdt.h requires some definitions from libfdt.h.
Differential Revision: https://reviews.freebsd.org/D13688
In the case of a simple dtbo where fragment uses target-path and the overlay
contains no references, /__fixups__ will not be included by either our dtc
or dtc from ports, but the file still has valid fragments to be applied.
Additional testing found that /__symbols__ might also be omitted if it's
empty, which is not necessarily an error.
Reviewed by: gonzo, imp
Differential Revision: https://reviews.freebsd.org/D13663
fdt_load_dtb_overlays was written to unload previous overlay when a new
valid one is come across. fdt_apply_overlays further down is written to
iterate over all .dtbo's currently loaded and apply them one-by-one. Correct
fdt_load_dtb_overlays to stop dropping valid overlays that were previously
loaded and match expectations.
Reviewed by: gonzo, imp
Differential Revision: https://reviews.freebsd.org/D13659
simd / no float stuff is centeralized here. Also centralise
-ffreestanding since it is specified everywhere.
This, along with a change to share/mk/bsd.cpu.mk to include -mno-avx2
in CFLAGS_NO_SIMD should fix building for newer machines (eg with
CPUTYPE=haswell) where clang was generating avx2 instructions.
Sponsored by: Netflix
