We want to keep our root file system open to preserve bcache segment
between file accesses, thus reducing physical disk IO.
Reviewed by: imp, allanjude, kevans (previous version)
Differential Revision: https://reviews.freebsd.org/D30848
MFC after: 1 month
- beriloader: archsw is declared extern and defined elsewhere
- ofwloader: ofw_elf{,64} are defined in elf_freebsd.c and
ppc64_elf_freebsd.c respectively
- ubldr: syscall_ptr is defined in start.S for whichever ubldr platform is
building
-fno-common will become the default in GCC10/LLVM11.
MFC after: 3 days
we have crc32(const void *, size_t) in libsa. Unfortunately zlib has
crc32(long, const unigned char *, unsigned) and we have conflict.
Since we do build libsa with zlib, we can use zlib version instead.
Reviewed by: allanjude
Differential Revision: https://reviews.freebsd.org/D24068
The vectx API, computes the hash for verifying a file as it is read.
This avoids the overhead of reading files twice - once to verify, then
again to load.
For doing an install via loader, avoiding the need to rewind
large files is critical.
This API is only used for modules, kernel and mdimage as these are the
biggest files read by the loader.
The reduction in boot time depends on how expensive the I/O is
on any given platform. On a fast VM we see 6% improvement.
For install via loader the first file to be verified is likely to be the
kernel, so some of the prep work (finding manifest etc) done by
verify_file() needs to be factored so it can be reused for
vectx_open().
For missing or unrecognized fingerprint entries, we fail
in vectx_open() unless verifying is disabled.
Otherwise fingerprint check happens in vectx_close() and
since this API is only used for files which must be verified
(VE_MUST) we panic if we get an incorrect hash.
Reviewed by: imp,tsoome
MFC after: 1 week
Sponsored by: Juniper Networks
Differential Revision: https://reviews.freebsd.org//D23827
Currently, our overlay blob will grow to include the size of the complete
overlay blob we're applying. This doesn't scale very well with a lot of
overlays- they tend to include a lot of overhead, and they will generally
only add a fraction of their total size to the blob they're being applied
to.
To combat this, pack the blob as we apply new overlays and keep track of how
many overlays we've applied. Only ubldr has any fixups to be applied after
overlays, so we only need to re-pad the blob in ubldr. Presumably the
allocation won't fail since we just did a lot worse in trying to apply
overlays and succeeded.
I have no intention of removing the padding in make_dtb.sh. There might be
an argument to be had over whether it should be configurable, since ubldr
*is* the only loader that actually has fixups to be applied and we can do
this at runtime, but I'm not too concerned about this.
This diff has been sitting in Phabricator for a year and a half, but I've
decided to flush it as it does make sure that we're scaling the blob
appropriately and leave room at the end for fixups in case of some freak
circumstance where applying overlays leaves us with a blob of insufficient
size.
Reviewed by: gonzo (a long time ago)
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D14133
It was pointed out that manually loading a .dtb to be used rather than
relying on platform-specific method for loading .dtb will result in overlays
not being applied. This was true because overlay loading was hacked into
fdt_platform_load_dtb, rather than done in a way more independent from how
the .dtb is loaded.
Instead, push overlay loading (for now) out into an
fdt_platform_load_overlays. This method easily allows ubldr to pull in any
fdt_overlays specified in the ub env, and omits overlay-checking on
platforms where they're not tested and/or not desired (e.g. powerpc). If we
eventually stop caring about fdt_overlays from ubenv (if we ever cared),
this method should get chopped out in favor of just calling
fdt_load_dtb_overlays() directly.
Reported by: Manuel Stühn (freebsdnewbie freenet de)
The values of the d_slice and d_partition fields of a disk_devdesc have a
few values with special meanings in the disk_open() routine. Through various
evolutions of the loader code over time, a d_partition value of -1 has
meant both "use the first ufs partition found in the bsd label" and "don't
open a bsd partition at all, open the raw slice."
This defines a new special value of -2 to mean open the raw slice, and it
gives symbolic names to all the special values used in d_slice and
d_partition, and adjusts all existing uses of those fields to use the new
constants.
The phab review for this timed out without being accepted, but I'm still
citing it below because there is useful commentary there.
Differential Revision: https://reviews.freebsd.org/D19262
When I added support for the standard loader(8) disk0s2a: type formats,
the parsing of legacy format was broken because it also contains a colon,
but it comes before the slice and partition. That would cause disk_parsedev()
to return success with the slice and partition set to wildcard values.
This change examines the string first, and if it contains spaces, dots, or
a colon at any position other than the end, it must be a legacy-format
string and we don't even try to use disk_parsedev() on it.
Reported by: Manuel Stuhn
way: device<unit>[s|p]<slice><partition>. E.g., disk0s2a or disk3p12.
The code first tries to parse the variable in this format using the
standard disk_parsedev(). If that fails, it falls back to parsing the
legacy format that has been supported by ubldr for years.
In addition to 'disk', all the valid uboot device names can also be used:
mmc, sata, usb, ide, scsi. The 'disk' device serves as an alias for all
those types and will match the Nth storage-type device found (where N is
the unit number).
loaderdev variable works correctly.
The uboot_devdesc struct is variously cast back and forth between
uboot_devdesc and disk_devdesc as pointers are handed off through various
opaque interfaces. uboot_devdesc attempted to mimic the layout of
disk_devdesc by having a devdesc struct, followed by a union of some
device-specific stuff that included a struct that contains the same fields
as a disk_devdesc. However, one of those fields inside the struct is 64-bit
which causes the entire union to be 64-bit aligned -- 32 bits of padding
is added between the struct devdesc and the union, so the whole mess ends
up NOT properly mimicking a disk_devdesc after all. (In disk_devdesc there
is also 32 bits of padding, but it shows up immediately before the d_offset
field, rather than before the whole collection of d_* fields.)
This fixes the problem by using an anonymous union to overlay the devdesc
field uboot network devices need with the disk_devdesc that uboot storage
devices need. This is a different solution than the one contributed with
the PR (so if anything goes wrong, the blame goes to me), but 95% of the
credit for this fix goes to Pawel Worach and Manuel Stuhn who analyzed the
problem and proposed a fix.
PR: 233097
1MB was leaving very little margin in some of the worse-case scenarios with
lualoader. 2MB is still low enough that we shouldn't have any problems with
UBoot-supported boards.
MFC after: 1 week
U-Boot setup a few variables :
- fdt_addr which is the board static dtb (most of the time loaded before
u-boot or coming from some hardware like a ROM)
- fdt_addr_r which is a location in RAM that holds the DTB loaded by
u-boot or before u-boot
In the case of u-boot + rpi firmware the DTB is loaded in RAM but the location
still end up in the fdt_addr variable and the fdt_addr_r variable exist.
Change the behavior so we test that a DTB exists for every possible variable :
- fdt_addr_r is checked first as if u-boot needed to modify it the
correct DTB will live there.
- fdt_addr is checked second as if we run on a hardware with DTB in ROM
it means that we what/need to run that
- fdtaddr looks like a FreeBSD-ism but since I'm not sure leave it.
Reviewed by: gonzo
Differential Revision: https://reviews.freebsd.org/D16101
r330809 replaced duplication of devdesc struct fields with an embedded copy
of the devdesc struct, to avoid fragility. That means all the scattered
comments indicating that structs must match are no longer valid. Likewise
asserts that attempted to mitigate some of the old fragility.
Reviewed by: imp@
lualoader in itself only uses another ~200K, but there seems to be no reason
not to bump it a little higher to give us some more wiggle room.
With this, I can boot using a menu-enabled lualoader, no problem and
reasonably fast. Some heap usage datapoints from the review:
forthloader, no menus, kernel loaded:
heap base at 0x1203d5b0, top at 0x1208e000, used 330320
lualoader, no menus, kernel loaded:
heap base at 0x42050028, top at 0x420ab000, used 372696
lualoader, menus, kernel loaded:
heap base at 0x42050028, top at 0x420d5000, used 544728
Since then, the no menu case for lualoader should have decreased slightly as
I've made some changes to make sure that it no longer loads any of th emenu
bits with beastie disabled.
While here, split heap size out into a HEAP_SIZE macro.
Reviewed by: ian, imp
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D14471
A foolish consistency is the hobgoblin of little minds, adored by
little statesmen and philosophers and divines. With consistency a
great soul has simply nothing to do. -- Ralph Waldo Emerson
The current system is fragile and requires very careful layout of all
*_devdesc structures. It also makes it hard to change the base
devdesc. Take a page from CAM and put the 'header' in all the derived
classes and adjust the code to match.
For OFW, move the iHandle h_handle out of a slot conflicting with
d_opendata. Due to quirks in the alignment rules, this worked.
However changing the code to use d_opendata storage now that it's a
pointer is hard, so just have a separate field for it.
All other cleanups were to make the *_devdesc structures match where
they'd taken some liberties that were none-the-less compatible enough
to work.
Make sure { on the same line as struct for all struct *devdesc. Move
some type definitions to next to the dv_type define, since that's what
sets the d_type.
One does not simply convert to SUBDIR.yes in stand without making everything
else in the affected files SUBDIR.yes -- there are better ways to do this.
Use SUBDIR.${MK_*} where appropriate. r330248 eliminated most of the
offenders, sweep the rest under the rug.
Differential Revision: https://reviews.freebsd.org/D14545
OK. We don't really need a bsd.stand.mk, and it was causing a -fPIC
for the toolchain to be added (bogusly) when building on amd64. Pull
all relevant defs back into defs.mk and delete bsd.stand.mk.
This saves about 15-20k on i386 loader and zfsloader which when
combined with Lua give us a lot more stack space in those constrained
environments.
Summary:
All metadata.c files are very similar, with only trivial changes. Unify them
into a single common file, with minor special-casing where needed.
Reviewed By: imp
Differential Revision: https://reviews.freebsd.org/D13978
MK_CTF, MK_SSP, MK_PROFILE, NO_PIC, and INTERNALLIB are always the
same, so set them in defs.mk. MAN= is common, so set it here too.
This removes a lot of boring repetition from the Makefiles that added
almost no value.
This should have been done as part of r327350, but due to lack of foresight
it came later. In the different places we apply overlays, we duplicate the
bits that check for fdt_overlays in the environment and supplement that with
any other places we need to check for overlays to load. These "other places"
will be loader specific and are not candidates for consolidation.
Provide an fdt_load_dtb_overlays to capture the common logic, allow passing
in an additional list of overlays to be loaded. This additional list of
overlays is used in practice for ubldr to pull in any fdt_overlays passed to
it from U-Boot environment, but it can be used for any other source of
overlays.
These additional overlays supplement loader.conf(5) fdt_overlays, rather
than replace, so that we're not restricted to specifying overlays in only
one place. This is a change from previous behavior where loader.conf(5)
supplied fdt_overlays would cause us to ignore U-Boot environment, and this
seems nonsensical- user should have sufficient control over both of these
aspects, or lack of control for good reasons.
A knob could be considered in the future to ignore U-Boot supplied overlays,
but the supplemental treatment seems like a good start.
Reviewed by: imp (earlier version), gonzo (earlier version)
Differential Revision: https://reviews.freebsd.org/D13993
Overlays were previously not applied when U-Boot provides FDT or EFI
provides FDT, only when we load FDT from /boot/dtb given name from U-Boot.
Make all three paths lead to loading fdt_overlays and applying them, so that
fdt_overlays can be expected to Just Work.
Reviewed by: gonzo, imp, manu
Differential Revision: https://reviews.freebsd.org/D13664
removing this argument, and expanding when rc is NULL. This
effectively completes the back out of custom scripts for tftp booted
loaders from r269153 that was started in r292344 with the new path
tricks that obsoleted it.
Submitted by: Netflix
simd / no float stuff is centeralized here. Also centralise
-ffreestanding since it is specified everywhere.
This, along with a change to share/mk/bsd.cpu.mk to include -mno-avx2
in CFLAGS_NO_SIMD should fix building for newer machines (eg with
CPUTYPE=haswell) where clang was generating avx2 instructions.
Sponsored by: Netflix
