The vectx API, computes the hash for verifying a file as it is read.
This avoids the overhead of reading files twice - once to verify, then
again to load.
For doing an install via loader, avoiding the need to rewind
large files is critical.
This API is only used for modules, kernel and mdimage as these are the
biggest files read by the loader.
The reduction in boot time depends on how expensive the I/O is
on any given platform. On a fast VM we see 6% improvement.
For install via loader the first file to be verified is likely to be the
kernel, so some of the prep work (finding manifest etc) done by
verify_file() needs to be factored so it can be reused for
vectx_open().
For missing or unrecognized fingerprint entries, we fail
in vectx_open() unless verifying is disabled.
Otherwise fingerprint check happens in vectx_close() and
since this API is only used for files which must be verified
(VE_MUST) we panic if we get an incorrect hash.
Reviewed by: imp,tsoome
MFC after: 1 week
Sponsored by: Juniper Networks
Differential Revision: https://reviews.freebsd.org//D23827
The vectx API, computes the hash for verifying a file as it is read.
This avoids the overhead of reading files twice - once to verify, then
again to load.
For doing an install via loader, avoiding the need to rewind
large files is critical.
This API is only used for modules, kernel and mdimage as these are the
biggest files read by the loader.
The reduction in boot time depends on how expensive the I/O is
on any given platform. On a fast VM we see 6% improvement.
For install via loader the first file to be verified is likely to be the
kernel, so some of the prep work (finding manifest etc) done by
verify_file() needs to be factored so it can be reused for
vectx_open().
For missing or unrecognized fingerprint entries, we fail
in vectx_open() unless verifying is disabled.
Otherwise fingerprint check happens in vectx_close() and
since this API is only used for files which must be verified
(VE_MUST) we panic if we get an incorrect hash.
Reviewed by: imp,tsoome
MFC after: 1 week
Sponsored by: Juniper Networks
Differential Revision: https://reviews.freebsd.org//D23827
The vectx API, computes the hash for verifying a file as it is read.
This avoids the overhead of reading files twice - once to verify, then
again to load.
For doing an install via loader, avoiding the need to rewind
large files is critical.
This API is only used for modules, kernel and mdimage as these are the
biggest files read by the loader.
The reduction in boot time depends on how expensive the I/O is
on any given platform. On a fast VM we see 6% improvement.
For install via loader the first file to be verified is likely to be the
kernel, so some of the prep work (finding manifest etc) done by
verify_file() needs to be factored so it can be reused for
vectx_open().
For missing or unrecognized fingerprint entries, we fail
in vectx_open() unless verifying is disabled.
Otherwise fingerprint check happens in vectx_close() and
since this API is only used for files which must be verified
(VE_MUST) we panic if we get an incorrect hash.
Reviewed by: imp,tsoome
MFC after: 1 week
Sponsored by: Juniper Networks
Differential Revision: https://reviews.freebsd.org//D23827
LLD 10.0.0 changed the behavior of the -Ttext option, so that using
-Ttext=0x0 now causes linking of the loaders to fail with:
ld: error: output file too large: 18446744073707016908 bytes
I reported this in https://bugs.llvm.org/show_bug.cgi?id=44715, and
initially reverted the upstream change in r357259 to work around it.
However, after some discussion with Fangrui Song in the upstream ticket,
I think we can classify this as an unfortunate interaction between using
-Ttext=0 in combination with --no-rosegment. (We added the latter
in r332090, because btxld does not correctly handle input with more
than 2 PT_LOAD segments.)
Fangrui suggested to use a linker script instead, and Warner was already
attempting this in r305353, but had to revert it due to "crypto-using
boot problems" (not sure what those were :).
This review updates the stand/i386/boot.ldscript to handle more
sections, inserts some symbols like _edata and such that we use in
libsa, and also discards any .interp section.
It uses ORG which is defined on the linker command line using
--defsym ORG=value to set the start of all the sections.
Reviewed by: imp
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D23952
Before this change, LLD10 was creating several extra PT_LOAD sections,
which OFW does not understand.
Like we do for the kernel already, specify the program headers manually.
Additionally, to work around a crash in our base ld.bfd, we need to
actually assign something to the output section. LLD does not need this.
One side effect of this change is the removal of the GNU_STACK header.
This is more correct, since we are using a statically-allocated stack and
RWX mappings across the board this early in boot.
Reviewed by: jhibbits, Fangrui Song <i@maskray.me>
Sponsored by: Tag1 Consulting, Inc.
Differential Revision: https://reviews.freebsd.org/D23778
smbios used to be an i386 only kinda weird quirk to the x86
architecture. But UEFI picked it up, dusted it off and now it's many
other locations. Make it base technology by moving it to libsa and
fixing up the compliation. The code has issues with unaligned access
still, but that will be addressed in a followup commit.
Differential Revision: https://reviews.freebsd.org/D23660
Create a ptov() function. It's basically the same as the btx PTOV
macro, but works everywhere. smbios needs this to translate addresses,
but the translation differs between BIOS booting and EFI booting. Make
it a function so one smbios.o can be used everywhere. Provide
definitions for it in the two loaders affected.
Differential Revision: https://reviews.freebsd.org/D23660
We still need to set the hooks to prevent improper manipulations thereafter
at the loader prompt, but as it is we're actively preventing loaddev from
being set correctly in some circumstances (ZFS in particular) and doing more
work than needed with currdev -- that hook in particular validates it as a
correct device, which we can assume isn't needed in this context.
Reviewed by: imp, sigsys@gmail.com
Submitted/Diagnosed by: sigsys@gmail.com
Differential Revision: https://reviews.freebsd.org/D23390
If we hit an error at this level, we are almost certainly not in any kind
of sane state where autoboot can do the right thing. Instead of letting it
try and probably failing, disable autoboot so they immediately get kicked
into a loader prompt for manual remediation/diagnosis.
Reviewed by: tsoome
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D23611
vdev_read_pad2() does read VDEV_PAD_SIZE of data, and will copy size bytes
of it, hence, we need buffer of VDEV_PAD_SIZE bytes.
Issue introduced in r357497.
Reported by: se
CD boot is broken for some systems since bioscd and biosdisk merge. The issue is that we can not use anything else than int 13 ah=0x4b to query cd information.
The patch does restore the same probe as was originally used in bioscd.c. Additionally extra buffer padding is used to avoid memory corruption caused by some systems.
PR: 234031
Reported by: ultramage and others
MFC after: 1 day
stand/i386/gptboot/gptldr.S:141:3: error: value of 36878 is too large for field of 2 bytes.
jmp MEM_JMP # Start BTX
^
Use the same construct as in stand/i386/boot2/boot1.S, which ensures the
jump distance does not become too large.
MFC after: 3 days
This may be used for the local module to hook in and load any additional
modules that it wants, since it can't modify the modules table internal to
config. We may consider adding API to do so at a later time, but I suspect
it will be more complicated to use with little return.
status is captured but ignored for the purpose of loading the hook. status
will be false if *any* module failed to load, but we typically don't let
that halt the boot so there's no reason to let it halt hooks. Some vendors
or setups may have expected fails that would be actively thwarted by
checking it.
We may, at a later date, consider adding an API for letting non-config
modules check which modules have successfully (or not) loaded in case an
unexpected failure *should* halt whatever they are doing.
MFC after: 3 days
The major problem with the current ordering is that loader.conf may contain
all of the magic we need to actually setup the console, so loading local.lua
prior to that can make it excessively difficult and annoying to debug
(whoops, sorry Ravi & Warner).
The new ordering has some implications, but I suspect they are a non-issue.
The first is that it's no longer possible for the local module to inject any
logic prior to loading config -- I suspect no one has relied on this. The
second implication is that the config.loaded hook is now useless, as the
local module will always be included after that hook would have fired.
For config.loaded, I will opt to leave it in, just in case we add an early
point for local lua to get injected or in case one wants to schedule some
deferred logic in a custom loader.lua. The overhead of having it if no hooks
will be invoked is relatively minimal.
Diagnosed by: imp
Reported by: imp, rpokala (most likely)
MFC after: 3 days
As a new x86 CPU vendor, Chengdu Haiguang IC Design Co., Ltd (Hygon)
is a joint venture between AMD and Haiguang Information Technology Co.,
Ltd., aims at providing x86 processors for China server market.
The first generation Hygon processor(Dhyana) shares most architecture
with AMD's family 17h, but with different CPU vendor ID("HygonGenuine")
and PCI vendor ID(0x1d94) and family series number 18h(Hygon negotiated
with AMD to confirm that only Hygon use family 18h).
To enable Hygon Dhyana support in FreeBSD, add new definitions
HYGON_VENDOR_ID("HygonGenuine") and X86_VENDOR_HYGON(0x1d94) to identify
Hygon Dhyana CPU.
Initialize the CPU features(topology, local APIC ext, MSI, TSC, hwpstate,
MCA, DEBUG_CTL, etc) for amd64 and i386 mode by sharing the code path of
AMD family 17h.
The changes have been applied on FreeBSD 13.0-CURRENT and tested
successfully on Hygon Dhyana processor.
References:
[1] Linux kernel patches for Hygon Dhyana, merged in 4.20:
https://git.kernel.org/tip/c9661c1e80b609cd038db7c908e061f0535804ef
[2] MSR and CPUID definition:
https://www.amd.com/system/files/TechDocs/54945_PPR_Family_17h_Models_00h-0Fh.pdf
Submitted by: Pu Wen <puwen@hygon.cn>
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D23163
Due to the way u-boot for 32-bit powerpc is compiled, the interrupt code
assumes that the GOT pointer (r30) on u-boot is always intact.
When making syscalls to u-boot, ensure that we have restored r30 like we
found it before we enable interrupts to prevent u-boot from crashing if a
timer interrupt was pending.
This fixes ubldr on e500 qemu (assuming you have recompiled qemu's u-boot
with API support!)
Reviewed by: jhibbits
Sponsored by: Tag1 Consulting, Inc.
Differential Revision: https://reviews.freebsd.org/D23258
Currently, our overlay blob will grow to include the size of the complete
overlay blob we're applying. This doesn't scale very well with a lot of
overlays- they tend to include a lot of overhead, and they will generally
only add a fraction of their total size to the blob they're being applied
to.
To combat this, pack the blob as we apply new overlays and keep track of how
many overlays we've applied. Only ubldr has any fixups to be applied after
overlays, so we only need to re-pad the blob in ubldr. Presumably the
allocation won't fail since we just did a lot worse in trying to apply
overlays and succeeded.
I have no intention of removing the padding in make_dtb.sh. There might be
an argument to be had over whether it should be configurable, since ubldr
*is* the only loader that actually has fixups to be applied and we can do
this at runtime, but I'm not too concerned about this.
This diff has been sitting in Phabricator for a year and a half, but I've
decided to flush it as it does make sure that we're scaling the blob
appropriately and leave room at the end for fixups in case of some freak
circumstance where applying overlays leaves us with a blob of insufficient
size.
Reviewed by: gonzo (a long time ago)
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D14133
Since rS330365, there has been no particular reason for libofw to be in a
subdirectory of ofw. Move libofw up a level to make it fit in better with
the other top level libraries.
Also add a LIBOFWSRC to stand/defs.mk to match what all the other
libraries are doing.
Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D23000
Summary:
This patch is to support ongoing work for replacing "GCC/BFD" by "CLANG/LLD" on
target PowerPC64 [1], by proposing a way to specify and/or locate a secondary
ld.bfd linker.
This is necessary as LLD currently doesn't support PowerPC 32 bits, so we keep
using BFD for the 32 bit stuff on PowePC64(LIB32 compatibility and
STAND/slof/loader.)
- creates LD_BFD variable pointing to ld.bfd
- use LD_BFD as linker for LIB32/compat
- Default behavior for other platforms aren't changed.
[1] https://wiki.freebsd.org/powerpc/llvm-elfv2
Submitted by: alfredo.junior_eldorado.org.br
Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D20261
partitions can have 128 partitions, so parse contiguous digits and then
validate that the number is between 1-128 inclusive.
I'm not sure 128 is a hard limit in the GPT standard, but it's the common
number in use, and it's a better upper limit than 9.
This uses the new layout of the upstream repository, which was recently
migrated to GitHub, and converted into a "monorepo". That is, most of
the earlier separate sub-projects with their own branches and tags were
consolidated into one top-level directory, and are now branched and
tagged together.
Updating the vendor area to match this layout is next.
From UEFI specification 2.8, page 434:
"It should also be noted that certain input devices may not be able to produce
shift or toggle state information, and in those cases the high order bit in
the respective Toggle and Shift state fields should not be active."
But we still need to check for ScanCode and UnicodeChar.
PR: 242660
Reported by: Trond Endrestol
MFC after: 1 week
For all the INT13 calls, use symbolic names instead of magic numbers. This makes
it easier to understand what the code is doing w/o a trip to google to find what
these numbers mean.
In some cases the pool discovery will get stuck in infinite loop while setting
up the vdev children.
To fix, we split the vdev setup into two parts, first we create vdevs based on
configuration we do get from pool label, then, we process pool config from MOS
and update the pool config if needed.
Testing done: confirm previously hung loader is not hung any more.
MFC after: 1 week
Instead of passing NULL to fzap_name_equal and crashing, just return
ENOENT. This happened when higher bits of a hash of the searched key
(its hash prefix) matched a hash prefix of some key in the ZAP, but the
full hash value of the searched key did not match any key in the ZAP.
I observerved this problem when loader tried to look up
"features_for_read" in a particular old pool that predates pool
features.
MFC after: 2 weeks
Sponsored by: Panzura
We do allocate amount of memory (void * or char *), and then assign this
buffer to struct iso_primary_descriptor *vd. Make sure we do
allocate enough bytes.
In fact we do allocate enough, but it is good idea to make sure this really
is so.
MFC after: 1 week
Lua allocates LUAL_BUFFERSIZE buffers on the stack for various string
functions (string.format, string.gsub) -- this works out to be somewhat
significant and not necessary, based on how we use string operations.
Dropping it risks having to allocate per call to format/gsub, but this is
not the case for our usage. This simply stops allocating 8K buffers on the
stack when luaL_Buffer is used.
Reviewed by: imp
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D22500
Update a bunch of Makefile.depend files as
a result of adding Makefile.depend.options files
Reviewed by: bdrewery
MFC after: 1 week
Sponsored by: Juniper Networks
Differential Revision: https://reviews.freebsd.org/D22494
