Commit Graph

1362 Commits

Author SHA1 Message Date
Toomas Soome
98e805b4a1 loader: net_open() should not replace f->f_devdata
net_open() does replace f_devdata with pointer to netdev_sock,
this will cause memory leak when device is closed, but also does
alter the devopen() logic.

We should store &netdev_sock to dev->d_opendata instead, this
would preserve and follow the devopen() logic.

Fixes network boot on aarch64 (tested by bz).

Reviewed-by:	imp
MFC After:	2 weeks
Differential Revision: https://reviews.freebsd.org/D32227
2021-10-19 19:43:56 +03:00
Leandro Lupori
8ecf9a8bab powerpc64: make radix with superpages default
As Radix MMU with superpages enabled is now stable, make it the
default choice on supported hardware (POWER9 and above), since its
performance is greater than that of HPT MMU.

Reviewed by:		alfredo, jhibbits
Sponsored by:		Instituto de Pesquisas Eldorado (eldorado.org.br)
Differential Revision:	https://reviews.freebsd.org/D30797
2021-10-14 13:13:27 -03:00
Kyle Evans
d586c978b9 stand: fix build after recent opencrypto changes
Pass the ivlen along through, and just drop this KASSERT() if we're
building _STANDALONE for the time being.

Fixes:	1833d6042c ("crypto: Permit variable-sized IVs ...")
2021-10-06 20:23:44 -05:00
Colin Percival
248682a589 loader bcache: Allow readahead up to 256 kB I/Os
Prior to this commit, the loader would perform readaheads of up to
128 kB; when booting on a UFS filesystem this resulted in a series
of 160 kB reads (32 kB request + 128 kB readahead).

This commit allows readaheads to be longer, subject to a total I/O
size limit of 256 kB; i.e. 32 kB read requests will have added
readaheads of up to 224 kB.

In my testing on an EC2 c5.xlarge instance, this change reduces the
boot time by roughly 80 ms.

Reviewed by:	tsoome
MFC after:	1 week
Sponsored by:	https://www.patreon.com/cperciva
Differential Revision:	https://reviews.freebsd.org/D32251
2021-10-03 14:55:10 -07:00
Colin Percival
04b9b7c507 loader bcache: Track unconsumed readahead
The loader bcache attempts to determine whether readahead is useful,
increasing or decreasing its readahead length based on whether a
read could be serviced out of the cache.  This resulted in two
unfortunate behaviours:

1. A series of consecutive 32 kB reads are requested and bcache
performs 16 kB readaheads.  For each read, bcache determines that,
since only the first 16 kB is already in the cache, the readahead
was not useful, and keeps the readahead at the minimum (16 kB) level.

2. A series of consecutive 32 kB reads are requested and bcache
starts with a 32 kB readahead resulting in a 64 kB being read on
the first request.  The second 32 kB request can be serviced out of
the cache, and bcache responds by doubling its readahead length to
64 kB.  The third 32 kB request cannot be serviced out of the cache,
and bcache reduces its readahead length back down to 32 kB.

The first syndrome converts a series of 32 kB reads into a series of
(misaligned) 32 kB reads, while the second syndrome converts a series
of 32 kB reads into a series of 64 kB reads; in both cases we do not
increase the readahead length to its limit (currently 128 kB) no
matter how many consecutive read requests are made.

This change avoids this problem by tracking the "unconsumed
readahead" length; readahead is deemed to be useful (and the
read-ahead length is potentially increased) not only if a request was
completely serviced out of the cache, but also if *any* of the request
was serviced out of the cache and that length matches the amount of
unconsumed readahead.  Conversely, we now only reduce the readahead
length in cases where there was unconsumed readahead data.

In my testing on an EC2 c5.xlarge instance, this change reduces the
boot time by roughly 120 ms.

Reviewed by:	imp, tsoome
MFC after:	1 week
Sponsored by:	https://patreon.com/cperciva
Differential Revision:	https://reviews.freebsd.org/D32250
2021-10-03 14:54:09 -07:00
Colin Percival
b841148bbb loader: Refactor readahead adjustment in bcache
While I'm here, add an explanatory comment.

No functional change intended.

Reviewed by:	imp, tsoome (previous version)
MFC after:	1 week
Sponsored by:	https://patreon.com/cperciva
Differential Revision:	https://reviews.freebsd.org/D32249
2021-10-03 12:10:36 -07:00
Colin Percival
ce73f768b7 EFI loader: Don't free bcache for DEVT_DISK devs
Booting on an EC2 c5.xlarge instance, this reduces the number of I/Os
performed from 609 to 432, reduces the total number of blocks read
from 61963 to 60797, and reduces the time spent in the loader by 39 ms.

Note that b4cb3fe0e3 allowed the bcache to be retained for most of
the boot process, but relies on mounting filesystems; this commit
allows the bcache to be retained at the start of the boot process,
before the root filesystem has been located.

Reviewed by:	imp, tsoome
MFC after:	1 week
Sponsored by:	https://www.patreon.com/cperciva
Differential Revision:	https://reviews.freebsd.org/D32239
2021-09-30 14:48:14 -07:00
Warner Losh
8ea95b2fba loader.efi: remove extra extern ST
The definition for 'ST' is in efilib.h, so we don't need extern ST here.

Sponsored by:		Netflix
Reviewed by:		tsoome, kevans
Differential Revision:	https://reviews.freebsd.org/D32225
2021-09-29 20:07:13 -06:00
Warner Losh
9aa29457d5 loader_lua.8: Fix first version
Lua bindings appeared in FreeBSD 12.0. Delete the authors section of the
man page, since it's unclear who wrote different parts of the man
page.

Noted by:	Trond Endrestol
Sponsored by:	Netflix
2021-09-29 17:18:51 -06:00
Warner Losh
9e1dc7bec3 loader: create separate man pages for each of the loaders
Create a man page per loader. Loader(8) will have information common to
all of them, while loader_${INTERP}(8) will have information relevant to
that specific loader. Rewrite loader(8) to give an overview and point to
the appropriate man page. Rewrite each of the loader_${INTER}(8) man
pages to contain only the relevant information to that loader. Put all
the common commands, environment variables, etc in loader_simp(8) and
refernce that from the loader_lua or loader_4th man pages. The
loader_lua(8) could use more details about the Lua
integration. Additional organization may be benefitial.

Sponsored by:		Netflix
Differential Revision:	https://reviews.freebsd.org/D31340
2021-09-29 09:24:47 -06:00
Colin Percival
7457840230 loader: Set twiddle globaldiv to 16 by default
Booting FreeBSD on an EC2 c5.xlarge instance, the loader "twiddles"
810 times over the course of 510 ms, a rate of 1.59 kHz. Even accepting
that many systems are slower than this particular VM and will take
longer to boot (especially if using spinning-rust disks), this seems
like an unhelpfully large amount of twiddling when compared to the
~60 Hz frame rate of many displays; printing the twiddles also consumes
roughly 10% of the boot time on the aforementioned VM.

Setting the default globaldiv to 16 dramatically reduces the time spent
printing twiddles to the console while still twiddling at roughly 100
Hz; this should be ample even for systems which take longer to boot and
consequently twiddle slower.

Note that this can adjusted via the twiddle_divisor variable in
loader.conf, but that file is not processed until nearly halfway
through the loader's runtime.

Reviewed by:	allanjude, jrtc27, kevans
MFC after:	1 week
Sponsored by:	https://www.patreon.com/cperciva
Differential Revision:	<https://reviews.freebsd.org/D32163>
2021-09-28 15:24:02 -07:00
Colin Percival
19e4f2f289 efi loader: Call tslog_init from efi_main
We were calling tslog_init from main; no reason to wait that long.

Fixes:		f49381ccb6 efi/loader: Call tslog_init
Sponsored by:	https://www.patreon.com/cperciva
2021-09-25 12:06:07 -07:00
Colin Percival
0a35c4b3ca loader printf: Profile with TSLOG
Now that the loader tslog code doesn't call printf, we can profile
printf using TSLOG.  On an EC2 c5.xlarge instance, we spend roughly
45 ms here (out of roughly 500 ms), presumably due to the time spent
writing output to the console.

MFC after:	1 week
Sponsored by:	https://www.patreon.com/cperciva
2021-09-24 20:23:49 -07:00
Colin Percival
242923eb84 loader tslog: Don't use sprintf
Instead, append the log entry "manually".

MFC after:	1 week
Sponsored by:	https://www.patreon.com/cperciva
2021-09-24 20:23:37 -07:00
Toomas Soome
1a25c51e38 loader: dev_net.c should use __func__ with printf
We have printf calls with function name hardwired to string,
sometimes wrong name. Use __func__ instead.

MFC after:	1 week
2021-09-24 17:12:19 +03:00
Leandro Lupori
680ca73945 powerpc64: fix loader regression
After b4cb3fe0e3, loader started crashing on PowerPC64, with a
Program Exception (700) error. The problem was that archsw was
used before being initialized, with the new mount feature. This
change fixes the issue by initializing archsw earlier, before
setting currdev, that triggers the mount.

Reviewed by:		tsoome
MFC after:		1 month
X-MFC-With:		b4cb3fe0e3
Sponsored by:		Instituto de Pesquisas Eldorado (eldorado.org.br)
Differential Revision:	https://reviews.freebsd.org/D32027
2021-09-21 17:22:42 -03:00
Mark Johnston
0e3ce6d082 efi loader: Typo
MFC after:	3 days
2021-09-21 12:09:55 -04:00
Colin Percival
5a01dea7e8 style: Fix leading whitespace in bcache.c
MFC after:	2 weeks
X-MFC-with:	Further bcache changes to come
2021-09-19 15:24:00 -07:00
Leandro Lupori
a58abcde2c powerpc64: change CAS to support Radix MMU
Use radix_mmu environment variable to select between Hash or Radix
MMU, when performing the CAS method call. This matches kernel's
behavior, by selecting Hash MMU by default and Radix if radix_mmu
is not zero, to make sure that both loader and kernel always select
the same MMU.

The device tree is queried to detect Radix/GTSE support and to
find out if CAS is supported, making the old CPU version and HV
bit checks unnecessary now.

Reviewed by:		jhibbits
MFC after:		2 weeks
Sponsored by:		Instituto de Pesquisas Eldorado (eldorado.org.br)
Differential Revision:	https://reviews.freebsd.org/D31951
2021-09-15 15:24:40 -03:00
Kyle Evans
ed107537b4 lualoader: use more concise verbiage for autoboot
The behavior remains the same, but lualoader now uses the more concise
verbiage that forthloader used.  This is particularly important because
the previous line would exceed the right boundary of the menu and run
straight into space that would typically be allowed for the logo.

This makes it slightly easier to port logos from forthloader to
lualoader.
2021-09-09 02:01:50 -05:00
Kyle Evans
e19d9a9b6e lualoader: allow brands to specify a shift
Some brands will want to specify a shift to make sure they're properly
centered; let it happen.
2021-09-09 02:01:50 -05:00
Kyle Evans
763bcebe0b lualoader: remove shadowed local from graphics:drawitem()
for loop vars are local already and distinct from this earlier
declaration; remove it.
2021-09-09 02:01:50 -05:00
Toomas Soome
b4cb3fe0e3 loader: implement mount/unmount rootfs
We want to keep our root file system open to preserve bcache segment
between file accesses, thus reducing physical disk IO.

Reviewed by:	imp, allanjude, kevans (previous version)
Differential Revision:	https://reviews.freebsd.org/D30848
MFC after:	1 month
2021-09-08 04:01:20 +03:00
Dimitry Andric
c90cab0d66 i386 loaders: avoid lld 13 garbage collecting linker sets
Because lld 13 and higher default to garbage collecting start/stop
symbols when using --gc-sections, the linker sets used in the i386 boot
loaders will disappear. This leads to the loaders not recognizing any
commands, and failure to boot.

Until we have a good set of linker scripts for the loaders, work around
it by disabling the start-stop-gc feature.

MFC after:	1 week
2021-09-08 14:04:13 +02:00
Toomas Soome
4c7a3a70e0 loader.efi: fix console output after BS off
When Boot Services (BS) are switched off, we can not use BS
functions any more. Since drawn console does implement our own
Blt(), we can use it to draw the console.

However, SimpleTextOutput protocol based console output must be
blocked.

Tested by inserting printf() after ExitBootServices() call.

MFC after:	1 week
2021-09-03 00:48:56 +03:00
Gordon Bergling
005fe24f2a libsa: Fix a typo in source code comments
- s/mininum/minimum/

MFC after:	3 days
2021-08-29 10:09:58 +02:00
Konstantin Belousov
b54eec8366 efi loader: disallow user to configure staging area size less than default
We need to round it up to 2M, for instance.  Having staging area too small
might cause the first resize to use negative size for memmove()/memcpy(),
which kills loader.

Tested by:	Harry Schmalzbauer <freebsd@omnilan.de>
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2021-08-27 19:49:01 +03:00
Konstantin Belousov
b850806921 Restore the definition of EFI_STAGING_SIZE
The definition can be overridden by users, and before f75caed644 it
was in MBs.  Make the symbol' unit MB, to be compatible with users
customizations.

Reported and tested by:	Harry Schmalzbauer <freebsd@omnilan.de>
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2021-08-27 19:48:53 +03:00
Kyle Evans
3daa8e165c pxeboot: improve and simplify rx handling
This pushes the bulk of the rx servicing into a single loop that's only
slightly convoluted, and it addresses a problem with rx handling in the
process.  If we hit a tx interrupt while we're processing, we'd
previously drop the frame on the floor completely and ultimately
timeout, increasing boot time on particularly busy hosts as we keep
having to backoff and resend.

After this patch, we don't seem to hit timeouts at all on zoo anymore
though loading a 27M kernel is still relatively slow (~1m20s).

Reviewed by:	tsoome
Triage by:	Ash Gokhale <ashfixit gmail com>
Sponsored By:	National Bureau of Economic Research
Sponsored by:	Klara, Inc.
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D31512
2021-08-25 21:59:08 -05:00
Konstantin Belousov
6032b6ba95 amd64 UEFI loader: enable automatic disable of staging area copying
Discussed with:	emaste
Sponsored by:	The FreeBSD Foundation
MFC after:	1 month
2021-08-25 22:26:52 +03:00
Toomas Soome
7b0d05d56d loader: loader_lua can run command_more twice
When we quit pager, the return value 1 is returned and command_more()
interprets it as error.

when lua loader gets error from command, it will try to
interpret it once more, so we get the same file shown once more.

There is no reason why we should return error from command_more().

MFC after:	1 week
2021-08-21 21:28:54 +03:00
Toomas Soome
e5a50b0329 loader: FB console does leave garbage on screen while scrolling
Scrolling screen will leave "trail" of chars from first column.
Apparently caused by cursor location mismanagement.
Make sure we do not [attempt to] set cursor out of the screen.

MFC after:	1 week
2021-08-21 19:17:25 +03:00
Maxim Sobolev
0d13f5343f Only trigger read-ahead if two adjacent blocks have been requested.
The change makes block caching algorithm to work better for remote
media on low-BW/high-delay links.

This cuts boot time over IP KVMs noticeably, since the initialization
stage reads bunch of small 4th (and now lua) files that are not in
the same cache stripe (usually), thus wasting lot of bandwidth and
increasing latency even further.

The original regression came in 2017 with revision 87ed2b7f5. We've
seen increase of time it takes for the loader to get to the kernel
loading from under a minute to 10-15 minutes in many cases.

Reviewed by:	tsoome
MFC after:	1 month
Differential Revision:	https://reviews.freebsd.org/D31623
2021-08-20 14:08:01 -07:00
Toomas Soome
3ec0714d6d libsa: cstyle cleanup of dosfs.c
No functional changes intended.

MFC after:	1 week
2021-08-19 17:36:25 +03:00
Gordon Bergling
fa7a635f7e Fix a few typos in source code comments
- s/becase/because/

MFC after:	5 days
2021-08-14 09:06:09 +02:00
Toomas Soome
5d5a621664 loader: cstyle cleanup of userboot/devicename.c
No functional changes intended.

MFC after:	1 week
2021-08-11 10:13:46 +03:00
Roger Pau Monné
5e4279a8f3 loader: fix multiboot loading on UEFI
The Xen kernel has no symbol tables, so calling lookup_symbol against
it triggers the following Divide by Zero fault:

Loading Xen kernel...
/boot/xen data=0x2809c8+0x149638 |
!!!! X64 Exception Type - 00(#DE - Divide Error)  CPU Apic ID - 00000000 !!!!

Fix lookup_symbol to prevent the #DE fault from happening if the
symbol table is not loaded and also fix loadfile_raw to mark multiboot
kernels as relocatable, since the only multiboot kernel supported is
Xen and was already unconditionally booted as relocatable.

Fixes: f75caed644 ('amd64 UEFI loader: stop copying staging area to 2M physical')
Reviewed by: imp, kib
Differential Revision: https://reviews.freebsd.org/D31507
2021-08-12 09:18:33 +02:00
Toomas Soome
97cbd5e722 loader: open file list should be dynamic
Summary:
Open file list is currently created as statically allocated array (64 items).
Once this array is filled up, loader will not be able to operate with files.
In most cases, this mechanism is good enough, but the problem appears, when
we have many disks with zfs pool(s). In current loader implementation, all
discovered zfs pool configurations are kept in memory and disk devices open -
consuming the open file array. Rewrite the open file mechanism to use
dynamically allocated list.

Reviewed by:	imp
MFC after:	2 weeks
Differential Revision: https://reviews.freebsd.org/D31364
2021-08-10 21:54:32 +03:00
Warner Losh
879675e9a0 stand: Add MK_PIE=no to defs.mk
There's no need to build both pie and non-pie .o's for stand. There's
some other build thing with MK_BEAR_SSL=yes and/or MK_LOADER_VERIEXEC=yes
that causes the pie build to fail that the 'ar' stage now. Since we don't
need the PIE stuff and the non-PIE stuff, disable PIE for the boot loader.

Reviewed by:	emaste
Sponsored by:	Netflix
2021-08-11 11:03:19 -06:00
Konstantin Belousov
f75caed644 amd64 UEFI loader: stop copying staging area to 2M physical
On amd64, add a possibility to activate kernel with staging area in place.
Add 'copy_staging' command to control this.  For now, by default the
old mode of copying kernel to 2M phys is retained.  It is going to be
changed in several weeks.

On amd64, add some slop to the staging area to satisfy both requirements
of the kernel startup allocator, and to have space for minor staging data
increase after the final size is calculated.  Add a new command
'staging_slop' to control its size.

Improve staging area resizing, in particular, reallocate it anew if
we cannot grow it neither down nor up.

Reviewed by:	kevans, markj
Discussed with:	emaste (the delivery plan)
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D31121
2021-08-08 22:52:29 +03:00
Mitchell Horne
61ed578ee6 Prefer MK_SSP=no to SSP_CFLAGS=
It is more idiomatic. CFLAGS is only augmented with $SSP_CFLAGS when
$MK_SSP != "no".

Reviewed by:	imp
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D31401
2021-08-04 15:23:22 -03:00
Alex Richardson
5e9226f063 Fix build of stand/ when building world with ASAN
The userboot/test  program links against the default userspace libraries
(e.g. shared libgcc_s.so) that will be instrumented if WITH_ASAN is set.
All other programs link against libsa instead of libc and therefore can't
use the sanitizer runtime library. To fix the stand/ build with
sanitizers, we disable MK_ASAN/MK_UBSAN if -nostdlib is found in the
LDFLAGS (i.e. we are using libsa instead of libc).

Reviewed By:	imp, tsoome
Differential Revision: https://reviews.freebsd.org/D31047
2021-08-02 14:33:24 +01:00
Toomas Soome
1b1bb6f178 loader: tftp client should use server address from rootip
servip is set from bootp bp_siaddr (if present) and rootip is
set immediately from servip in tha sane bootp code.

However, the common/dev_net.c does only set rootip (based on
url processing etc). Therefore, we should also use rootip in tftp
reader.

Fixes hung tftp based boot when bp_siaddr is not provided.

MFC after: 1 week
2021-08-02 15:43:54 +03:00
Toomas Soome
bbb539b83c loader: cstyle cleanup of libsa/lseek.c
Clean up lseek.c, no functional changes intended. This is pre-patch
for open file list rewrite.

MFC after:	1 week
2021-08-01 10:11:17 +03:00
Li-Wen Hsu
de0c7fbe28
loader.conf(5): mention "efi" option for "console" parameter
PR:		213467
Reviewed by:	imp
MFC after:	3 days
Differential Revision: https://reviews.freebsd.org/D31368
2021-08-01 06:41:49 +08:00
Warner Losh
6b51baf6a8 cli.lua.8: make the command match the code
It's disable-device, not device-disable

Spotted by:		jrtc27
Sponsored by:		Netflix
2021-07-28 20:54:14 -06:00
Warner Losh
07c4b78d0a lua loader: Add disable-device to disable a device.
disable-device fooX will set hint.foo.X.disabled=1 as a way to easily
disable a device attaching during boot.

Reviewed by:		tsoome
Sponsored by:		Netflix
Differential Revision:	https://reviews.freebsd.org/D31297
2021-07-28 16:53:00 -06:00
John Hood
dbdf2b52f5 loader: support.4th resets the read buffer incorrectly
Large nextboot.conf files (over 80 bytes) are not read correctly by the
Forth loader, causing file parsing to abort, and nextboot configuration
fails to apply.

Simple repro:

nextboot -e foo=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
shutdown -r now

That will cause the bug to cause a parse failure but shouldn't otherwise
affect the boot.  Depending on your loader configuration, you may also
have to set beastie_disable and/or reduce the number of modules loaded
to see the error on a small console screen.  12.0 or CURRENT users will
also have to explicitly use the Forth loader instead of the Lua loader.
The error will look something like:

Warning: syntax error on file /boot/loader.conf.local
foo="xxxxxxxxxxxxxxnextboot_enable="YES"
                                    ^
/boot/support.4th has crude file I/O buffering, which uses a buffer
'read_buffer', defined to be 80 bytes by the 'read_buffer_size'
constant.  The loader first tastes nextboot.conf, reading and parsing
the first line in it for nextboot_enable="YES".  If this is true, then
it reopens the file and parses it like other loader .conf files.

Unfortunately, the file I/O buffering code does not fully reset the
buffer state in the reset_line_reading word.  If the last file was read
to the end, that doesn't matter; the file buffer is treated as empty
anyway.  But in the nextboot.conf case, the loader will not read to the
end of file if it is over 80 bytes, and the file buffer may be reused
when reading the next file.  When the file is reread, the corrupt text
may cause file parsing to abort on bad syntax (if the corrupt line has
<>2 quotes in it), the wrong variable to be set, no variable to be set
at all, or (if the splice happens to land at a line ending) something
approximating normal operation.

The bug is very old, dating back to at least 2000 if not before, and is
still present in 12.0 and CURRENT r345863 (though it is now hidden by
the Lua loader by default).

Suggested one-line attached.  This does change the behavior of the
reset_line_reading word, which is exported in the line-reading
dictionary (though the export is not documented in loader man pages).
But repo history shows it was probably exported for the PNP support
code, which was never included in the loader build, and was removed 5
months ago.

One thing that puzzles me: how has this bug gone unnoticed/unfixed for
nearly 2 decades?  I find it hard to believe that nobody's tried to do
something interesting with nextboot, like load a kernel and filesystem,
which is what I'm doing.

Tested by:		Gary Jennejohn
PR:			239315
MFC After:		3 weeks
Reviewed by:		imp (and correctly applied this time)
Differential Revision:	https://reviews.freebsd.org/D31328
2021-07-28 13:50:38 -06:00
Warner Losh
4783fb730f Revert "loader: support.4th resets the read buffer incorrectly"
This reverts commit 9c1c02093b. It seems
to have broken all old nextboot.conf files causing hangs on boot.

Sponsored by:		Netflix
2021-07-26 16:40:41 -06:00
Warner Losh
2b720db8d4 type: becauce -> because
Noticed by:	Piotr P. Stefaniak
Sponsored by:	Netflix
2021-07-21 20:03:35 -06:00