Commit Graph

86 Commits

Author SHA1 Message Date
Warner Losh
8a744de27e stand: Remove i386-only support fire firewire
Remove support for booting off of firewire, and for having dcons via
firewire in the loader. Kernel support for these things is unchanged.
Discussed on arch@ and the current state is not working (and the build
was wrong to boot).

Sponsored by:		Netflix
Discussed:		https://lists.freebsd.org/archives/freebsd-arch/2022-November/000267.html
Reviewed by:		kevans, melifaro, emaste
Differential Revision:	https://reviews.freebsd.org/D37334
2022-11-18 10:43:59 -07:00
Warner Losh
ef8b872301 stand: Flesh out the zfs image creation
Move the mbr non-geli zfs cases to no-priv creation with makefs / mkimg.
Add comments about the weird thing we do for MBR + ZFS + Legacy.  Add
comments about other architectures. Still need to think through how to
leverage a completed universe to do all the architectures...

Sponsored by:		Netflix
2022-08-17 11:39:53 -06:00
Warner Losh
3d3f2fb069 stand: Make makeroot.sh use makefs for ZFS
Start to use makefs for ZFS. This covers the gpt nogeli variants.  ZFS
MBR booting is tricky and complicated, so will need some additional
tweaks that makefs/mkimg isn't able to do at the moment. This means that
all gpt nogeli amd64 combinations can be built w/o root.

In addition, tweak the generated qemu.sh files to use stdio for the
console. We grep the output for SUCCESS and report each of the booting
types. Create a all.sh that will run these automatically. These all can
also run w/o root.

In the future, I'll add support for a make univers followed by this
script to create other architectures' tests and/or generate stand tests
for /usr/tests...

Sponsored by: Netflix
2022-08-16 15:36:55 -06:00
Warner Losh
729c458bf7 stand: Abstract out fstab creation
Now that all the fstabs are the same, abstract it out into a function.

Sponsored by:		Netflix
2022-08-15 12:46:31 -06:00
Warner Losh
26a241a0e2 stand: Update GELI UFS cases to use ufs labels
GELI images are created in a different manner than non-GELI
images. Update them to set the label of 'root' on the UFS partition and
use that in fstab. Drop comments about needing to do this, as well as
the 'dev' variable now that it's unused.

Sponsored by:		Netflix
2022-08-15 12:38:32 -06:00
Warner Losh
159b158a1d stand: Update testing harness to start to use UFS labels
Minor changes to the non-geli UFS targets: migrate to using ufs labels
so we don't need to know the name of the device we booted off of. This
doesn't change the GELI test cass just yet since I've not tested them.
ZFS doesn't need these changes since we don't need to encode the device
for it.

Sponsored by:		Netflix
2022-08-15 12:38:32 -06:00
Warner Losh
aff95057bc stand: Update testing script for arch retirement
And catch up to changes in booting methods currently supported. Add
riscv.

Sponsored by:		Netflix
2022-08-15 12:38:32 -06:00
Warner Losh
08585e7799 tools/boot: Fix universe.sh's ZFS exclusion test
MK_LOADER_ZFS is the current spelling of enabling or disabling ZFS
builds. Use it instead of MK_ZFS.

Sponsored by:		Netflix
Reviewed by:		tsoome
Differential Revision:	https://reviews.freebsd.org/D35896
2022-07-24 16:53:34 -06:00
Warner Losh
9e9ba3cdbe test: Update boot universe build architectures
We build lua by default, so we don't need another build to build it
enabled w/o FORTH. That gives little value over the without forth
builds. Remove all mips, they are no longer relevant. Build aarch64
everywhere we build amd64 (except firewire which is x86 only). Build a
few more architectures once so we have at least one of every arch we
support in at least the default build. This should increase coverage
and still take less time than before.

Sponsored by:		Netflix
2022-07-08 11:47:37 -06:00
Jessica Clarke
17160f21ef install-boot.sh: Avoid - in function names for POSIX compatibility
FreeBSD sh supports this but other common POSIX shells do not; in
particular, dash does not, unlike bash and zsh. This allows the script
to be used on non-FreeBSD systems for release media building.

Reviewed by:	emaste, brooks
Differential Revision:	https://reviews.freebsd.org/D34000
2022-02-28 22:37:21 +00:00
Ed Maste
bba96bb143 CI: add arm64 support to ci-qemu-test.sh
Reviewed by:	imp (earlier)
MFC after:	3 weeks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D30907
2021-06-26 14:26:41 -04:00
Ed Maste
7d9794b34b CI: use amd64 EDK II firmware included with QEMU
QEMU (now) includes a prebuilt EDK II firmare in edk2-x86_64-code.fd.
Use that instead of requring a standalone uefi-edk2-qemu-x86_64 package.

Reviewed by:	imp
MFC after:	3 weeks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D30915
2021-06-26 14:22:48 -04:00
Kyle Evans
8eeeee38f4 tools: boot: use four jobs for building stand
Parallel builds of stand should be assumed both possible and safe as of
7012461c9b, so let's start using some jobs to speed up lualoader test
harness builds.
2021-01-31 10:11:41 -06:00
Kyle Evans
29842cb36e lualoader: fix lua-lint run
luacheck rightfully complains that i is unused in the show-module-options
loop at the end (it was used for some debugging in the process).

We've added a new pager module that's compiled in, so declare that as an
acceptable global.
2020-12-17 18:29:30 +00:00
Mitchell Horne
d46a2a0013 riscv: allow building virtual machine images
RISC-V has the same booting requirements as arm64 (loader.efi, no legacy
boot options), so generated images for both architectures have the same
partition layout.

Reviewed by:	gjb
Differential Revision:	https://reviews.freebsd.org/D27044
2020-12-08 00:37:11 +00:00
Warner Losh
4c4e2b3de3 Speciy the dev in an easily changed variable
Rather than hard coding ada0 everywhere, use ${dev}. Also, set
dev=vtbd0 since both qemu and bhyve support this. More work
should be done to use labels instead for fstab.

qemu scripts likely need adjustment. And we should also
likely generate byhve scripts too.
2020-09-28 06:00:56 +00:00
Ed Maste
08d0b468f1 ci-qemu-test.sh: use pkgbase
Reviewed by:	bcran (earlier), manu (earlier), imp
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D24276
2020-09-05 19:03:34 +00:00
Matt Macy
9e5787d228 Merge OpenZFS support in to HEAD.
The primary benefit is maintaining a completely shared
code base with the community allowing FreeBSD to receive
new features sooner and with less effort.

I would advise against doing 'zpool upgrade'
or creating indispensable pools using new
features until this change has had a month+
to soak.

Work on merging FreeBSD support in to what was
at the time "ZFS on Linux" began in August 2018.
I first publicly proposed transitioning FreeBSD
to (new) OpenZFS on December 18th, 2018. FreeBSD
support in OpenZFS was finally completed in December
2019. A CFT for downstreaming OpenZFS support in
to FreeBSD was first issued on July 8th. All issues
that were reported have been addressed or, for
a couple of less critical matters there are
pull requests in progress with OpenZFS. iXsystems
has tested and dogfooded extensively internally.
The TrueNAS 12 release is based on OpenZFS with
some additional features that have not yet made
it upstream.

Improvements include:
  project quotas, encrypted datasets,
  allocation classes, vectorized raidz,
  vectorized checksums, various command line
  improvements, zstd compression.

Thanks to those who have helped along the way:
Ryan Moeller, Allan Jude, Zack Welch, and many
others.

Sponsored by:	iXsystems, Inc.
Differential Revision:	https://reviews.freebsd.org/D25872
2020-08-25 02:21:27 +00:00
Warner Losh
25dcf0e535 Spell LOADER correctly so we test lua build. 2020-05-01 17:50:26 +00:00
Warner Losh
ce6fd19966 Catch up to arm/arm and sparc64 removal. 2020-05-01 17:17:01 +00:00
Warner Losh
22ea5a7250 When we have an invalid build option, don't rm -rf the current
directory.

Add a quick sanity check to objdir before using it.  It must start
with /. If there was a make error getting it, report that and continue
with the next target. If there was anything else, bail out.
2020-05-01 17:16:57 +00:00
Ed Maste
f63070048b CI: print wired page count on boot
This is a first step in tracking kernel memory use over time.
2020-02-29 19:59:21 +00:00
Ed Maste
3e6b8ce30e ci-qemu-test: if firmware is not available, hint at pkg to install
uefi-edk2-qemu-x86_64 provides the firmware ci-qemu-test.sh expects to
use.

Sponsored by:	The FreeBSD Foundation
2019-09-09 14:51:25 +00:00
Matt Macy
14113f123e Use makefs -t msdos in make_esp_file
With this last piece in place, make -C /usr/src/release release.iso is
finally able to run in a jail. This was not possible before because
msdosfs cannot be mounted inside a jail.

Submitted by:	ryan@ixsystems.com
Reviewed by:	emaste@, imp@, gjb@
MFC after:	1 week
Sponsored by:	iXsystems, Inc.
Differential Revision:	https://reviews.freebsd.org/D21385
2019-09-03 18:37:55 +00:00
Ed Maste
dc93026776 revert QEMU q35 platform use from r346748
It seems to be incompatible with the OVMF.fd (of unknown provenance)
in use by the Cirrus-CI config.  We will soon have a known OVMF build
via a port/package (see review D19869) and we can switch back to q35
once packages are available.

Discussed with:	bcran
2019-04-30 15:28:52 +00:00
Rebecca Cran
c1ffbf2da0 Fix tools/boot/ci-qemu-test.sh and make some improvements
Update ci-qemu-test.sh

o Update the path to the OVMF file, which is now in /usr/local/share/uefi-edk2-qemu.
o Use the more modern q35, pc-q35-3.0 (Q35 + ICH9, 2009) QEMU machine
  instead of the default, obsolete pc, pc-i440fx-3.0 (i440FX + PIIX, 1996).
  For example this adds ACPI support.
o Specify the system firmware using the newer pflash drive syntax instead
  of bios.
o Remove extra, unneeded devices by passing -nodefaults.
o Change text to talk about 'firmware' instead of 'bios', since UEFI
  isn't a BIOS.

Reviewed by:	imp, emaste
Differential Revision:	https://reviews.freebsd.org/D20074
2019-04-26 14:44:46 +00:00
Ian Lepore
0c0ffba5d6 For the geli-gpt-zfs test images, both bios and uefi flavors, add a dummy
ufs partition as p2, and put the zfs partition at p3, to test the ability
of the zfs probe code to find a zfs pool on something other than the first
partition.
2019-04-25 00:08:15 +00:00
Ian Lepore
827f1973cb The zfs module has grown a dependency on zcl_nfs4.ko, so copy it into all
the test images.
2019-04-24 23:52:38 +00:00
Ian Lepore
f576caac25 Complain and exit the script if the 'make install' phase fails. Also,
there is no need to install any debug files.
2019-04-24 23:51:12 +00:00
Rebecca Cran
0661cd79ca Fix install-boot.sh and rootgen.sh UEFI bugs
tools/boot/install-boot.sh was assuming that if a device was passed in,
it should operate on the current system and run efibootmgr etc. to
update the boot manager. However, rootgen.sh passes a md(4) device and
not a fixed disk.

Add a -u option to install-boot.sh to tell it to update the system
in-place and run efibootmgr etc.

Also, source install-boot.sh in rootgen.sh to allow it to find and
call make_esp_file etc. And pass the loader file to make_esp_file instead
of a directory name.

Reported by:	ian
Reviewed by:	ian,imp,tsoome
Differential Revision:	https://reviews.freebsd.org/D19992
2019-04-24 22:35:29 +00:00
Ed Maste
79b02e7017 Put QEMU CI smoke test boot log in /tmp if TMPDIR not set
Sponsored by:	The FreeBSD Foundation
2019-04-17 19:24:41 +00:00
Ed Maste
297ce4ce2d Install some entropy for QEMU CI smoke test
See r346250 and followup commits and mailing list discussion.
We currently fail to boot properly in the absense of boot-time entropy.

Sponsored by:	The FreeBSD Foundation
2019-04-17 19:16:26 +00:00
Ian Lepore
2614c4835f Allow this test script to be run from within src/tools/boot dir, and create
the temporary image in $TMPDIR.

Allow the script to be run from the src/tools/boot directory by using make
-V SRCTOP to find the top of the tree, because this script is handy for
quick smoke-testing of loader changes, as well as being useful in CI testing.

Also, use a temp directory in $TMPDIR to assemble the boot image, and write
the boot log file to $TMPDIR.  Arrange to have the temporary image clean
itself up, but leave the log file in $TMPDIR for post-mortem analysis of
failures when the script is run interactively.

Differential Revision:	https://reviews.freebsd.org/D19876
2019-04-17 16:27:43 +00:00
Ed Maste
cc2e55c7ea Add a smoke test QEMU boot script for CI
Reviewed by:	imp, lwhsu (earlier)
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D19661
2019-04-10 13:41:34 +00:00
Rebecca Cran
db8b561345 Rework UEFI ESP generation
Currently, the installer uses pre-created 800KB FAT12 filesystems that
it dd's onto the ESP partition.
This changeset improves that by having the installer generate a FAT32
filesystem directly onto the ESP using newfs_msdos and then copying
loader.efi into /EFI/freebsd.
For live installs it then runs efibootmgr to add a FreeBSD boot entry
in the BIOS.

Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D17947
2018-12-20 19:39:37 +00:00
Kyle Evans
f98c671d07 Update lualoader test script a little bit
Use userboot.so from the test directory if possible, fall back to .OBJDIR.
This avoids a problem that we've had since userboot coexistence was added,
where userboot.so alone no longer exists in the .OBJDIR but is instead just
a link installed later.
2018-10-25 02:04:01 +00:00
Marius Strobl
e94a7c27df Following r335259, don't copy boot1 from the running system for sparc64
either.
2018-08-23 22:57:42 +00:00
Warner Losh
1c70f60e23 also set multicons for tests 2018-07-27 21:50:10 +00:00
Ian Lepore
dfe34cea0f Make generated scripts executable. 2018-07-13 16:37:42 +00:00
Ian Lepore
394641cf3d More little fixes... fix a function name typo (eps vs esp), and cope with
newer versions of gpart that show mbr efi partition types with the name
'efi' rather than as '!239'.
2018-07-13 16:33:21 +00:00
Ian Lepore
7b4b55e520 Revert r336240, which contained unrelated changes accidentally committed. 2018-07-13 16:32:31 +00:00
Ian Lepore
0d55ac1622 More little fixes... fix a function name typo (eps vs esp), and cope with
newer versions of gpart that show mbr efi partition types with the name
'efi' rather than as '!239'.
2018-07-13 16:30:54 +00:00
Ian Lepore
e74c8ac80c Wee fix: $dev is passed in to the function with the /dev prefix already in
place, don't add it.
2018-07-12 17:22:25 +00:00
Warner Losh
c89deb8fcc Small refactor of install-boot
Use newfs_msdos to create the filesystem for the partition requested
and use loader.efi instead of boot1.efi.
2018-07-11 21:53:04 +00:00
Warner Losh
405111fde0 Remove armeb from the list of architectures we build for the boot
laoder testing scripts. It just adds to the build time, and will be
retired soon anyway, per discussions in freebsd-arch@ last month.
2018-07-09 18:02:23 +00:00
Ian Lepore
c7940ca2ad Fix the fstab entries for tests that have an efi partition in slice 1 and
put the rootfs on s2.
2018-07-03 16:07:04 +00:00
Ian Lepore
28acc09dc0 Doh! Commit the proper default values, not my local tweaks. 2018-07-02 22:59:29 +00:00
Ian Lepore
595deaf7f1 Move the passphrase and iteration count into variables.
This is purely to make it easier to tweak them locally; the machine I have
for testing takes forever to do 50,000 pw strengthening iterations, and
we're not testing the strength of geli's anti-pw-guessing logic here
(especially given that our test passphrase is "passphrase", except that
I tend to tweak that also, to 'x', because typing is hard).

Some day these should be settable as cmdline args.  But then, some day this
whole script should probably get a rewrite. :)
2018-07-02 22:58:04 +00:00
Ian Lepore
432b49211c Allow making just one flavor of boot-test image based on cmdline args. 2018-07-02 22:51:48 +00:00
Warner Losh
d050cd6b99 Switch to creating UEFI ESPs using loader.efi instead of boot1.efi.
Cope for the fact that laoder.efi, not being boot1, doesn't read
/boot.config by setting boot_serial and force the serial console.

Also add sysctl so we can display the boot method.

Provide a variable, do_boot1_efi, if you want to use boot1 for
testing. But since it's transient, it's just a variable and not
available on the command line.
2018-06-18 23:16:47 +00:00