Commit Graph

213 Commits

Author SHA1 Message Date
Toomas Soome
d36341f7b8 loader: we should support pools without features
nvlist_check_features_for_read() does return error when there
are no features for read.

MFC after: 5 days
2021-04-04 02:01:03 +03:00
Alex Richardson
59b2caef05 libsa: Remove conflicting .global/.weak directive
LLVM12 complains if you change the symbol binding:
`error: _longjmp changed binding to STB_GLOBAL`
In this case LLVM actually ignored the weak directive and used the
later .global, but GNU as would mark the symbol as weak.
None of the other architectures mark the libsa _setjmp as weak so
just drop this directive.
2021-03-30 14:59:41 +01:00
Ed Maste
7f72497ef7 libc: Use musl's optimized strchr and strchrnul
Parentheses added to HASZERO macro to avoid a GCC warning, and formatted
with clang-format as we have adopted these and don't consider them
'contrib' code.

Obtained from:	musl (snapshot at commit 4d0a82170a25)
Reviewed by:	kib (libc integration), mjg (both earlier)
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D17630
2021-03-01 21:09:59 -05:00
Martin Matuska
ba27dd8be8 zfs: merge OpenZFS master-9312e0fd1
Notable upstream changes:
  778869fa1 Fix reporting of mount progress
  e7adccf7f Disable use of hardware crypto offload drivers on FreeBSD
  03e02e5b5 Fix checksum errors not being counted on repeated repair
  64e0fe14f Restore FreeBSD resource usage accounting
  11f2e9a49 Fix panic if scrubbing after removing a slog device

MFC after:	2 weeks
2021-02-22 13:01:17 +01:00
Simon J. Gerraty
b14cfecbf0 pkgfs_open: follow symlinks
Caller is not interested in symlinks follow them.
Throw an error if too many links encountered.

Reviewed by:	stevek
Sponsored by:	Juniper Networks
--This line, and those below, will be ignored--
> Description of fields to fill in above:                     76 columns --|
> PR:                       If a GNATS PR is affected by the change.
> Differential Revision:    https://reviews.freebsd.org/D### (*full* phabric URL needed).
> Submitted by:             If someone else sent in the change.
> Reviewed by:              If someone else reviewed your modification.
> Approved by:              If you needed approval for this commit.
> Obtained from:            If the change is from a third party.
> MFC after:                N [day[s]|week[s]|month[s]].  Request a reminder email.
> MFH:                      Ports tree branch name.  Request approval for merge.
> Relnotes:                 Set to 'yes' for mention in release notes.
> Security:                 Vulnerability reference (one per line) or description.
> Sponsored by:             If the change was sponsored by an organization.
> Empty fields above will be automatically removed.

Change-Id: I4ef92ff256f503c620dc5bba79ed93b32cb2330d
2021-01-14 17:33:05 -08:00
Toomas Soome
2ae5841472 libsa: xdrproc_t should return bool
Since our xdr translation function are returning bool, so should
xdrproc_t.

Issue reported by gcc 10 build.
2020-12-27 23:25:18 +02:00
Toomas Soome
d3eae37553 loader: allow loader to accept zfs feature com.delphix:bookmark_written
We do not need to actually read bookmarks, just whitelist this feature.

Reported by:	mjg
2020-12-15 22:37:03 +00:00
Toomas Soome
eba724f361 loader: allow loader to accept zfs feature com.datto:bookmark_v2
We do not need to actually read bookmarks, just whitelist this feature

Reported by:	jpaetzel
2020-12-15 21:48:35 +00:00
Toomas Soome
9de6a13e0f loader: xdr_array is missing count
The integer arrays are encoded in nvlist as counted array <count, i0, i1...>,
loader xdr_array() is missing the count. This will affect the pool import when
there are hole devices in pool.

Also fix the new data add and print functions.
2020-12-07 11:25:18 +00:00
Conrad Meyer
78599c32ef Add CFI start/end proc directives to arm64, i386, and ppc
Follow-up to r353959 and r368070: do the same for other architectures.

arm32 already seems to use its own .fnstart/.fnend directives, which
appear to be ARM-specific variants of the same thing.  Likewise, MIPS
uses .frame directives.

Reviewed by:	arichardson
Differential Revision:	https://reviews.freebsd.org/D27387
2020-12-05 00:33:28 +00:00
Warner Losh
1ef64e3da5 loader: add aarch64 support for zstd
We don't have NEON available in the boot loader, so we have to disable
it. OpenZFS included ZSTD which used the wrong symbol to bring in neon
support. Change to use the code that's been submitted upstream as a
pull request to both.

__ARM_NEON is the proper symbol, defined in ARM C Language Extensions
Release 2.1 (https://developer.arm.com/documentation/ihi0053/d/). Some
sources suggest __ARM_NEON__, but that's the obsolete spelling from
prior versions of the standard.

OpenZFS Pull Request: https://github.com/openzfs/zfs/pull/11055
ZSTD Pull Request: https://github.com/facebook/zstd/pull/2356
2020-10-28 21:18:04 +00:00
Toomas Soome
e416eecbb9 loader: revert r342161 and r342151
We are using asize property from pool label and we do not depend
on partition data to find last two pool labels and to validate LBA for disk IO.

This does allow us to re-enable support for partitionless disk setups.
2020-10-22 20:02:02 +00:00
Warner Losh
e694696956 Force __BMI__ experimental instructions off.
The OpenZFS code that uses the BMI instructions is broken. Forcibly
disable them to prevent their use. When enabled, the build breaks.
This fixes the build when compiled for a core with BMI instructions.
This is the same fix committed in r364777, for the same issue.

Submitted by: Jung-uk Kim
2020-10-13 04:37:57 +00:00
Warner Losh
df13933e33 Add back org.freebsd:zstd_compress to features_for_read
This list is the of features that are allowed on the whole pool,
not the list of features that are implemented.
2020-10-13 03:49:12 +00:00
Warner Losh
9257c69b1c Turn off zstd on aarch64
loader support for zstd and zfs doesn't work for aarch64. Disable it
to unbreak the build.
2020-10-13 02:36:16 +00:00
Warner Losh
2fec3ae896 Add zstd support to the boot loader.
Add support to the _STANDALONE environment enough bits of the kernel
that we can compile it. We still have a small zstd_shim.c since there
were 3 items that were a bit hard to nail down and may be cleaned up
in the future. These go hand in hand with a number of commits to
sys/sys in the past weeks, should this need be MFCd.

Discussed with: mmacy (in review and on IRC/Slack)
Reviewed by: freqlabs (on openzfs repo)
Differential Revision: https://reviews.freebsd.org/D26218
2020-10-12 22:19:07 +00:00
Warner Losh
6ab1ffcbfe Implement some time variables from kernel
OpenZFS will start using some of the kernel timekeeping bits
shortly. This implements the bare minimum of that which currently
is just the time_seconds variable.
2020-09-29 16:29:50 +00:00
Toomas Soome
867ae3c38d loader: zfs_probe_dev should pick first matching zfs pool
During devswitch probe, we pick boot pool based on boot disk, if the boot
disk happens to have multiple pools in freebsd-zfs partitions, the current
code does pick last pool from boot disk as boot pool. While there is no
way at that stage to test, the more logical approach would be to pick
first matching pool.

This patch is assuming we do pass pool guid pointer with guid value 0,
this will help us to determine, if the guid value is already set or not.

The general suggestion would be not to share disk between different pools.

Reported by:	Alexander Leidinger
2020-09-23 08:22:14 +00:00
Warner Losh
a5ebda464e Work around cp breakage in current from last week
There was a small window cp was broken. Work around this by using :>
instead of cp /dev/null. Ideally, we'd keep the cp /dev/null in the
build as a regression test, but doing so breaks people that upgraded
during the cp breakage and this is simpler than bootstrapping a
working cp since there's no good __FreeBSD_version sign posts for
that.

Suggested by: lots of people
Too stubborn for his own good: imp
2020-09-23 01:04:25 +00:00
Toomas Soome
e307eb94ae loader: zfs should support bootonce an nextboot
bootonce feature is temporary, one time boot, activated by
"bectl activate -t BE", "bectl activate -T BE" will reset the bootonce flag.

By default, the bootonce setting is reset on attempt to boot and the next
boot will use previously active BE.

By setting zfs_bootonce_activate="YES" in rc.conf, the bootonce BE will
be set permanently active.

bootonce dataset name is recorded in boot pool labels, bootenv area.

in case of nextboot, the nextboot_enable boolean variable is recorded in
freebsd:nvstore nvlist, also stored in boot pool label bootenv area.
On boot, the loader will process /boot/nextboot.conf if nextboot_enable
is "YES", and will set nextboot_enable to "NO", preventing /boot/nextboot.conf
processing on next boot.

bootonce and nextboot features are usable in both UEFI and BIOS boot.

To use bootonce/nextboot features, the boot loader needs to be updated on disk;
if loader.efi is stored on ESP, then ESP needs to be updated and
for BIOS boot, stage2 (zfsboot or gptzfsboot) needs to be updated
(gpart or other tools).

At this time, only lua loader is updated.

Sponsored by:	Netflix, Klara Inc.
Differential Revision:	https://reviews.freebsd.org/D25512
2020-09-21 09:01:10 +00:00
Simon J. Gerraty
ed19b7c525 zalloc_malloc:Free hexdump preceeding buffer when we detect overflow
Move hexdump from stand/common/misc.c to stand/libsa/hexdump.c
(svn cp)
Disable use of pager - causes linking issue for boot1
can be re-enabled by defining HEXDUMP_PAGER.

Reviewed by:	stevek, imp
MFC after:	1 week
Sponsored by:	Juniper Networks
Differential Revision: https://reviews.freebsd.org/D26235
2020-08-29 21:05:43 +00:00
Warner Losh
b9c5b43263 Declare time()
Time is used and was accidentally brought in through header
pollution. Declare it in stand.h directly instead.
2020-08-28 05:40:02 +00:00
Matt Macy
962619075c ZFS: remove duplicate "com.datto:encryption" from loader 2020-08-27 21:37:35 +00:00
Toomas Soome
0fec8f03cf libsa: only skein_block.c is using SKEIN_LOOP
Only use SKEIN_LOOP while compiling skein_block.c
2020-08-26 17:52:32 +00:00
Matt Macy
3e1470776b ZFS: whitelist zstd and encryption in the loader
Please note that neither zstd nor encryption is
supported by the loader at this instant. This
change makes it safe to use those features in
one's root pool, but not in one's root dataset.
2020-08-25 23:26:52 +00:00
Emmanuel Vadot
9060f2c392 libsa: smbios: Parse the chassis type and export it as smbios.chassis.type
It can be useful to know what type of machine we are running on for desktop
related thing.
It also allow us to support all the DMI variable that linux driver can fetch.

MFC after:	1 week
Sponsored by:	Sponsored-by: The FreeBSD Foundation
2020-08-20 12:50:49 +00:00
Toomas Soome
5949d13fab libsa: remove leftover whitespace
Tiny cleanup, no functional changes.
2020-08-19 20:41:22 +00:00
Toomas Soome
588f0a1e29 libsa: make env_discard() public
Allow env_discard() to be used outside environment.c
2020-08-19 15:27:09 +00:00
Toomas Soome
16aabe28c6 libsa: cstyle cleanup for environment.c
No functional changes.
2020-08-19 15:20:33 +00:00
Mariusz Zaborski
277f38abff zfs: add an option to the bootloader to rewind the ZFS checkpoint
The checkpoints are another way of keeping the state of ZFS.
During the rewind, the pool has to be exported.
This makes checkpoints unusable when using ZFS as root.
Add the option to rewind the ZFS checkpoint at the boot time.
If checkpoint exists, a new option for rewinding a checkpoint will appear in
the bootloader menu.
We fully support boot environments.
If the rewind option is selected, the boot loader will show a list of
boot environments that existed before the checkpoint.

Reviewed by:	tsoome, allanjude, kevans (ok with high-level overview)
Differential Revision:	https://reviews.freebsd.org/D24920
2020-08-18 19:48:04 +00:00
Toomas Soome
de776da323 loader: implement GELI writes
Bug: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=247482

This patch is based on initial work from allanjude.

PR:		247482
Obtained from:	https://reviews.freebsd.org/D10236
Differential Revision:	https://reviews.freebsd.org/D25605
2020-07-11 06:51:42 +00:00
Toomas Soome
1a4b982e1e loader: setting vdev size based on label asize is not working
Because we need to read asize from vdev_tree. We also need to consider
different vdev type difference.

Reviewed by:	allanjude
Sponsored by:	Netflix, Klara Inc.
Differential Revision:	https://reviews.freebsd.org/D25586
2020-07-09 13:19:00 +00:00
Toomas Soome
0d1a620681 loader: geli_dev_ioctl does return huge mediasize
The DIOCGMEDIASIZE is calculated md->md_sectorsize * md->md_provsize, and
for boot disk, the md_sectorsize is 4k. However, the md_provsize is already
in units of bytes.

Sponsored by:	Netflix, Klara Inc.
2020-07-07 12:24:40 +00:00
Toomas Soome
771f100ceb loader: zfs reader does not need BOOT2 bits
After switching zfsloader to use full libsa, we do not need
spa_get_primary() and spa_get_primary_vdev() any more.

Sponsored by:	Netflix, Klara Inc.
2020-07-02 07:15:48 +00:00
Toomas Soome
045f497cbc loader: potential memory leak and check return values
Need to free nvlist before return from vdev_from_nvlist().

Sponsored by:	Netflix, Klara Inc.
2020-07-02 07:03:15 +00:00
Toomas Soome
b93b14dc75 loader: can not read zfs pool with slog removed
The vdev_init() does check for "known" vdev types, the [log] device removal will create "hole"
device, but vdev_init() does not allow it.

Obtained from: illumos
MFC after:	1 week
2020-06-26 21:21:35 +00:00
Toomas Soome
3830659e99 loader: create single zfs nextboot implementation
We should have nextboot feature implemented in libsa zfs code.
To get there, I have created zfs_nextboot() implementation based on
two sources, our current simple textual string based approach with added
structured boot label PAD structure from OpenZFS.

Secondly, all nvlist details are moved to separate source file and
restructured a bit. This is done to provide base support to add nvlist
add/update feature in followup updates.

And finally, the zfsboot/gptzfsboot disk access functions are swapped to use
libi386 and libsa.

Sponsored by:	Netflix, Klara Inc.
Differential Revision:	https://reviews.freebsd.org/D25324
2020-06-20 06:23:31 +00:00
Kirk McKusick
34816cb9ae Move the pointers stored in the superblock into a separate
fs_summary_info structure. This change was originally done
by the CheriBSD project as they need larger pointers that
do not fit in the existing superblock.

This cleanup of the superblock eases the task of the commit
that immediately follows this one.

Suggested by: brooks
Reviewed by:  kib
PR:           246983
Sponsored by: Netflix
2020-06-19 01:02:53 +00:00
Toomas Soome
aba2397e74 loader: r362262 did miss the pathlen check
While we are checking the "/dev/" prefix, we can skip the paths shorter than
this prefix.

Sponsored by:	Netflix, Klara Inc.
2020-06-17 10:56:58 +00:00
Toomas Soome
c98740b5e2 loader: strings in nvlist are counted strings, not c-strings
We need to fetch both string size and data, then handle the data.

Reviewed by:	allanjude
Sponsored by:	Netflix, Klara Inc.
2020-06-17 10:41:01 +00:00
John Baldwin
4f98ffdd1d Fix libstand build breakage after r361298.
- Use enc_xform_aes_xts.setkey() directly instead of duplicating the code
  now that it no longer calls malloc().
- Rather than bringing back all of xform_userland.h, add a conditional
  #include of <stand.h> to xform_enc.h.
- Update calls to encrypt/decrypt callbacks in enc_xform_aes_xts for
  separate input/output pointers.

Pointy hat to:	jhb
2020-05-20 22:25:41 +00:00
Eric van Gyzen
fac6dee9eb Remove tests for obsolete compilers in the build system
Assume gcc is at least 6.4, the oldest xtoolchain in the ports tree.
Assume clang is at least 6, which was in 11.2-RELEASE.  Drop conditions
for older compilers.

Reviewed by:	imp (earlier version), emaste, jhb
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D24802
2020-05-12 15:22:40 +00:00
Toomas Soome
4a2d7cee02 loader: vdev_read() can corrupt memory
When reading less than sector size but from sector boundary,
the vdev_read() will read full sector into the provided buffer
and therefore corrupting memory past buffer end.

MFC after:	2 days
2020-05-09 06:25:20 +00:00
Adrian Chadd
6c88ef1c81 [libsa] Fix typecast of pointer for st_dev
This code was trying to use a pointer value for st_dev, which is definitely
not a pointer.  Instead, cast to uintptr_t so it becomes a non-pointer value
before casting it.

Tested: mips-gcc cross compile, mips32 build
2020-04-16 23:29:49 +00:00
Simon J. Gerraty
723f904176 Improve interaction of vectx and tftp
On slow platforms, it helps to spread the hashing load
over time so that tftp does not timeout.

Also, some .4th files are too big to fit in cache of pkgfs,
so increase cache size and ensure fully populated.

Reviewed by:	stevek
MFC after:	1 week
Differential Revision: https://reviews.freebsd.org/D24287
2020-04-07 16:56:34 +00:00
Toomas Soome
5292d0a7f3 loader: use #elif defined() in stand.h
Small cleanup.

Reported by:	imp
2020-03-29 06:48:59 +00:00
Toomas Soome
6c01b7102d loader: typo did slip into 359408
somehow r was lost from 'char'.
2020-03-28 21:55:42 +00:00
Toomas Soome
215597f05f loader: strdup name strings from dataset walker
The removal of zfs scratch buffer did miss the fact the dataset
lookup was picking up the names from zap list.
2020-03-28 21:50:27 +00:00
Toomas Soome
c907ec9e16 loader: add knob to build with user malloc
This option is intended to aid development, to allow building with user malloc.
The use case would be to build userboot & test with libc (or other) malloc and
use extra malloc debug features.
2020-03-28 21:47:44 +00:00
Simon J. Gerraty
53f151f906 Fix pkgfs stat so it satisfies libsecureboot
We need a valid st_dev, st_ino and st_mtime
to correctly track which files have been verified
and to update our notion of time.

ve_utc_set(): ignore utc if it would jump our current time
by more than VE_UTC_MAX_JUMP (20 years).

Allow testing of install command via userboot.
Need to fix its stat implementation too.

bhyveload also needs stat fixed - due to change to userboot.h

Call ve_error_get() from vectx_close() when hash is wrong.

Track the names of files we have hashed into pcr

For the purposes of measured boot, it is important
to be able to reproduce the hash reflected in
loader.ve.pcr
so loader.ve.hashed provides a list of names in the order they
were added.

Reviewed by:	imp
MFC after:	1 week
Sponsored by:	Juniper Networks
Differential Revision:	https://reviews.freebsd.org//D24027
2020-03-25 19:12:19 +00:00