While toying around with lua bindings for libbe(3), I discovered that I
apparently never documented this, despite having documented
be_is_auto_snapshot_name that references it.
MFC after: 1 week
libbe will never need to mutate these as we either process them into a local
buffer or we just don't touch them and write to a separate out argument.
MFC after: 1 week
bootonce feature is temporary, one time boot, activated by
"bectl activate -t BE", "bectl activate -T BE" will reset the bootonce flag.
By default, the bootonce setting is reset on attempt to boot and the next
boot will use previously active BE.
By setting zfs_bootonce_activate="YES" in rc.conf, the bootonce BE will
be set permanently active.
bootonce dataset name is recorded in boot pool labels, bootenv area.
in case of nextboot, the nextboot_enable boolean variable is recorded in
freebsd:nvstore nvlist, also stored in boot pool label bootenv area.
On boot, the loader will process /boot/nextboot.conf if nextboot_enable
is "YES", and will set nextboot_enable to "NO", preventing /boot/nextboot.conf
processing on next boot.
bootonce and nextboot features are usable in both UEFI and BIOS boot.
To use bootonce/nextboot features, the boot loader needs to be updated on disk;
if loader.efi is stored on ESP, then ESP needs to be updated and
for BIOS boot, stage2 (zfsboot or gptzfsboot) needs to be updated
(gpart or other tools).
At this time, only lua loader is updated.
Sponsored by: Netflix, Klara Inc.
Differential Revision: https://reviews.freebsd.org/D25512
sys/ccompile.h no longer uses #pragma ident, so we no longer need to worry
about unknown pragmas.
I fixed one WARNS issue in r363409 by annotating be_is_auto_snapshot_name's
lbh parameter __unused, then upstreamed the following changes to OpenZFS
that rode in with the merge:
- zfs_path_to_zhandle now takes a const char *path rather than a char *path,
since it won't be mutating the string it receives and I had no reason to
believe it will need to in the future. [OpenZFS PR #10605]
- Annotated some unused parameters on definitions inlined into headers as
such. [OpenZFS PR #10606]
The primary benefit is maintaining a completely shared
code base with the community allowing FreeBSD to receive
new features sooner and with less effort.
I would advise against doing 'zpool upgrade'
or creating indispensable pools using new
features until this change has had a month+
to soak.
Work on merging FreeBSD support in to what was
at the time "ZFS on Linux" began in August 2018.
I first publicly proposed transitioning FreeBSD
to (new) OpenZFS on December 18th, 2018. FreeBSD
support in OpenZFS was finally completed in December
2019. A CFT for downstreaming OpenZFS support in
to FreeBSD was first issued on July 8th. All issues
that were reported have been addressed or, for
a couple of less critical matters there are
pull requests in progress with OpenZFS. iXsystems
has tested and dogfooded extensively internally.
The TrueNAS 12 release is based on OpenZFS with
some additional features that have not yet made
it upstream.
Improvements include:
project quotas, encrypted datasets,
allocation classes, vectorized raidz,
vectorized checksums, various command line
improvements, zstd compression.
Thanks to those who have helped along the way:
Ryan Moeller, Allan Jude, Zack Welch, and many
others.
Sponsored by: iXsystems, Inc.
Differential Revision: https://reviews.freebsd.org/D25872
lbh is included for consistency with other functions and in case
future work needs to use it, but it is currently unused. Mark it,
and a post-OpenZFS-import world will be able to raise WARNS of
libbe to the default (pending some minor changes to openzfs libzfs).
MFC after: 3 days
I sent out an e-mail on 2020/01/21 with a plan to do this to Kyle, Rob, and
Wes; all parties have responded in the affirmative that it's OK to drop it
from these files.
More background is available in r356876, but this new interface is more
portable across ZFS implementations and cleaner for what libbe is attempting
to achieve anyways.
MFC after: 3 days
When removing a boot environment iterate over the dependents and process the
snapshots by grabbing any clones. Promote the clones we found and then
remove the target environment.
This fixes the ability to destroy a boot environment when it has been used
to spawn one or more other boot environments.
PR: 242592
Submitted by: Wes Maag <jwmaag gmail com> (with changes by myself)
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D22953
Update a bunch of Makefile.depend files as
a result of adding Makefile.depend.options files
Reviewed by: bdrewery
MFC after: 1 week
Sponsored by: Juniper Networks
Differential Revision: https://reviews.freebsd.org/D22494
This is the half of the changes required that work as-is with both in-tree
ZFS and the new hotness, sysutils/openzfs. Highlights are less dependency
on header pollution (from somewhere) and using 'mnttab' instead of
'extmnttab'. In the in-tree ZFS, the latter is a #define for the former,
but in the port extmnttab is actually a distinct struct that's a super-set
of mnttab. We really want mnttab here anyways, so just use it.
Imported BE, much like the activated BE, will not have an origin that we can
fetch/examine for destruction. be_destroy should not return BE_ERR_NOORIGIN
for failure to get the origin property for BE_DESTROY_AUTOORIGIN, because
we don't really know going into it that there's even an origin to be
destroyed.
BE_DESTROY_NEEDORIGIN has been renamed to BE_DESTROY_WANTORIGIN because only
a subset of it *needs* the origin, so 'need' is too strong of verbiage.
This was caught by jenkins and the bectl tests, but kevans failed to run the
bectl tests prior to commit.
Reported by: lwhsu
New BEs can be created from either an existing snapshot or an existing BE.
If an existing BE is chosen (either implicitly via 'bectl create' or
explicitly via 'bectl create -e foo bar', for instance), then bectl will
create a snapshot of the current BE or "foo" with be_snapshot, with a name
formatted like: strftime("%F-%T") and a serial added to it.
This commit adds the needed bits for libbe or consumers to determine if a
snapshot names matches one of these auto-created snapshots (with some light
validation of the date/time/serial), and also a be_destroy flag to specify
that the origin should be automatically destroyed if possible.
A future commit to bectl will specify BE_DESTROY_AUTOORIGIN by default so we
clean up the origin in the most common case, non-user-managed snapshots.
The default package use to be FreeBSD-runtime but it should only contain
binaries and libs enough to boot to single user and repair the system, it
is also very handy to have a package that can be tranform to a small mfsroot.
So create a new package named FreeBSD-utilities and make it the default one.
Also move a few binaries and lib into this package when it make sense.
Reviewed by: bapt, gjb
Differential Revision: https://reviews.freebsd.org/D21506
Further cleanup after r349380; loader and kernel will both ignore canmount
on the root dataset as well, so we should not be so strict about it when
mounting it. be_mount is restructured to make it more clear that depth==0 is
special, and to not try fetching these properties that we won't care about.
MFC after: 3 days
Other parts of libbe(3) were fairly strict on the mountpoint property of the
BE dataset, and be_mount was not much better. It was improved in r347027 to
allow mountpoint=none for depth==0, but this bit was still sensitive to
mountpoint != / and mountpoint != none. Given that other parts of libbe(3)
no longer restrict the mountpoint property here, and the rest of the base
system is generally OK and will assume that a BE is mounted at /, let's do
the same.
Reported by: ler
MFC after: 3 days
These datasets will generally be canmount=noauto,mountpoint=none (e.g.
zroot/var) but have children that may need to be mounted. Instead of
skipping that segment for no good reason, descend.
Submitted by: Wes Maag
Reported by: Wes Maag
MFC after: 3 days
Instead of pretending to successfully mount them while not actually
mounting anything, we'll now actually mount them *and* claim we mounted them
successfully.
Reported by: ler
MFC after: 3 days
SHLIBDIR should still be optionally set, just before src.opts.mk is included
so that libcompat can properly override it. This fixes lib32 failures
reported by both Jenkins and Michael Butler.
Reported by: Michael Butler <imb@protected-networks.net>
MFC after: 3 days
X-MFC-With: r346546
Rob's patch in D18564 cemented the SHLIBDIR because bsd.own.mk (included by
src.opts.mk) sets it to /usr/lib. r346546 did somehow not apply this part of
the patch, leaving it to get installed to the wrong place and subsequently
removed via ObsoleteFiles.
Reported by: jkim
MFC after: 3 days
X-MFC-With: r346546
libbe currently only provides an API to create a recursive boot environment,
without any formal support for intentionally limiting the depth. This
changeset adds an API, be_create_depth, that may be used to arbitrarily
restrict the depth of the new BE.
Submitted by: Rob Fairbanks <rob.fx907 gmail com>
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D18564
Our home-rolled solution didn't quite capture all of the details, and we
didn't actually validate snapshot names at all. zfs_name_valid captures the
important details, but it doesn't necessarily expose the errors that we're
wanting to see in the be_validate_* functions. Validating lengths
independently, then the names, should make this a non-issue.
To use bectl in an example, when one creates a new boot environment with
either `bectl create <be>` or `bectl create -e <otherbe> <be>`, libbe will
take a snapshot of the original boot environment to clone. Previously, this
used %F-%T date format as the snapshot name, but this has some limitations-
attempting to create multiple boot environments in quick succession may
collide if done within the same second.
Tack a serial onto it to reduce the chances of a collision... we could still
collide if multiple processes/threads are creating boot environments at the
same time, but this is likely not a big concern as this has only been
reported as occurring in freebsd-ci setup.
MFC after: 3 days
'be_destroy' can destroy a boot environment (by name) or a given snapshot.
If the target to be destroyed is a dataset, check if it's mounted. We don't
want to check if the origin dataset is mounted when destroying a snapshot.
PR: 236043
Submitted by: Rob Fairbanks <rob.fx907 gmail com>
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D19650
be_destroy is documented to recursively destroy a boot environment. In the
case of snapshots, one would take this to mean that these are also
recursively destroyed. However, this was previously not the case.
be_destroy would descend into the be_destroy callback and attempt to
zfs_iter_children on the top-level snapshot, which is bogus.
Our alternative approach is to take note of the snapshot name and iterate
through all of fs children of the BE to try destruction in the children.
The -o option is also fixed to work properly with deep BEs. If the BE was
created with `bectl create -e otherDeepBE newDeepBE`, for instance, then a
recursive snapshot of otherDeepBE would have been taken for construction of
newDeepBE but a subsequent destroy with BE_DESTROY_ORIGIN set would only
clean up the snapshot at the root of otherDeepBE: ${BEROOT}/otherDeepBE@...
The most recent iteration instead pretends not to know how these things
work, verifies that the origin is another BE and then passes that back
through be_destroy to DTRT when snapshots and deep BEs may be in play.
MFC after: 1 week
Currently origin snapshots are left behind when a BE is destroyed, whether
it was an auto-created snapshot or explicitly specified via, for example,
`bectl create -e be@mysnap ...`.
Removing it automatically could be argued as a POLA violation in some
circumstances, so provide a flag to be_destroy for it. An accompanying
option will be added to bectl(8) to utilize this.
Some minor style/consistency nits in the affected areas also addressed.
Reported by: Shawn Webb
MFC after: 1 week
Previously, we directly used libzfs_core's lzc_receive to import to a
temporary snapshot, then cloned the snapshot and setup the properties. This
failed when attempting to import replication streams with questionable
error.
libzfs's zfs_receive is a much better fit here, so we now use it instead
with the destination dataset and let libzfs take care of the dirty details.
be_import is greatly simplified as a result.
Reported by: Marie Helene Kvello-Aune <freebsd@mhka.no>
MFC after: 1 week
This set of changes is geared towards making bectl respect deep boot
environments when they exist and are mounted. The deep BE composition
functionality (`bectl add`) remains disabled for the time being. This set of
changes has no effect for the average user. but allows deep BE users to
upgrade properly with their current setup.
libbe(3): Open the target boot environment and get a zfs handle, then pass
that with the target mountpoint to be_mount_iter; If the BE_MNT_DEEP flag is
set call zfs_iter_filesystems and mount the child datasets.
Similar logic is employed when unmounting the datasets, save for children
are unmounted first.
bectl(8): Change bectl_cmd_jail to pass the BE_MNT_DEEP flag when
calling be_mount as well as call be_unmount when cleaning up after the
jail has exited instead of umount(2) directly.
PR: 234795
Submitted by: Wes Maag <jwmaag_gmail.com> (test additions by kevans)
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D18796
We could perhaps have a method that does this given a dataset, but it's yet
clear that we'll always want to bypass the altroot when we grab the
mountpoint. For now, we'll refactor things a bit so we grab the altroot
length when libbe is initialized and have a common method that does the
necessary augmentation (replace with / if it's the root, return a pointer to
later in the string if not).
This will be used in some upcoming work to make be_mount work properly for
deep BEs.
MFC after: 1 week
Previously, the following sequence of events was feasible under some
circumstance:
bectl create test
bectl activate test
# the test BE dataset gets promoted and set as bootfs
bectl destroy test
I was unable to reproduce the destroy succeeding, but we should be rejecting
this before it even gets to libzfs because it would leave the system in an
inconsistent state. Forcing the user to be explicit as to which environment
should be activated instead is much better.
Reported by: Graham Perrin <grahamperrin@gmail.com>
MFC after: 3 days
Those two manual pages are already referencing each other in the HISTORY
sections, which people might skip. Mention those manual pages explicitly in
the SEE ALSO sections. Also, remove a reference to be(1) from libbe(3).
Reviewed by: bcr
Approved by: krion (mentor, implicit), mat (mentor, implicit)
Differential Revision: https://reviews.freebsd.org/D18136
As requested by a TODO in the source code.
Reviewed by: bcr
Approved by: krion (mentor, implicit), mat (mentor, implicit)
Differential Revision: https://reviews.freebsd.org/D18063
If rootfs isn't ZFS, current version will emit an error claiming so and fail
to initialize libbe. As a consumer, bectl -r (undocumented) can be specified
to operate on a BE independently of whether on a UFS or ZFS root.
Unbreak this for the UFS case by only erroring out the init if we can't
determine a ZFS dataset for rootfs and no BE root was specified. Consumers
of libbe should take care to ensure that rootfs is non-empty if they're
trying to use it, because this could certainly be the case.
Some check is needed before zfs_path_to_zhandle because it will
unconditionally emit to stderr if the path isn't a ZFS filesystem, which is
unhelpful for our purposes.
This should also unbreak the bectl(8) tests on a UFS root, as is the case in
Jenkins' -test runs.
MFC after: 3 days
Previously we would blindly copy the 'mountpoint' property, which includes
the altroot. The altroot needs to be snipped off prior to setting it on the
new BE, though, or you'll end up with a new BE and a mountpoint of /mnt with
altroot=/mnt
MFC after: 3 days
Add an undocumented -r option preceding the bectl subcommand to specify a BE
root to operate out of. This will remain undocumented for now, as some
caveats apply:
- BEs cannot be activated in the pool that doesn't contain the rootfs
- bectl create cannot work out of the box without the -e option right now,
since it defaults to the rootfs and cross-pool cloning doesn't work like
that (IIRC)
Plumb the BE root through to libbe(3) so that some things -can- be done to
it, e.g.
bectl -r tank/ROOT create -e default upgrade
bectl -r tank/ROOT mount upgrade /mnt
this aides in some upgrade setups where rootfs is not necessarily ZFS, and
also makes it easier/possible to regression-test bectl when combined with a
file-backed zpool.
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D18029
Go through the ZFS layer instead; given a BE, we can derive the dataset,
zfs_open it, then zfs_unmount. ZFS takes care of the dirty details and
likely gets it more correct than we did for more interesting setups.
MFC after: 3 days
libbe(3) currently uses zfs_be_root and locates which of its children is
currently mounted at "/". This is reasonable, but not correct in the case of
a chroot, for two reasons:
- chroot root may be of a different zpool than zfs_be_root
- chroot root will not show up as mounted at "/"
Fix both of these by rewriting libbe_init to work from the rootfs down.
zfs_path_to_zhandle on / will resolve to the dataset mounted at the new
root, rather than the real root. From there, we can derive the BE root/pool
and grab the bootfs off of the new pool. This does no harm in the average
case, and opens up bectl to operating on different pools for scenarios where
one may be, for instance, updating a pool that generally gets re-rooted into
from a separate UFS root or zfs bootpool.
While here, I've also:
- Eliminated the check for /boot and / to be on the same partition. This
leaves one open to a setup where /boot (and consequently, kernel/modules)
are not included in the boot environment. This may very well be an
intentional setup done by someone that knows what they're doing, we should
not kill BE usage because of it.
- Eliminated the validation bits of BEs and snapshots that enforced
'mountpoint' to be "/" -- this broke when trying to operate on an imported
pool with an altroot, but we need not be this picky.
Reported by: philip
Reviewed by: philip, allanjude (previous version)
Tested by: philip
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D18012
The previously activated BE should have canmount=noauto set on it upon
activation of the new BE, but we previously did not touch canmount on either
old or new BE.
PR: 233113
MFC after: 3 days
Most easily reproducible by attempting to activate the currently activated
BE, one would get a "not a cloned filesystem" error instead of success or a
sane message.
PR: 232488
MFC after: 3 days
This allows older BEs to be destroyed as they become replaced by a BE
created from them: e.g.
bectl create -e brokenworld fixedworld
bectl activate fixedworld
bectl destroy brokenworld
Submitted by: Shawn Webb
Approved by: re (gjb)
Obtained from: HardenedBSD (5948c0581e)
