Independent of all of the commands, bectl itself takes an `-r` flag that
specifies the BE root to use. This was originally added to facilitate
testing, but it was later discovered to be incredibly useful in other
scenarios; e.g., trying to recover some boot environments in rescue
media.
The "BE root" described here is the parent dataset that holds boot
environments, but I've no idea if that's an accepted definition for that
dataset.
Reviewed by: gallatin, imp, Pau Amma
MFC after: 1 week
Differential Review: https://reviews.freebsd.org/D39710
Notable upstream pull request merges:
#12194 Fix short-lived txg caused by autotrim
#13368 ZFS_IOC_COUNT_FILLED does unnecessary txg_wait_synced()
#13392 Implementation of block cloning for ZFS
#13741 SHA2 reworking and API for iterating over multiple implementations
#14282 Sync thread should avoid holding the spa config write lock
when possible
#14283 txg_sync should handle write errors in ZIL
#14359 More adaptive ARC eviction
#14469 Fix NULL pointer dereference in zio_ready()
#14479 zfs redact fails when dnodesize=auto
#14496 improve error message of zfs redact
#14500 Skip memory allocation when compressing holes
#14501 FreeBSD: don't verify recycled vnode for zfs control directory
#14502 partially revert PR 14304 (eee9362a7)
#14509 Fix per-jail zfs.mount_snapshot setting
#14514 Fix data race between zil_commit() and zil_suspend()
#14516 System-wide speculative prefetch limit
#14517 Use rw_tryupgrade() in dmu_bonus_hold_by_dnode()
#14519 Do not hold spa_config in ZIL while blocked on IO
#14523 Move dmu_buf_rele() after dsl_dataset_sync_done()
#14524 Ignore too large stack in case of dsl_deadlist_merge
#14526 Use .section .rodata instead of .rodata on FreeBSD
#14528 ICP: AES-GCM: Refactor gcm_clear_ctx()
#14529 ICP: AES-GCM: Unify gcm_init_ctx() and gmac_init_ctx()
#14532 Handle unexpected errors in zil_lwb_commit() without ASSERT()
#14544 icp: Prevent compilers from optimizing away memset()
in gcm_clear_ctx()
#14546 Revert zfeature_active() to static
#14556 Remove bad kmem_free() oversight from previous zfsdev_state_list
patch
#14563 Optimize the is_l2cacheable functions
#14565 FreeBSD: zfs_znode_alloc: lock the vnode earlier
#14566 FreeBSD: fix false assert in cache_vop_rmdir when replaying ZIL
#14567 spl: Add cmn_err_once() to log a message only on the first call
#14568 Fix incremental receive silently failing for recursive sends
#14569 Restore ASMABI and other Unify work
#14576 Fix detection of IBM Power8 machines (ISA 2.07)
#14577 Better handling for future crypto parameters
#14600 zcommon: Refactor FPU state handling in fletcher4
#14603 Fix prefetching of indirect blocks while destroying
#14633 Fixes in persistent error log
#14639 FreeBSD: Remove extra arc_reduce_target_size() call
#14641 Additional limits on hole reporting
#14649 Drop lying to the compiler in the fletcher4 code
#14652 panic loop when removing slog device
#14653 Update vdev state for spare vdev
#14655 Fix cloning into already dirty dbufs
#14678 Revert "Do not hold spa_config in ZIL while blocked on IO"
Obtained from: OpenZFS
OpenZFS commit: 431083f75b
When recovering a system that is unbootable due to some
problem with the active BE, it is likely you'll be booted
from a rescue image running UFS. In this case, bectl
needs help finding the zpool root that you want to operate
on. In this case, improve the error message to suggest
specifying a root, rather than just emitting a generic
error message that might imply, to the naive user, that
there is a ZFS compatibility issue between the rescue
image and the on-disk ZFS pool.
Reviewed by: imp, kevans
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D39346
This check was previously in `create` only, not applying to renames. It
should really be applied at the libbe level, so that we can avoid
writing about this restriction over and over again.
While we're here: `bectl rename` always succeeds, even when it doesn't.
Start returning the error.
Reported By: Christian McDonald <cmcdonald netgate com>
Reviewed by: rew, jwmaag_gmail.com (earlier version)
Differential Revision: https://reviews.freebsd.org/D34605
Boot environment datasets that contain spaces are not bootable.
When a user attempts to create a boot environment with a space, abort
the creation and print an error message.
PR: 254441
Reviewed by: allanjude
Differential Revision: https://reviews.freebsd.org/D30194
This rode in with the OpenZFS import. It may have been necessary at some
point, but it is no longer and it breaks the WITHOUT_DYNAMICROOT build as
it collides with the definition in libspl.
Reported-by: Michael Dexter
Flag -o for destroy subcommand is missing in the help message,
but present in the man page. Fix it.
PR: 249325
Approved by: kevans
Differential Revision: https://reviews.freebsd.org/D26429
bootonce feature is temporary, one time boot, activated by
"bectl activate -t BE", "bectl activate -T BE" will reset the bootonce flag.
By default, the bootonce setting is reset on attempt to boot and the next
boot will use previously active BE.
By setting zfs_bootonce_activate="YES" in rc.conf, the bootonce BE will
be set permanently active.
bootonce dataset name is recorded in boot pool labels, bootenv area.
in case of nextboot, the nextboot_enable boolean variable is recorded in
freebsd:nvstore nvlist, also stored in boot pool label bootenv area.
On boot, the loader will process /boot/nextboot.conf if nextboot_enable
is "YES", and will set nextboot_enable to "NO", preventing /boot/nextboot.conf
processing on next boot.
bootonce and nextboot features are usable in both UEFI and BIOS boot.
To use bootonce/nextboot features, the boot loader needs to be updated on disk;
if loader.efi is stored on ESP, then ESP needs to be updated and
for BIOS boot, stage2 (zfsboot or gptzfsboot) needs to be updated
(gpart or other tools).
At this time, only lua loader is updated.
Sponsored by: Netflix, Klara Inc.
Differential Revision: https://reviews.freebsd.org/D25512
The primary benefit is maintaining a completely shared
code base with the community allowing FreeBSD to receive
new features sooner and with less effort.
I would advise against doing 'zpool upgrade'
or creating indispensable pools using new
features until this change has had a month+
to soak.
Work on merging FreeBSD support in to what was
at the time "ZFS on Linux" began in August 2018.
I first publicly proposed transitioning FreeBSD
to (new) OpenZFS on December 18th, 2018. FreeBSD
support in OpenZFS was finally completed in December
2019. A CFT for downstreaming OpenZFS support in
to FreeBSD was first issued on July 8th. All issues
that were reported have been addressed or, for
a couple of less critical matters there are
pull requests in progress with OpenZFS. iXsystems
has tested and dogfooded extensively internally.
The TrueNAS 12 release is based on OpenZFS with
some additional features that have not yet made
it upstream.
Improvements include:
project quotas, encrypted datasets,
allocation classes, vectorized raidz,
vectorized checksums, various command line
improvements, zstd compression.
Thanks to those who have helped along the way:
Ryan Moeller, Allan Jude, Zack Welch, and many
others.
Sponsored by: iXsystems, Inc.
Differential Revision: https://reviews.freebsd.org/D25872
I sent out an e-mail on 2020/01/21 with a plan to do this to Kyle, Rob, and
Wes; all parties have responded in the affirmative that it's OK to drop it
from these files.
-o will force the origin to be destroyed unconditionally.
BE_DESTROY_AUTOORIGIN, on the other hand, will only destroy the origin if it
matches the format used by be_snapshot. This lets us clean up the snapshots
that are clearly not user-managed (because we're creating them) while
leaving user-created snapshots in place and warning that they're still
around when the BE created goes away.
This command simply returns 0 at the moment and explicitly takes no
arguments. This should be used by utilities wanting to see if bectl can
operate on the system they're running, or with a specific root (`bectl -r`).
It may grow more checks than "will libbe successfully init" in the future,
but for now this is enough as that checks for the dataset mounted at "/" and
that it looks capable of being a BE root (e.g. it's not a top-level dataset)
bectl commands can now specify if they want to be silent, and this will turn
off libbe_print_on_error so they can control the output as needed. This is
already used in `bectl check`, and may be turned on in the future for some
other commands where libbe errors are better suppressed as the failure mode
may be obvious.
Requested by: David Fullard
MFC after: 3 days
Allow 'bectl list' to sort output by a given property name. The property
name is passed in using a command-line flag, '-c' for ascending order and
'-C' for descending order. The properties allowed to sort by are:
- name (the default output, even if '-c' or '-C' are not used)
- creation
- origin
- used
- usedds
- usedsnap
- usedrefreserv
The default output for 'bectl list' is now ascending alphabetical order of
BE name.
To sort by creation time from earliest to latest, the command would be
'bectl list -c creation'
Submitted by: Rob Fairbanks <rob.fx907 gmail com>
Reviewed by: ler
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D20818
bectl advertises that it has the ability to create recursive and
non-recursive boot environments. This patch implements that functionality
using the be_create_depth API provided by libbe. With this patch, bectl now
works as bectl(8) describes in regards to creating recursive/non-recursive
boot environments.
Submitted by: Rob Fairbanks <rob.fx907 gmail com> (with minor changes)
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D20240
We can't predict when destruction of origin is needed, and currently we have
a precedent for not prompting for things. Leave the decision up to the user
of bectl(8) if they want the origin snapshot to be destroyed or not.
Emits a warning when -o isn't used and an origin snapshot is left to be
cleaned up, for the time being. This is handy when one drops the -o flag but
really did want to clean up the origin.
A couple of -e ignore's have been sprinkled around the test suite for places
that we don't care that the origin's not been cleaned up. -o functionality
tests will be added in the future, but are omitted for now to reduce
conflicts with work in flight to fix bits of the tests.
Reported by: Shawn Webb
MFC after: 1 week
This set of changes is geared towards making bectl respect deep boot
environments when they exist and are mounted. The deep BE composition
functionality (`bectl add`) remains disabled for the time being. This set of
changes has no effect for the average user. but allows deep BE users to
upgrade properly with their current setup.
libbe(3): Open the target boot environment and get a zfs handle, then pass
that with the target mountpoint to be_mount_iter; If the BE_MNT_DEEP flag is
set call zfs_iter_filesystems and mount the child datasets.
Similar logic is employed when unmounting the datasets, save for children
are unmounted first.
bectl(8): Change bectl_cmd_jail to pass the BE_MNT_DEEP flag when
calling be_mount as well as call be_unmount when cleaning up after the
jail has exited instead of umount(2) directly.
PR: 234795
Submitted by: Wes Maag <jwmaag_gmail.com> (test additions by kevans)
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D18796
Add an undocumented -r option preceding the bectl subcommand to specify a BE
root to operate out of. This will remain undocumented for now, as some
caveats apply:
- BEs cannot be activated in the pool that doesn't contain the rootfs
- bectl create cannot work out of the box without the -e option right now,
since it defaults to the rootfs and cross-pool cloning doesn't work like
that (IIRC)
Plumb the BE root through to libbe(3) so that some things -can- be done to
it, e.g.
bectl -r tank/ROOT create -e default upgrade
bectl -r tank/ROOT mount upgrade /mnt
this aides in some upgrade setups where rootfs is not necessarily ZFS, and
also makes it easier/possible to regression-test bectl when combined with a
file-backed zpool.
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D18029
vermaden (maintainer of beadm) points out the following inconsistencies:
- "missing command" is not printed prior to usage if the error is simply a
missing command; this should be obvious from the context
- "bectl rename" isn't using the "don't unmount" flag (zfs rename -u), so
the active BE can't be renamed. It doesn't make sense in our context to
*not* use -u, so use it.
Documentation updates reflect the above and note an inconsistency with the
'destroy' command that is consistent with other parts of the base system.
A fix for libbe(3) not properly being installed to /lib is included.
SHLIBDIR should have been added when it was moved in r337995.
Approved by: re (kib)
- Some overly-long lines
- Consistently using .Brq ({})
- Consistently using .Cm
- Not using .Ao/.Ac around .Ar
PR: 230576
Submitted by: Yuri Pankov (with a fair amount of rebasing pre-commit)
Add a -U flag to get back the old behavior. The new behavior is a little
more friendly to the common use cases, jail the BE and execute a script.
Having the jail torn down automatically when the script is finished, or when
you exit the shell, is a little more friendly than having to remember to
`bectl ujail`.
Batch mode (-b) will continue to leave the jail up, as it's assumed the
caller has other intentions.
Submitted by: Shawn Webb (partially)
Instead of always running /bin/sh, allow the user to specify the command
to run. The jail is not removed when the command finishes. Meaning,
`bectl unjail` will still need to be run.
For example:
```
bectl jail newBE pkg upgrade
bectl ujail newBE
```
Submitted by: Shawn Webb
Obtained from: HardenedBSD (8b451014ab)
Adding batch mode to the jail `bectl(8)` subcommand enables jailing of
ZFS Boot Environments in a scriptable fashion.
Submitted by: Shawn Webb
Obtained from: HardenedBSD (9e72d1c59a and ef7b6d9e1c with minor edit)
The mostly-undocumented 'add' functionality, from initial read-through, is
intended for construction of deep ("bdrewery style") boot environments.
However, it's mostly broken at this point. `#if SOON` it out on both sides
so that we're not exposing a broken API/feature.
Work will resume on it in due time.
- File names don't necessarily need to be repeated
- Add SPDX tags
- Add a missing copyright for Kyle Kneitinger in bectl.8, originally written
by him in GSoC 2017; his standard copyright notice has been copied from
other files within the same directory to remain consistent with how he
clearly wished to portray it
- Support passing arbitrary jail arguments via -o
- Split the related (and rewritten since the GSoC) jail bits out into a new
bectl_jail.c file, to reduce clutter in bectl.c
- Don't use RFC 1918 IP space [0]; we'll instead set no default IPv4 and let
the user pass in any address options they wish via -o
Reported by: rgrimes [0], Shawn Webb [0]
be_get_dataset_snapshots has been added to libbe(3), effectively returning
the same information as be_get_bootenv_props but for snapshots of the given
dataset. The assumption is that one will have the BE dataset name before
wanting to grab snapshots.
This also accomplishes the following:
- Proxy through zfs_nicenum as be_nicenum, because it looks better than
humanize_number and would presumably be useful to other libbe consumers.
- Rename be_get_snapshot_props to be_get_dataset_props, make it more useful
-H is for a scripted format, where all fields are tab-delimited and the
headers go away. We accomplish this by splitting out pad printing to a
separate function that'll take into account whether we're supposed to be
scripted or not.
This has the nice side effect of maintaining positive column sizes again.
While it could be preferred to do this at insert in libbe(3), there's no
convenient way to insert at the head of an nvlist. Instead, we'll make two
passes over- once to print anything active either now or at nextboot, and
another to print everything else.
This doesn't actually impact performance in a significant way here, so we'll
worry about further optimizations if the need actually arises.
bectl list -a should show the boot environment, its relevant dataset, and
the snapshot it was created from. The header also changes to indicate the
rough order in which these things will show.
While here, start doing a little bit of spring cleaning and splitting
different bits out.
The jail is created with allow.mount, allow.mount.devfs, and
enforce_statfs=1. Upon creation, we immediately attach, chdir to "/", and
drop the user into a shell inside the jail.
The default IP for this is arbitrarily 10.20.30.40.
The given parameter may either be a jid, jail name, or a BE name. In all
cases, the parameter will be resolved to a jid and bectl(8) will
sanity-check that there's actually a BE mounted at the requested jail root
before invoking jail_remove(2).
