Commit Graph

262 Commits

Author SHA1 Message Date
kpande
13c59bb76b Add quotations for ${ENCRYPTIONROOT}
Add quotations for ${ENCRYPTIONROOT} to avoid breaking systems
with a space in the name.

Reviewed-by: bunder2015 <omfgbunder@gmail.com>
Reviewed-by: Tom Caputi <tcaputi@datto.com>
Reviewed-by: George Melikov <mail@gmelikov.ru>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Kash Pande <kash@tripleback.net>
Related-to: #8093 
Closes #8114
2018-11-09 09:32:01 -08:00
George Melikov
e871a8f058 Allow use of pool GUID as root pool
It's helpful if there are pools with same names,
but you need to use only one of them.

Main case is twin servers, meanwhile some software
requires the same name of pools (e.g. Proxmox).

Reviewed-by: Kash Pande <kash@tripleback.net>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: George Melikov <mail@gmelikov.ru>
Signed-off-by: Igor ‘guardian’ Lidin of Moscow, Russia
Closes #8052
2018-10-23 20:06:40 -07:00
Garrett Fields
5b3bfd86a4 Check scheduler for "noop" before setting "noop"
Originally code only checked for presence of "/sys/block/$i/queue/
scheduler".  "sh: write error: Invalid argument" was produced when
trying to set "noop" on certain devices (eg. virtio) when it isn't
a listed option. This modification continues to check for the presence
of "/sys/block/$i/queue/scheduler" and also checks that it contains
"noop" as an option before setting "noop".

Reviewed-by: Richard Laager <rlaager@wiktel.com>
Reviewed-by: Tony Hutter <hutter2@llnl.gov>
Reviewed-by: George Melikov <mail@gmelikov.ru>
Reviewed-by: Richard Elling <Richard.Elling@RichardElling.com>
Signed-off-by: Garrett Fields <ghfields@gmail.com>
Closes #8004
2018-10-10 08:46:22 -07:00
LOLi
c962fd6c4e pyzfs: add missing libzfs_core functions
This change adds the following libzfs_core functions to pyzfs:
lzc_remap, lzc_pool_checkpoint, lzc_pool_checkpoint_discard

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: loli10K <ezomori.nozomu@gmail.com>
Closes #7793 
Closes #7800
2018-08-20 10:11:52 -07:00
Tom Caputi
d9c460a0b6 Added encryption support for zfs recv -o / -x
One small integration that was absent from b52563 was
support for zfs recv -o / -x with regards to encryption
parameters. The main use cases of this are as follows:

* Receiving an unencrypted stream as encrypted without
  needing to create a "dummy" encrypted parent so that
  encryption can be inheritted.

* Allowing users to change their keylocation on receive,
  so long as the receiving dataset is an encryption root.

* Allowing users to explicitly exclude or override the
  encryption property from an unencrypted properties stream,
  allowing it to be received as encrypted.

* Receiving a recursive heirarchy of unencrypted datasets,
  encrypting the top-level one and forcing all children to
  inherit the encryption.

Reviewed-by: Jorgen Lundman <lundman@lundman.net>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Richard Elling <Richard.Elling@RichardElling.com>
Signed-off-by: Tom Caputi <tcaputi@datto.com>
Closes #7650
2018-08-15 09:48:49 -07:00
Antonio Russo
9b9d1adc38 Use zfs-import.target in contrib/dracut
The new zfs-import.target should be used in place of the
zfs-import-*.service units.

Reviewed by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Manuel Amador (Rudd-O) <rudd-o@rudd-o.com>
Signed-off-by: Antonio Russo <antonio.e.russo@gmail.com>
Closes #6964
2018-07-31 10:15:41 -07:00
George Diamantopoulos
fb7307b892 Fix initramfs missing systemd binaries
Systemd binaries necessary for mounting an encrypted root dataset
weren't copied to initramfs generated by dracut. This patch fixes
this and copies these binaries unconditionally, that is
regardless of whether native ZFS encryption is used for the
root dataset.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: George Diamantopoulos <georgediam@gmail.com>
Closes #7607
Closes #7719
2018-07-27 09:29:43 -07:00
Tom Caputi
c634808ebb Add pyzfs build directories to gitignore
The recent addition of pyzfs does not include the generated 'build'
and 'pyzfs.egg-info' directories in the pyzfs .gitignore or the
'make clean' target. This patch simply corrects this problem.

Reviewed-by: George Melikov <mail@gmelikov.ru>
Reviewed-by: loli10K <ezomori.nozomu@gmail.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Tom Caputi <tcaputi@datto.com>
Closes #7612
2018-06-11 18:42:12 -07:00
Steffen Müthing
3c28c63642 Install basename utility into dracut initramfs
vdev_id requires the program `basename` when handling short aliases
defined in `vdev_id.conf` (those defined without a leading path), but
`basename` is not always available in the dracut environment. This
causes the pool device names to change when using `by-vdev/` devices
or (in extreme cases) can make the pool import fail in dracut.

This commit fixes the problem by explicitly installing `basename`.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Steffen Müthing <steffen.muething@iwr.uni-heidelberg.de>
Closes #7562
2018-05-29 17:32:05 -07:00
Tomohiro Kusumi
e1245d83e9 Prevent make distclean removing 0 sized file
__init__.py used by Python packages typically has nothing in it
including contrib/pyzfs/libzfs_core/test/__init__.py, however this
causes `make distclean` to delete the file.

This is the only file with size 0, and it seems reasonable to have
a comment to avoid being deleted, rather than trying to modify
distclean behavior.

 # find . -size 0
 ./contrib/pyzfs/libzfs_core/test/__init__.py
 # ./autogen.sh ; ./configure ; make -j8
 # make distclean
 # ls contrib/pyzfs/libzfs_core/test/__init__.py
 ls: cannot access 'contrib/pyzfs/libzfs_core/test/__init__.py':
     No such file or directory
 # git diff
 diff --git a/contrib/pyzfs/libzfs_core/test/__init__.py
     b/contrib/pyzfs/libzfs_core/test/__init__.py
 deleted file mode 100644

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Tomohiro Kusumi <kusumi.tomohiro@osnexus.com>
Closes #7505
2018-05-06 20:46:13 -07:00
loli10K
85ce3f4fd1 Adopt pyzfs from ClusterHQ
This commit introduces several changes:

 * Update LICENSE and project information

 * Give a good PEP8 talk to existing Python source code

 * Add RPM/DEB packaging for pyzfs

 * Fix some outstanding issues with the existing pyzfs code caused by
   changes in the ABI since the last time the code was updated

 * Integrate pyzfs Python unittest with the ZFS Test Suite

 * Add missing libzfs_core functions: lzc_change_key,
   lzc_channel_program, lzc_channel_program_nosync, lzc_load_key,
   lzc_receive_one, lzc_receive_resumable, lzc_receive_with_cmdprops,
   lzc_receive_with_header, lzc_reopen, lzc_send_resume, lzc_sync,
   lzc_unload_key, lzc_remap

Note: this commit slightly changes zfs_ioc_unload_key() ABI. This allow
to differentiate the case where we tried to unload a key on a
non-existing dataset (ENOENT) from the situation where a dataset has
no key loaded: this is consistent with the "change" case where trying
to zfs_ioc_change_key() from a dataset with no key results in EACCES.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: loli10K <ezomori.nozomu@gmail.com>
Closes #7230
2018-05-01 10:33:35 -07:00
Andriy Gapon
6abf922574 Import pyzfs source code from ClusterHQ
libzfs_core is intended to be a stable interface for programmatic
administration of ZFS.

This wrapper provides one-to-one wrappers for libzfs_core API functions,
but the signatures and types are more natural to Python.
nvlists are wrapped as dictionaries or lists depending on their usage.
Some parameters have default values depending on typical use for
increased convenience.
Enumerations and bit flags become strings and lists of strings in
Python.
Errors are reported as exceptions rather than integer errno-style
error codes.  The wrapper takes care to provide one-to-many mapping
of the error codes to the exceptions by interpreting a context
in which the error code is produced.

Unit tests and automated test for the libzfs_core API are provided
with this package.

Please note that the API tests perform lots of ZFS dataset level
operations and ZFS tries hard to ensure that any modifications
do reach stable storage. That means that the operations are done
synchronously and that, for example, disk caches are flushed.
Thus, the tests can be very slow on real hardware.
It is recommended to place the default temporary directory or
a temporary directory specified by, for instance, TMP environment
variable on a memory backed filesystem.

Original-patch-by: Andriy Gapon <avg@FreeBSD.org>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Ported-by: loli10K <ezomori.nozomu@gmail.com>
Signed-off-by: loli10K <ezomori.nozomu@gmail.com>
Closes #7230
2018-05-01 10:31:11 -07:00
Tony Hutter
6c9af9e8f4 Fix "file is executable, but no shebang" warnings
Fedora 28's RPM build checks warn when executable files don't have a
shebang line.  These warnings are caused when we (incorrectly)
include data & config files in the_SCRIPTS automake lines. Files in
_SCRIPTS are marked executable by automake. This patch fixes the
issue by including non-executable scripts in a _DATA line instead.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Giuseppe Di Natale <dinatale2@llnl.gov>
Signed-off-by: Tony Hutter <hutter2@llnl.gov>
Closes #7359 
Closes #7395
2018-04-06 16:34:21 -07:00
kpande
05747eca5b modprobe zfs during dracut mount
Resolves importing root pool during boot in dracut.  This case was
inadvertently broken with the module autoloading change in #7287.

Reviewed-by: Matthew Thode <prometheanfire@gentoo.org>
Reviewed-by: George Melikov <mail@gmelikov.ru>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Giuseppe Di Natale <dinatale2@llnl.gov>
Signed-off-by: Kash Pande <kash@tripleback.net>
Closes #7322
2018-03-22 10:14:29 -07:00
John Eismeier
d699aaef09 Fix some typos
Reviewed-by: Giuseppe Di Natale <dinatale2@llnl.gov>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed by: George Melikov <mail@gmelikov.ru>
Signed-off-by: John Eismeier <john.eismeier@gmail.com>
Closes #7237
2018-02-28 08:57:10 -08:00
Kash Pande
41532e5a29 Shellcheck cleanup for initrd scripts
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Co-authored-by: Kash Pande <kash@tripleback.net>
Co-authored-by: Matthew Thode <mthode@mthode.org>
Signed-off-by: Kash Pande <kash@tripleback.net>
Signed-off-by: Matthew Thode <mthode@mthode.org>
Closes #7214
2018-02-23 12:57:41 -08:00
Kash Pande
7280d58197 Enable booting from nested encrypted datasets
- enable booting from nested encrypted datasets
- fix plymouth boot splash passphrase entry
- optimize unlock process

Co-authored-by: Kash Pande <kash@tripleback.net>
Co-authored-by: Matthew Thode <mthode@mthode.org>
Signed-off-by: Kash Pande <kash@tripleback.net>
Signed-off-by: Matthew Thode <mthode@mthode.org>
Closes #7214
2018-02-23 12:57:28 -08:00
LOLi
a893627fac contrib/initramfs: add missing conf.d/zfs
When upgrading from the distribution-provided zfs-initramfs package on
root-on-zfs Ubuntu and Debian the system may fail to boot: this change
adds the missing initramfs configuration file.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Richard Laager <rlaager@wiktel.com>
Signed-off-by: loli10K <ezomori.nozomu@gmail.com>
Closes #7158
2018-02-12 11:40:00 -08:00
Chris Williamson
d99a015343 OpenZFS 7431 - ZFS Channel Programs
Authored by: Chris Williamson <chris.williamson@delphix.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: John Kennedy <john.kennedy@delphix.com>
Reviewed by: Dan Kimmel <dan.kimmel@delphix.com>
Approved by: Garrett D'Amore <garrett@damore.org>
Ported-by: Don Brady <don.brady@delphix.com>
Ported-by: John Kennedy <john.kennedy@delphix.com>

OpenZFS-issue: https://www.illumos.org/issues/7431
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/dfc11533

Porting Notes:
* The CLI long option arguments for '-t' and '-m' don't parse on linux
* Switched from kmem_alloc to vmem_alloc in zcp_lua_alloc
* Lua implementation is built as its own module (zlua.ko)
* Lua headers consumed directly by zfs code moved to 'include/sys/lua/'
* There is no native setjmp/longjump available in stock Linux kernel.
  Brought over implementations from illumos and FreeBSD
* The get_temporary_prop() was adapted due to VFS platform differences
* Use of inline functions in lua parser to reduce stack usage per C call
* Skip some ZFS Test Suite ZCP tests on sparc64 to avoid stack overflow
2018-02-08 15:28:18 -08:00
Matthew Thode
6f259b59cf Only run pre-mount hook zfs-load-key on systemd
Reviewed-by: Kash Pande <kash@tripleback.net>
Reviewed-by: bunder2015 <omfgbunder@gmail.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Matthew Thode <mthode@mthode.org>
Closes #7136 
Closes #7140
2018-02-07 18:31:54 -08:00
Brian Behlendorf
6d82b79699
Add zfs-load-key.sh to .gitignore
The generated zfs-load-key.sh file should have been added to
the .gitignore file as part of commit 7da8f8d8.  And the
generated file should not be included in the repo.

Reviewed-by: Matthew Thode <prometheanfire@gentoo.org>
Reviewed-by: Giuseppe Di Natale <dinatale2@llnl.gov>
Reviewed by: George Melikov <mail@gmelikov.ru>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #7134
2018-02-06 16:39:18 -08:00
Tom Caputi
ae76f45cda Encryption Stability and On-Disk Format Fixes
The on-disk format for encrypted datasets protects not only
the encrypted and authenticated blocks themselves, but also
the order and interpretation of these blocks. In order to
make this work while maintaining the ability to do raw
sends, the indirect bps maintain a secure checksum of all
the MACs in the block below it along with a few other
fields that determine how the data is interpreted.

Unfortunately, the current on-disk format erroneously
includes some fields which are not portable and thus cannot
support raw sends. It is not possible to easily work around
this issue due to a separate and much smaller bug which
causes indirect blocks for encrypted dnodes to not be
compressed, which conflicts with the previous bug. In
addition, the current code generates incompatible on-disk
formats on big endian and little endian systems due to an
issue with how block pointers are authenticated. Finally,
raw send streams do not currently include dn_maxblkid when
sending both the metadnode and normal dnodes which are
needed in order to ensure that we are correctly maintaining
the portable objset MAC.

This patch zero's out the offending fields when computing
the bp MAC and ensures that these MACs are always
calculated in little endian order (regardless of the host
system's byte order). This patch also registers an errata
for the old on-disk format, which we detect by adding a
"version" field to newly created DSL Crypto Keys. We allow
datasets without a version (version 0) to only be mounted
for read so that they can easily be migrated. We also now
include dn_maxblkid in raw send streams to ensure the MAC
can be maintained correctly.

This patch also contains minor bug fixes and cleanups.

Reviewed-by: Jorgen Lundman <lundman@lundman.net>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Signed-off-by: Tom Caputi <tcaputi@datto.com>
Closes #6845
Closes #6864
Closes #7052
2018-02-02 11:37:16 -08:00
Brian Behlendorf
f55a8757a6
Update README.initramfs.markdown
Fix several typos and grammar.

Reviewed-by: Giuseppe Di Natale <dinatale2@llnl.gov>
Reviewed by: George Melikov <mail@gmelikov.ru>
Signed-off-by: Arno van Wyk <avw1987@users.noreply.github.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #7080
2018-01-26 09:55:16 -08:00
Matthew Thode
7da8f8d81b Run zfs load-key if needed in dracut
'zfs load-key -a' will only be called if needed.  If a dataset not
needed for boot does not have its key loaded (home directories for
example) boot can still continue.

zfs:AUTO was not working via dracut, so we still need the generator
script to do its thing.

Reviewed-by: Richard Yao <ryao@gentoo.org>
Reviewed-by: Manuel Amador (Rudd-O) <rudd-o@rudd-o.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: loli10K <ezomori.nozomu@gmail.com>
Signed-off-by: Matthew Thode <mthode@mthode.org>
Closes #6982 
Closes #7004
2018-01-18 10:20:34 -08:00
LOLi
e2d936e0f8 Honor --with-mounthelperdir where applicable
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Giuseppe Di Natale <dinatale2@llnl.gov>
Signed-off-by: loli10K <ezomori.nozomu@gmail.com>
Closes #6962
2017-12-17 14:14:07 -08:00
JKDingwall
9717fe052b Add /usr/bin/env to COPY_EXEC_LIST initramfs hook
5dc1ff29 changed the user space program to mount a zfs snapshot
from /bin/sh to /usr/bin/env.  If the executable is not present
in the initramfs then snapshots cannot be automounted.

Reviewed-by: George Melikov <mail@gmelikov.ru>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Richard Laager <rlaager@wiktel.com>
Reviewed-by: Giuseppe Di Natale <dinatale2@llnl.gov>
Signed-off-by: James Dingwall <james.dingwall@zynstra.com>
Closes #5360 
Closes #6913
2017-12-04 11:53:57 -08:00
Richard Laager
48ac22d855 initramfs: Honor canmount=off
The initramfs script was not honoring canmount=off.  With this change,
it does.  If the administrator has asked that a filesystem not be
mounted, that should be honored.

As an exception, the initramfs script ignores canmount=off on the
rootfs.  The rootfs should not have canmount=off set either.  However,
mounting it anyway seems harmless because it is being asked for
explicitly.  The point of this exception is to avoid the risk of
breaking existing systems, just in case someone has canmount=off set on
their rootfs.

The initramfs still mounts filesystems with canmount=noauto.  This is
necessary because it is typical to set that on the rootfs so that it can
be cloned.  Without canmount=noauto, the clones' duplicate mountpoints
would conflict.

This is the remainder of the fix for:
https://github.com/zfsonlinux/pkg-zfs/issues/221

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: George Melikov <mail@gmelikov.ru>
Signed-off-by: Richard Laager <rlaager@wiktel.com>
Closes #6897
2017-11-28 09:38:13 -08:00
Richard Laager
bd2958dea0 initramfs: Honor mountpoint=none/legacy
For filesystems that are children of the rootfs, when mountpoint=none or
mountpoint=legacy, the initrafms script would assume a mountpoint based
on the dataset path.  Given that the rootfs should have mountpoint=/ and
mountpoint inheritance is is the default behavior of ZFS, this behavior
seems unnecessary.  In any event, it turns mountpoint=none into a no-op.
That removes this option from the administrator, and if someone uses it,
it does not work as expected.  Worse yet, if the mountpoint directory
does not exist (which is the typical case for mountpoint=none), the
mounting and thus the boot process will fail.  For the case of
mountpoint=legacy, the assumed mountpoint may not be the correct value
set in /etc/fstab.

This change makes the initramfs script not mount the filesystem in
either case.  For mountpoint=none, this means we are correctly honoring
the setting.  For mountpoint=legacy, there are two scenarios:  If
canmount=on, the filesystem will be mounted by the normal mechanisms
later in the boot process.  If canmount=noauto, the filesystem will not
be mounted at all, unless the administrator has done something special.
If they're not doing something special and they want it mounted by the
initramfs, they can simply not set mountpoint=legacy.

This is part of the fix for:
https://github.com/zfsonlinux/pkg-zfs/issues/221

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: George Melikov <mail@gmelikov.ru>
Signed-off-by: Richard Laager <rlaager@wiktel.com>
Closes #6897
2017-11-28 09:38:00 -08:00
LOLi
cb3b0419ba contrib/initramfs: switch to automake
Use automake to build initramfs scripts and hooks.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: loli10K <ezomori.nozomu@gmail.com>
Closes #6761
2017-11-07 14:53:57 -08:00
Richard Laager
4fc411f7a3 initramfs: Set elevator=noop on the rpool's disks
ZFS already sets elevator=noop for wholedisk vdevs (for all pools), but
typical root-on-ZFS installations use partitions.  This sets
elevator=noop on the disks in the root pool.

Ubuntu 16.04 and 16.10 had this.  It was lost in 17.04 due to Debian
switching to this upstream initramfs script.

Signed-off-by: Richard Laager <rlaager@wiktel.com>
2017-11-01 21:54:56 -05:00
Richard Laager
11b9dcfb2d initramfs: Fix a spelling error
This fixes a typo in a comment.

Signed-off-by: Richard Laager <rlaager@wiktel.com>
2017-11-01 21:54:28 -05:00
Richard Laager
4767c7a14e initramfs: Fix inconsistent whitespace
This fixes one instance of inconsistent whitespace.

Signed-off-by: Richard Laager <rlaager@wiktel.com>
2017-11-01 21:53:22 -05:00
Brian Behlendorf
21a932b83c Post-Encryption Followup
This PR includes fixes for bugs and documentation issues found 
after the encryption patch was merged and general code improvements 
for long-term maintainability.

Reviewed-by: Jorgen Lundman <lundman@lundman.net>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Tom Caputi <tcaputi@datto.com>
Issue #6526
Closes #6639
Closes #6703
Cloese #6706
Closes #6714
Closes #6595
2017-10-13 10:02:39 -07:00
Antonio Russo
085b501fb8 Explicitly depend on icp module in initramfs hook
Automatic dependency resolution is unreliable on many systems.
Follow suit with existing code, and explicitly include icp
in module dependencies.

Reviewed-by: George Melikov <mail@gmelikov.ru>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Antonio Russo <antonio.e.russo@gmail.com>
Closes #6751
2017-10-12 09:39:45 -07:00
Tom Caputi
4807c0badb Encryption patch follow-up
* PBKDF2 implementation changed to OpenSSL implementation.

* HKDF implementation moved to its own file and tests
  added to ensure correctness.

* Removed libzfs's now unnecessary dependency on libzpool
  and libicp.

* Ztest can now create and test encrypted datasets. This is
  currently disabled until issue #6526 is resolved, but
  otherwise functions as advertised.

* Several small bug fixes discovered after enabling ztest
  to run on encrypted datasets.

* Fixed coverity defects added by the encryption patch.

* Updated man pages for encrypted send / receive behavior.

* Fixed a bug where encrypted datasets could receive
  DRR_WRITE_EMBEDDED records.

* Minor code cleanups / consolidation.

Signed-off-by: Tom Caputi <tcaputi@datto.com>
2017-10-11 16:54:48 -04:00
aun
d4404c3fdb Fix boot from ZFS issues
* Correct ZFS snapshot listing
* Disable "lvm is not available" message on quiet boot

Reviewed-by: George Melikov <mail@gmelikov.ru>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Alar Aun <spamtoaun@gmail.com>
Closes #6700 
Closes #6747
2017-10-11 10:06:20 -07:00
privb0x23
4f23c5d0c4 Fix inclusion of libgcc_s.so on Void
On Void Linux (x86_64 musl) libgcc_s.so is located in "/usr/lib"
so it is not found by dracut and it produces an error.

Add a simple additional path check for "/usr/lib/libgcc_s.so*"
and install it in the initramfs.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: privb0x23 <privb0x23@users.noreply.github.com>
Closes #6715
2017-10-09 14:34:26 -07:00
Fabian-Gruenbichler
42a76fc8d7 dracut: make module-setup.sh shebang explicit
while these are source by dracut (which is a bash script)
the practical difference is small, but it is more correct:

/bin/sh is not bash on all systems (e.g. Debian and its
derivatives use /bin/dash as /bin/sh by default).

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Closes #6491
2017-08-14 10:56:04 -07:00
Karsten Kretschmer
d19a6d5c80 dracut: Install commands required for vdev_id
The vdev_id script requires awk, grep, and head.  Use dracut_install to
ensure that these commands are available in the initrd environment.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Karsten Kretschmer <kkretschmer@gmail.com>
Closes #6443
Closes #6452
2017-08-04 11:14:48 -07:00
Matt Kemp
912e2ba92f Fix initramfs hook for merged /usr/lib and /lib
Under a merged `/lib` -> `/usr/lib` which renders `/lib` as a symlink,
`find /lib -type f -name libgcc_s.so.1` will not return a result as
`find` will not traverse the symlink. Modifying it to `find /lib/ -type
f -name libgcc_s.so.1` should work for both symlinked and non-symlinked
`/lib` directories.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Matt Kemp <matt@mattikus.com>
Closes #5834
2017-02-27 12:03:23 -08:00
Aron Xu
3d91261599 contrib/dracut: fix syntax error in module-setup.sh
The dracut/02zfsexpandknowledge/module-setup.sh.in has a syntax error
which makes the script unusable by a POSIX compliant shell like Dash
on Debian based systems.

Reviewed-by: Giuseppe Di Natale <dinatale2@llnl.gov>
Reviewed-by: George Melikov <mail@gmelikov.ru>
Signed-off-by: Aron Xu <happyaron.xu@gmail.com>
Closes #5712
2017-01-31 14:25:49 -08:00
ka7
4e33ba4c38 Fix spelling
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov
Reviewed-by: Giuseppe Di Natale <dinatale2@llnl.gov>>
Reviewed-by: George Melikov <mail@gmelikov.ru>
Reviewed-by: Haakan T Johansson <f96hajo@chalmers.se>
Closes #5547 
Closes #5543
2017-01-03 11:31:18 -06:00
Brian Behlendorf
9d70aec6fd Update .gitignore
Two additional files were recently introduced and should be
ignored by git.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #5299
2016-10-19 14:29:33 -07:00
Rudd-O
f8e87e205c Properly use the Dracut cleanup hook to order pool shutdown
When Dracut starts up, it needs to determine whether a pool will remain
"hanging open" before the system shuts off. In such a case, then the
code to clean up the pool (using the previous export -F work) must
be invoked. Since Dracut has had a recent change that makes
mount-zfs.sh simply not run when the root dataset is already mounted,
we must use the cleanup hook to order Dracut to do shutdown cleanup.

Important note: this code will not accomplish its stated goal until this
bug is fixed: https://bugzilla.redhat.com/show_bug.cgi?id=1385432

That bug impacts more than just ZFS. It impacts LUKS, dmraid, and
unmount during poweroff. It is a Fedora-wide bug.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Manuel Amador (Rudd-O) <rudd-o@rudd-o.com>
Closes #5287
2016-10-17 11:51:15 -07:00
Rudd-O
7e8a2d0b75 Use -F to export pools so as not to dirty up device labels
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Manuel Amador (Rudd-O) <rudd-o@rudd-o.com>
Closes #5228 
Closes #5238
2016-10-15 20:30:53 -07:00
Rudd-O
b0f578a807 Use a different technique to detect whether to mount-zfs
The behavior of the Dracut module was very wrong before.

The correct behavior: initramfs should not run `zfs-mount` to completion
if the two generator files exist.  If, however, one of them is missing,
it indicates one of three cases:

* The kernel command line did not specify a root ZFS file system, and
  another Dracut module is already handling root mount (via systemd).
  `mount-zfs` can run, but it will do nothing.
* There is no systemd to run `sysroot.mount` to begin with.
  `mount-zfs` must run.
* The root parameter is zfs:AUTO, which cannot be run in sysroot.mount.
  `mount-zfs` must run.

In any of these three cases, it is safe to run `zfs-mount` to completion.

`zfs-mount` must also delete itself if it determines it should not run,
or else Dracut will do the insane thing of running it over and over again.
Literally, the definition of insanity, doing the same thing that did not
work before, expecting different results.  Doing that may have had a great
result before, when we had a race between devices appearing and pools
being mounted, and `mount-zfs` was tasked with the full responsibility
of importing the needed pool, but nowadays it is wrong behavior and
should be suppressed.

I deduced that self-deletion was the correct thing to do by looking at
other Dracut code, because (as we all are very fully aware of) Dracut
is entirely, ahem, "implementation-defined".

Tested-by: @wphilips 
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Manuel Amador (Rudd-O) <rudd-o@rudd-o.com>
Closes #5157 
Closes #5204
2016-10-06 10:26:47 -07:00
Moritz Maxeiner
70cc394d91 Fix regression that broke dracut initramfs generation
Based upon @ryao's initial fix for 1c73494394fc9de9283b3fd4f00bcdf4bd300a7
( 5e9843405f63fdabe76e87b92b81a127d488abc7 ) this one also uses
`command -v` instead of `type`, but additionally only applies the
fix to close zfsonlinux/zfs#4749 when `libgcc_s.so.1` has not been included
by dracut automatically (verified by whether `zpool` links directly to
`libgcc_s.so`), as well as change the fallback option to match `libgcc_s.so*`.

Tested-by: Ben Jencks <ben@bjencks.net>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Moritz Maxeiner <moritz@ucworks.org>
Closes #5089 
Closed #5138
2016-09-21 13:35:16 -07:00
slashdd
792517389f Change /etc/mtab to /proc/self/mounts
Fix misleading error message:

 "The /dev/zfs device is missing and must be created.", if /etc/mtab is missing.

Reviewed-by: Richard Laager <rlaager@wiktel.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Eric Desrochers <eric.desrochers@canonical.com>
Closes #4680 
Closes #5029
2016-09-20 10:07:58 -07:00
Moritz Maxeiner
8516203d53 zfs dracut module should not assume systemd presence
Signed-off-by: Moritz Maxeiner <moritz@ucworks.org>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #4749
Closes #5058
2016-09-09 10:17:27 -07:00
Moritz Maxeiner
61c7349439 Adapt genkernel fix for zfsonlinux/zfs#4749 to zfs dracut module
Signed-off-by: Moritz Maxeiner <moritz@ucworks.org>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #4749
Closes #5058
2016-09-09 10:16:52 -07:00
Chunwei Chen
5b1bc1a1d8 Set proper dependency for string replacement targets
A lot of string replacement target don't have dependency or incorrect
dependency. We setup proper dependency by pattern rules.

Signed-off-by: Chunwei Chen <david.chen@osnexus.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #4908
2016-08-02 10:28:29 -07:00
jyxent
dabe1c42f9 Fix the test to use the variable
Signed-off-by: Manuel Amador (Rudd-O) <rudd-o@rudd-o.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #4645
2016-05-13 20:44:03 -07:00
Manuel Amador (Rudd-O)
d402c18dd6 A collection of dracut fixes
- In older systems without sysroot.mount, import before dracut-mount,
  and re-enable old dracut mount hook
- rootflags MUST be present even if the administrator neglected to
  specify it explicitly
- Check that mount.zfs exists in sbindir
- Remove awk and head as (now unused) requirements, add grep, and
  install the right mount.zfs
- Eliminate one use of grep in Dracut
- Use a more accurate grepping statement to identify zfsutil in rootflags
- Ensure that pooldev is nonempty
- Properly handle /dev/sd* devices and more
- Use new -P to get list of zpool devices
- Bail out of the generator when zfs:AUTO is on the root command line
- Ignore errors from systemctl trying to load sysroot.mount, we only
  care about the output
- Determine which one is the correct initqueuedir at run time.
- Add a compatibility getargbool for our detection / setup script.
- Update dracut .gitignore files

Signed-off-by: <Matthew Thode mthode@mthode.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #4558
Closes #4562
2016-05-12 14:31:15 -07:00
Matthew Thode
a5a370227e fix booting via dracut generated initramfs
Dracut and Systemd updated how they integrate with each other, because
of this our current integrations stopped working (around the time
4.1.13 came out).  This patch addresses that issue and gets us booting
again.

Thanks to @Rudd-O for doing the work to get dracut working again and
letting me submit this on his behalf.

Signed-off-by: Manuel Amador (Rudd-O) <rudd-o@rudd-o.com>
Signed-off-by: Matthew Thode <mthode@mthode.org>
Closes #3605
Closes #4478
2016-04-25 08:51:38 -07:00
Benjamin Albrecht
82dba185c8 Activate LVM volume groups before looking for zpools.
Original-patch-by: @jgoerzen
Signed-off-by: Benjamin Albrecht <git@albrecht.io>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes zfsonlinux/pkg-zfs#102
Closes #4029
2015-12-18 13:46:23 -08:00
Turbo Fredriksson
57732964d3 Init script fixes
* Fix regression - "OVERLAY_MOUNTS" should have been "DO_OVERLAY_MOUNTS".
* Fix update-rc.d commands in postinst.  Thanx to subzero79@GitHub.
* Fix make sure a filesystem exists before trying to mount in mount_fs()
* Fix local variable usage.
* Fix to read_mtab():
  * Strip control characters (space - \040) from /proc/mounts GLOBALY,
    not just first occurrence.
  * Don't replace unprintable characters ([/-. ]) for use in the variable
    name with underscore. No need, just remove them all together.
* Add check_boolean() to check if a user configure option is
  set ('yes', 'Yes', 'YES' or any combination there of) OR '1'.
  Anything else is considered 'unset'.
* Add a ZFS_POOL_IMPORT to the default config.
  * This is a semi colon separated list of pools to import ONLY.
  * This is intended for systems which have _a lot_ of pools (from
    a SAN for example) and it would be to many to put in the
    ZFS_POOL_EXCEPTIONS variable..
* Add a config option "ZPOOL_IMPORT_OPTS" for adding additional options
  to "zpool import".
* Add documentation and the chance of overriding the ZPOOL_CACHE
  variable in the config file.
* Remove "sort" from find_pools() and setup_snapshot_booting().
  Sometimes not available, and not really necessary.

Signed-off-by: Turbo Fredriksson <turbo@bayour.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ned Bass <bass6@llnl.gov>
Issue #3816
2015-09-29 11:42:24 -07:00
Turbo Fredriksson
21d41d6806 Make sure that POOL_IMPORTED is set, unset and checked where appropriate.
* If it's unset in find_rootfs(), no pool is imported so no point in
  looking for a rootfs.
* If find_rootfs() couldn't find a rootfs, the pool is exported. Remember
  to unset POOL_IMPORTED after doing so.
* Set POOL_IMPORTED if/when a pool have been imported in import_pool().
* Improve backup import (the one using cache file).

Signed-off-by: Turbo Fredriksson <turbo@bayour.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3636
2015-07-28 13:29:28 -07:00
Turbo Fredriksson
48511ea645 Fix some minor issues with the SYSV init and initramfs scripts.
This is some minor fixes to commits 2cac7f5f11
and 2a34db1bdb.

* Make sure to alien'ate the new initramfs rpm package as well!
  The rpm package is build correctly, but alien isn't run on it to
  create the deb.
* Before copying file from COPY_FILE_LIST, make sure the DESTDIR/dir exists.
* Include /lib/udev/vdev_id file in the initrd.
* Because the initrd needs to use '/sbin/modprobe' instead of 'modprobe',
  we need to use this in load_module() as well.
  * Make sure that load_module() can be used more globaly, instead of
    calling '/sbin/modprobe' all over the place.
  * Make sure that check_module_loaded() have a parameter - module to
    check.

Signed-off-by: Turbo Fredriksson <turbo@bayour.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3626
2015-07-24 15:05:33 -07:00
Turbo Fredriksson
47a4a6fd5f Support parallel build trees (VPATH builds)
Build products from an out of tree build should be written
relative to the build directory.  Sources should be referred
to by their locations in the source directory.

This is accomplished by adding the 'src' and 'obj' variables
for the module Makefile.am, using relative paths to reference
source files, and by setting VPATH when source files are not
co-located with the Makefile.  This enables the following:

  $ mkdir build
  $ cd build
  $ ../configure \
    --with-spl=$HOME/src/git/spl/ \
    --with-spl-obj=$HOME/src/git/spl/build
  $ make -s

This change also has the advantage of resolving the following
warning which is generated by modern versions of automake.

  Makefile.am:00: warning: source file 'xxx' is in a subdirectory,
  Makefile.am:00: but option 'subdir-objects' is disabled

Signed-off-by: Turbo Fredriksson <turbo@bayour.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1082
2015-07-17 13:42:51 -07:00
Brian Behlendorf
cc49250563 Move dracut directory to contrib
The dracut code is analogous to the initramfs code and as such
it should be located in the contrib with initramfs for consistency.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
2015-07-09 13:59:37 -07:00
Turbo Fredriksson
2cac7f5f11 Initramfs scripts for ZoL.
* Supports booting of a ZFS snapshot.
  Do this by cloning the snapshot into a dataset. If this, the resulting
  dataset, already exists, destroy it. Then mount it on root.
  * If snapshot does not exist, use base dataset (the part before '@')
    as boot filesystem instead.
  * If no snapshot is specified on the 'root=' kernel command line, but there
    is an '@', then get a list of snapshots below that filesystem and ask the
    user which to use.
  * Clone with 'mountpoint=none' and 'canmount=noauto' - we mount manually
    and explicitly.
    * For sub-filesystems, that doesn't have a mountpoint property set, we use
      the 'org.zol:mountpoint' to keep track of it's mountpoint.
  * Allow rollback of snapshots instead of clone it and boot from the clone.
* Allow mounting a root- and subfs with mountpoint=legacy set
* Allow mounting a filesystem which is using nativ encryption.
* Support all currently used kernel command line arguments
  All the different distributions have their own standard on what to specify
  on the kernel command line to boot of a ZFS filesystem.
  * Extra options:
    * zfsdebug=(on,yes,1)	Show extra debugging information
    * zfsforce=(on,yes,1)	Force import the pool
    * rollback=(on,yes,1)	Rollback (instead of clone) the snapshot
* Only try to import pool if it haven't already been imported
  * This will negate the need to force import a pool that have not been exported cleanly.
  * Support exclusion of pools to import by setting ZFS_POOL_EXCEPTIONS in /etc/default/zfs.
* Support additional configuration variable ZFS_INITRD_ADDITIONAL_DATASETS
  to mount additional filesystems not located under your root dataset.
* Include /etc/modprobe.d/{zfs,spl}.conf in the initrd if it/they exist.
* Include the udev rule to use by-vdev for pool imports.
* Include the /etc/default/zfs file to the initrd.
* Only try /dev/disk/by-* in the initrd if USE_DISK_BY_ID is set.
  * Use /dev/disk/by-vdev before anything.
  * Add /dev as a last ditch attempt.
  * Fallback to using the cache file if that exist if nothing else worked.
* Use /sbin/modprobe instead of built-in (BusyBox) modprobe.
  This gets rid of the message "modprobe: can't load module zcommon".
  Thanx to pcoultha for finding this.

Signed-off-by: Turbo Fredriksson <turbo@bayour.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #2116
Closes #2114
2015-07-08 18:14:34 -07:00
Turbo Fredriksson
1ffe90c5d3 Add bash completions by Aneurin Price.
These can be manually installed as needed by end users.  They
have been added to the repository so they can be kept up to date
with the latest code.

Signed-off-by: Turbo Fredriksson <turbo@bayour.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1588
2014-08-06 15:03:28 -07:00