Commit Graph

82 Commits

Author SHA1 Message Date
Toomas Soome
2c55d0903d Add SHA512, skein, large blocks support for loader zfs.
Updated sha512 from illumos.
Using skein from freebsd crypto tree.
Since loader itself is using 64MB memory for heap, updated zfsboot to
use same, and this also allows to support zfs large blocks.

Note, adding additional features does increate zfsboot code, therefore
this update does increase zfsboot code to 128k, also I have ported gptldr.S
update to zfsldr.S to support 64k+ code.

With this update, boot1.efi has almost reached the current limit of the size
set for it, so one of the future patches for boot1.efi will need to
increase the limit.

Currently known missing zfs features in boot loader are edonr and gzip support.

Reviewed by:	delphij, imp
Approved by:	imp (mentor)
Obtained from:	sha256.c update and skein_zfs.c stub from illumos.
Differential Revision:	https://reviews.freebsd.org/D7418
2016-08-18 00:37:07 +00:00
Allan Jude
4deb8929ea Make boot code and loader check for unsupported ZFS feature flags
OpenZFS uses feature flags instead of a zpool version number to track
features since the split from Oracle. In addition to avoiding confusion
on ZFS vs OpenZFS version numbers, this also allows features to be added
to different operating systems that use OpenZFS in different order.

The previous zfs boot code (gptzfsboot) and loader (zfsloader) blindly
tries to read the pool, and if failed provided only a vague error message.

With this change, both the boot code and loader check the MOS features
list in the ZFS label and compare it against the list of features that
the loader supports. If any unsupported feature is active, the pool is
not considered as a candidate for booting, and a helpful diagnostic
message is printed to the screen. Features that are merely enabled via
zpool upgrade, but not in use, do not block booting from the pool.

Submitted by:	Toomas Soome <tsoome@me.com>
Reviewed by:	delphij, mav
Relnotes:	yes
Differential Revision:	https://reviews.freebsd.org/D6857
2016-08-01 19:37:43 +00:00
Andriy Gapon
dc87c22a61 fix a zfs boot regression introduced in r300117 by accident
There is no reason to return non-zero value from zfs_probe_partition()
as that causes following partitions to not be probed for ZFS vdevs.
A particular scenario that I encountered is a GPT partitioned disk
where several partitions have freebsd-zfs type.  A partition with a lower
index is used as a cache (l2arc) vdev and in that case case zfs_probe()
returned a non-zero status.  That status was returned to ptable_iterate()
and caused it to abort the iteration.  Because of that the subsequent
partitions were not probed and a root pool was not discovered resulting
in a boot failure.

While there fix the style for nearby return statements.

Approved by:	re (kib)
2016-06-16 07:45:57 +00:00
Warner Losh
9a0b26ec6f Fix several instances where the boot loader ignored pager_output
return value when it could return 1 (indicating we should stop).
Fix a few instances of pager_open() / pager_close() not being called.
Actually use these routines for the environment variable printing code
I just committed.
2016-05-18 05:59:05 +00:00
Pedro F. Giffuni
d9c9c81c08 sys: use our roundup2/rounddown2() macros when param.h is available.
rounddown2 tends to produce longer lines than the original code
and when the code has a high indentation level it was not really
advantageous to do the replacement.

This tries to strike a balance between readability using the macros
and flexibility of having the expressions, so not everything is
converted.
2016-04-21 19:57:40 +00:00
Allan Jude
87ed2b7f5a A new implementation of the loader block cache
The block cache implementation in loader has proven to be almost useless, and in worst case even slowing down the disk reads due to insufficient cache size and extra memory copy.
Also the current cache implementation does not cache reads from CDs, or work with zfs built on top of multiple disks.
Instead of an LRU, this code uses a simple hash (O(1) read from cache), and instead of a single global cache, a separate cache per block device.
The cache also implements limited read-ahead to increase performance.
To simplify read ahead management, the read ahead will not wrap over bcache end, so in worst case, single block physical read will be performed to fill the last block in bcache.

Booting from a virtual CD over IPMI:
0ms latency, before: 27 second, after: 7 seconds
60ms latency, before: over 12 minutes, after: under 5 minutes.

Submitted by:	Toomas Soome <tsoome@me.com>
Reviewed by:	delphij (previous version), emaste (previous version)
Relnotes:	yes
Differential Revision:	https://reviews.freebsd.org/D4713
2016-04-18 23:09:22 +00:00
Allan Jude
b996592c3e Implement GELI (AES-XTS and AES-CBC only) in gptboot and gptzfsboot
Allows booting from a GELI encrypted root file system, via UFS or ZFS

Reviewed by:	gnn, smh (previous version), delphij (previous version)
Relnotes:	yes
Sponsored by:	ScaleEngine Inc.
Differential Revision:	https://reviews.freebsd.org/D4593
2016-03-16 23:12:19 +00:00
Bryan Drewery
a09f4b4d12 Add more .NOMETA missed in r291320
Sponsored by:	EMC / Isilon Storage Division
2016-03-11 23:45:51 +00:00
Allan Jude
9cdff681a4 Do not set vfs.root.mountfrom unnecessarily
This causes boot from external media (installer USB image) to mount / from
the default ZFS BE, rather than the USB device.

Reported by:	kmoore
MFC after:	5 days
Sponsored by:	ScaleEngine Inc.
2016-02-07 00:49:15 +00:00
Allan Jude
fbe958861e Move init_zfs_bootenv to sys/boot/zfs/zfs.c instead of having a copy in each loader
While here, add a filter to ignore special datasets

MFC after:	3 days
Sponsored by:	ScaleEngine Inc.
2016-01-15 05:45:10 +00:00
Steven Hartland
276a15676c Prevent bogus compiler in ZFS boot code
Silence a bogus compiler warning about indexing past the end of dn_bonus.

The ZFS code ensures this is not possible but the compiler can't determine
this so added an additional check to prevent this warning.

Sponsored by:	Multiplay
2016-01-14 21:31:26 +00:00
Allan Jude
076b613091 DIOCGSECTORSIZE expects to write to a u_int, but struct zfs_probe_args
member secsz was a uint16_t

sys/boot/zfs/zfs.c has a probe args structure member, secsz, that is a
uint16_t for media sector size; it is used as an argument for ioctl()
at line 484. however, this ioctl writes 32 bits of data (u_int *) and
therefore this ioctl will overwrite and corrupt 16 bits of memory.
other use cases seem to use correct u_int type for secsz.

PR:		204358
Submitted by:	Toomas Soome <tsoome at me.com>
Reviewed by:	asomers, delphij, smh
MFC after:	5 days
Differential Revision:	https://reviews.freebsd.org/D4811
2016-01-11 15:35:29 +00:00
Allan Jude
fce8d0e350 Only call init_zfs_bootenv() when the system was booted with ZFS
Add a few other safeguards to ensure things do not break when the
boot device cannot be determined

Reported by:	flo
MFC after:	3 days
Sponsored by:	ScaleEngine Inc.
2016-01-09 00:54:08 +00:00
Steven Hartland
5d07d143e6 Fix return from zfs_probe_dev
Ensure zfs_probe_dev returns the correct value.

Also fix a style(9) trailing whitespace issue while here.

MFC after:	2 weeks
X-MFC-With:	r293268
Sponsored by:	Multiplay
2016-01-06 20:25:41 +00:00
Allan Jude
f1aba489c1 Introduce the ZFS Boot Environments menu to the loader menu
If the system was booted with ZFS, a new menu item (#7) appears
It contains an autogenerated list of ZFS Boot Environments

This allows the user to switch to an alternate root file system
Use Cases:
 - Revert a failed upgrade
 - Concurrently run different versions of FreeBSD with common home directory
 - Easier integration with the sysadmin/beadm utility

Requested by:	many
Reviewed by:	dteske
MFC after:	10 days
Relnotes:	yes
Sponsored by:	ScaleEngine Inc.
Differential Revision:	https://reviews.freebsd.org/D3167
2015-12-31 20:00:53 +00:00
Bryan Drewery
b1f92fa229 META MODE: Update dependencies with 'the-lot' and add missing directories.
This is not properly respecting WITHOUT or ARCH dependencies in target/.
Doing so requires a massive effort to rework targets/ to do so.  A
better approach will be to either include the SUBDIR Makefiles directly
and map to DIRDEPS or just dynamically lookup the SUBDIR.  These lose
the benefit of having a userland/lib, userland/libexec, etc, though and
results in a massive package.  The current implementation of targets/ is
very unmaintainable.

Currently rescue/rescue and sys/modules are still not connected.

Sponsored by:	EMC / Isilon Storage Division
2015-12-01 05:23:19 +00:00
Warner Losh
9d2edd63d4 Use CFLAGS_NO_SIMD in preference to varying lists of -mno-xxxx flags.
Go ahead and defined -D_STANDALONE for all targets (only strictly
needed for some architecture, but harmless on those it isn't required
for). Also add -msoft-float to all architectures uniformly rather
that higgley piggley like it is today.

Differential Revision: https://reviews.freebsd.org/D3496
2015-08-27 23:46:42 +00:00
Andriy Gapon
649a80dc4c dnode_read: fixup r284025, BP_IS_HOLE macro expects a pointer
PR:		199804
Reported by:	sbruno
Pointyhat to:	avg
MFC after:	10 days
X-MFC with:	r284025
2015-06-05 17:02:21 +00:00
Andriy Gapon
dade6e27ab dnode_read: handle hole blocks in zfs boot code
A hole block pointer can be encountered at any level and the hole
can cover multiple data blocks, but we are reading the data blocks
one by one, so we care only about the current one.

PR:		199804
Reported by:	Toomas Soome <tsoome@me.com>
Submitted by:	Toomas Soome <tsoome@me.com> (earlier version)
Tested by:	Toomas Soome <tsoome@me.com>
MFC after:	11 days
2015-06-05 15:32:04 +00:00
Xin LI
8bcd603968 MFV r274273:
ZFS large block support.

Please note that booting from datasets that have recordsize greater
than 128KB is not supported (but it's Okay to enable the feature on
the pool).  This *may* remain unchanged because of memory constraint.

Limited safety belt is provided for mounted root filesystem but use
caution is advised.

Illumos issue:
    5027 zfs large block support

MFC after:	1 month
2014-11-10 08:20:21 +00:00
Xin LI
29441ba3fa MFV r267565:
4757 ZFS embedded-data block pointers ("zero block compression")
4913 zfs release should not be subject to space checks

MFC after:	2 weeks
2014-07-01 06:43:15 +00:00
Xin LI
f4c8ba8370 MFV r259170:
4370 avoid transmitting holes during zfs send

4371 DMU code clean up

illumos/illumos-gate@43466aae47

NOTE: Make sure the boot code is updated if a zpool upgrade is
done on boot zpool.

MFC after:	2 weeks
2014-01-01 00:45:28 +00:00
Dimitry Andric
0673132dcb For libstand and sys/boot, split off gcc-only flags into CFLAGS.gcc.
MFC after:	3 days
X-MFC-With:	r259730
2013-12-26 11:32:39 +00:00
Andriy Gapon
881a94fab8 boot: use -march=i386 for both i386 and amd64 builds
.. so that consistent compilation algorithms are used for both
architectures as in practice the binaries are expected to be
interchangeable (for time being).
Previously i386 used default setting which were equivalent to
-march=i486 -mtune=generic.
The only difference is using smaller but slower "leave" instructions.

Discussed with:	jhb, dim
MFC after:	29 days
2012-10-20 16:57:23 +00:00
Andriy Gapon
aae0c9de03 zfs boot: export boot/primary pool and vdev guid all the way to kenv
This is work in progress to for znextboot and it also provides
some convenient infrastructure.

MFC after:	20 days
2012-10-06 19:47:24 +00:00
Andriy Gapon
d39075208e zfs loader: treat plain pool name as a name of its root dataset
... as opposed to the previous behavior of treating it as boot
dataset (specified by bootfs or default)

MFC after:	19 days
2012-10-06 19:42:50 +00:00
Andriy Gapon
edfd4fce8f zfs boot spa_status: print bootfs for each reported pool
MFC after:	9 days
2012-10-06 19:42:05 +00:00
Andriy Gapon
164efe4010 boot/zfs: a small whitespace cleanup
MFC after:	5 days
2012-10-06 19:41:11 +00:00
Andriy Gapon
62c725a9db boot/zfs: call zfs_spa_init for all found pools
... and drop those for which it fails.
Also, add more sanity checking to the function.

MFC after:	16 days
2012-10-06 19:40:12 +00:00
Andriy Gapon
74b3e265c7 zfs boot: add code for listing child datasets of a given dataset
- only filesystem datasets are supported
- children names are printed to stdout

To do: allow to iterate over the list and fetch names programatically

MFC after:	17 days
2012-10-06 19:27:04 +00:00
Andriy Gapon
84b339ac4c zfs boot: chose a "first" pool if none is explicitly requested
MFC after:	8 days
2012-10-06 19:25:40 +00:00
Andriy Gapon
7ae0dc79b7 zfs boot: add a size check for a value in fzap_lookup
MFC after:	25 days
2012-09-11 07:15:11 +00:00
Andriy Gapon
4b7fc6b08e zfs boot: print only an attribute name in fzap_list
... this matches mzap_list behavior

MFC after:	12 days
2012-09-11 07:13:58 +00:00
Andriy Gapon
e644b976b0 zfs boot: fix/replace fzap_rlookup implementation
The previous one was totally bogus as it used hash value of
_output_ variable as an index for searching...
The only reliable way to do a reverse lookup here is to iterate
over all entries.

MFC after:	15 days
2012-09-11 07:12:48 +00:00
Andriy Gapon
b6910e777e zfs boot: bring zap_leaf_chunk field names in sync with kernel code
This change is cosmetic.

MFC after:	10 days
2012-09-11 07:11:32 +00:00
Andrey V. Elsukov
a188e43ef3 Explicitly terminate the string after strncpy(3). 2012-08-15 09:18:49 +00:00
Andrey V. Elsukov
4c89da6c18 Teach the ZFS use new partitions API when probing.
Note: now ZFS does probe only for partitions with type "freebsd-zfs"
and "freebsd".
2012-08-05 14:48:28 +00:00
Andrey A. Chernov
283e000db4 Try to avoid all files dependence on the modification time of the large and
often modified directory created symbolic links points to - it cause
unnecessary full rebuilds each time make runs when directory is changed.
So do it only if symbolic link does not exists, which usually means that
objdir is clean anyway.

MFC after:      1 week
2012-07-26 10:10:54 +00:00
Martin Matuska
253a9c296c Fix ZFS boot with pre-features pools (version <= 28) broken in r236884
Reported by:	mav
MFC after:	1 month
2012-06-13 08:00:32 +00:00
Martin Matuska
2d9cf57e18 Introduce "feature flags" for ZFS pools (bump SPA version to 5000).
Add first feature "com.delphix:async_destroy" (asynchronous destroy
of ZFS datasets).
Implement features support in ZFS boot code.

Illumos revisions merged:
13700:2889e2596bd6
13701:1949b688d5fb
2619 asynchronous destruction of ZFS file systems
2747 SPA versioning with zfs feature flags

References:
https://www.illumos.org/issues/2619
https://www.illumos.org/issues/2747

Obtained from:	illumos (issue #2619, #2747)
MFC after:	1 month
2012-06-11 11:35:22 +00:00
Andriy Gapon
0a56cdf866 zfs boot: cleanup remnants of temporary compat code
MFC after:	1 month
2012-05-13 10:54:43 +00:00
Andriy Gapon
dfc351d8e9 zfs boot code: mark spa_t arguments as const where they are used as such
MFC after:	1 month
2012-05-13 09:22:18 +00:00
Andriy Gapon
32df866434 sparc64/zfs boot: take advantage of new libzfsboot capabilities
Also drop the now unneeded compatibility shims.

Tested by:	marius
MFC after:	1 month
2012-05-12 20:27:33 +00:00
Andriy Gapon
f7f7b759f6 zfs boot code: use %j and uintmax_t instead %ll and uint64_t in printfs
This is to silence warnings that result from different definitions of
uint64_t on different architectures, specifically i386 and sparc64.

MFC after:	1 month
2012-05-12 20:23:30 +00:00
Andriy Gapon
1702e62f67 zfsboot/zfsloader: support accessing filesystems within a pool
In zfs loader zfs device name format now is "zfs:pool/fs",
fully qualified file path is "zfs:pool/fs:/path/to/file"
loader allows accessing files from various pools and filesystems as well
as changing currdev to a different pool/filesystem.

zfsboot accepts kernel/loader name in a format pool:fs:path/to/file or,
as before, pool:path/to/file; in the latter case a default filesystem
is used (pool root or bootfs).  zfsboot passes guids of the selected
pool and dataset to zfsloader to be used as its defaults.

zfs support should be architecture independent and is provided
in a separate library, but architectures wishing to use this zfs support
still have to provide some glue code and their devdesc should be
compatible with zfs_devdesc.
arch_zfs_probe method is used to discover all disk devices that may
be part of ZFS pool(s).

libi386 unconditionally includes zfs support, but some zfs-specific
functions are stubbed out as weak symbols.  The strong definitions
are provided in libzfsboot.
This change mean that the size of i386_devspec becomes larger
to match zfs_devspec.

Backward-compatibility shims are provided for recently added sparc64
zfs boot support.  Currently that architecture still works the old
way and does not support the new features.

TODO:
- clear up pool root filesystem vs pool bootfs filesystem distinction
- update sparc64 support
- set vfs.root.mountfrom based on currdev (for zfs)

Mid-future TODO:
- loader sub-menu for selecting alternative boot environment

Distant future TODO:
- support accessing snapshots, using a snapshot as readonly root

Reviewed by:	marius (sparc64),
		Gavin Mu <gavin.mu@gmail.com> (sparc64)
Tested by:	Florian Wagner <florian@wagner-flo.net> (x86),
		marius (sparc64)
No objections:	fs@, hackers@
MFC after:	1 month
2012-05-12 09:03:30 +00:00
Marius Strobl
2d75b8321f Add initial support for booting from ZFS on sparc64. At least on Sun Fire
V100, the firmware is known to be broken and not allowing to simultaneously
open disk devices, causing attempts to boot from a mirror or RAIDZ to cause
a crash. This will be worked around later. The firmwares of newer sun4u models
don't seem to exhibit this problem though.

Steps for ZFS booting:

1. create VTOC8 label
# gpart create -s vtoc8 da0

2. add partitions, f.e.:
# gpart add -t freebsd-zfs -s 60g da0
# gpart add -t freebsd-swap da0
resulting in something like:
# gpart show
=>        0  143331930  da0  VTOC8  (68G)
          0  125821080    1  freebsd-zfs  (60G)
  125821080   17510850    2  freebsd-swap  (8.4G)

3. create zpool
# zpool create bunker da0a
or for mirror/RAIDZ (after preparing additional disks as in steps 1. + 2.):
# zpool create bunker mirror da0a da1a
# zpool create bunker raidz da0a da1a da2a ...

4. set bootfs
# zpool set bootfs=bunker bunker

5. install zfsboot
# zpool export bunker
# gpart bootcode -p /boot/zfsboot da0

6. write zfsloader to the ZFS Boot Block (so far, there's no dedicated tool
for this, so dd(1) has to be used for this purpose)
When using mirror/RAIDZ, step 4. and the dd(1) invocation should be repeated
for the additional disks in order to be able to boot from another disk in
case of failure.
# sysctl kern.geom.debugflags=0x10
# dd if=/boot/zfsloader of=/dev/da0a bs=512 oseek=1024 conv=notrunc
# zpool import bunker

7. install system on ZFS filesystem
Don't forget to set 'zfs_load="YES"' and vfs.root.mountfrom="zfs:bunker" in
loader.conf as well as 'zfs_enable="YES"'in rc.conf.

8. copy zpool.cache to the ZFS filesystem
cp -p /boot/zfs/zpool.cache /bunker/boot/zfs/zpool.cache

9. set mountpoint
# zfs set mountpoint=/ bunker

10. Now, given that aliases for all disks in the zpool exists (check with
the `devalias` command on the boot monitor prompt) and disk0 corresponds
to da0 (likewise for additional disks), the system can be booted from the
ZFS with:
{1} ok boot disk0

PR:             165025
Submitted by:   Gavin Mu
2012-05-01 17:16:01 +00:00
Andriy Gapon
05b096ff61 zfs boot: allow file vdevs to be used in testing (e.g. with zfsboottest)
MFC after:	1 week
2011-12-04 21:29:56 +00:00
Pawel Jakub Dawidek
9838b8b0ee - Correctly read gang header from raidz.
- Decompress assembled gang block data if compressed.
- Verify checksum of a gang header.
- Verify checksum of assembled gang block data.
- Verify checksum of uber block.

Submitted by:	avg
MFC after:	3 days
2011-10-20 15:42:38 +00:00
Pawel Jakub Dawidek
81fdf04870 Always pass data size for checksum verification function, as using
physical block size declared in bp may not always be what we want.
For example in case of gang block header physical block size declared
in bp is much larger than SPA_GANGBLOCKSIZE (512 bytes) and checksum
calculation failed. This bug could lead to accessing unallocated
memory and resets/failures during boot.

MFC after:	3 days
2011-10-19 23:44:38 +00:00
Pawel Jakub Dawidek
8565a6f600 Never pass NULL block pointer when reading. This is neither expected nor
handled by lower layers like vdev_raidz, which uses bp for checksum
verification. This bug could lead to NULL pointer reference and resets
during boot.

MFC after:	3 days
2011-10-19 23:40:37 +00:00