4185 add new cryptographic checksums to ZFS: SHA-512, Skein, Edon-R
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Prakash Surya <prakash.surya@delphix.com>
Reviewed by: Saso Kiselkov <saso.kiselkov@nexenta.com>
Reviewed by: Richard Lowe <richlowe@richlowe.net>
Approved by: Garrett D'Amore <garrett@damore.org>
Author: Matthew Ahrens <mahrens@delphix.com>
illumos/illumos-gate@45818ee124
This is only a partial merge of respective ZFS infrastructure changes.
At this moment FreeBSD kernel has no those crypto algorithms, so the
parts of the code to enable them are commented out. When they are
implemented, it will be trivial to plug them in.
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Paul Dagnelie <pcd@delphix.com>
Reviewed by: Richard Elling <Richard.Elling@RichardElling.com>
Reviewed by: Xin Li <delphij@freebsd.org>
Reviewed by: Arne Jansen <sensille@gmx.net>
Approved by: Dan McDonald <danmcd@omniti.com>
Author: Matthew Ahrens <mahrens@delphix.com>
illumos/illumos-gate@9c3fd1216f
For more info, see:
- slides http://www.slideshare.net/MatthewAhrens/openzfs-send-and-receive
- video https://www.youtube.com/watch?v=iY44jPMvxog
- manpage changes (for zfs resume -s and zfs send -t)
- upcoming talk at the OpenZFS Developer Summit
The TL;DR is:
Use "zfs receive -s" to save the partially received state on failure.
On failure, get the receive token with "zfs get receive_resume_token <fs>"
Resume the send with "zfs send -t <token_value>"
Relnotes: yes
Previously, lockstat(1) would use a lock's address as its identifier when
consuming data describing lock contention and hold events. After collecting
the requested data, it would use ksyms(4) to resolve lock addresses to
names. Of course, this doesn't work too well for locks contained in
dynamically-allocated memory. This change modifies lockstat(1) to trace the
lock names obtained from the base struct lock_object instead, leading to
output that is generally much more useful.
This change also removes the -c option, which is used to coalesce data for
locks in an array. It's not possible to support this option without also
tracing lock addresses, and since lock arrays in which the lock names are
distinct are not very common in FreeBSD, it's simpler to just remove the
option.
Reviewed by: avg (earlier revision)
Differential Revision: https://reviews.freebsd.org/D3661
(userland portion that was not merged in r286677)
Update zdb to also print ltime, type, and level information
for these new style holes. Previously, only the logical birth
time would be printed.
This change reduces compiler warnings by removing duplicate defines
Line numbers are from r168404 (and r284648)
#define lbolt: lines 384 and 459 (531 and 648) (original was renamed later)
#define lbolt64: lines 385 and 460 (532 and 649) (original was renamed later)
#define gethrestime_sec: lines 390 and 465 (540 and 653)
uint64_t physmem: lines 402 and 463 (561 and 651)
Reviewed by: smh, delphij
Approved by: bapt (mentor)
Sponsored by: ScaleEngine Inc.
Differential Revision: https://reviews.freebsd.org/D2878
some ctfmerge crashes that started to occur on i386 weeks after r274569 was
committed. Some later investigation indicated that the crashes were caused
by malformed CTF info that led to a stack overflow. The issue with CTF
info in i386 kernels seems to have been resolved by r261246, which updated
libdwarf and libelf.
r274569 fixes a bug which caused duplicate types to appear in the kernel's
CTF info. This duplication generally does not cause problems when using
DTrace, but makes it easier to hit the limit of 2^15 - 1 distinct type
definitions in a CTF container.
MFC after: 2 weeks
Reviewed by: Toomas Soome <tsoome@me.com>
Reviewed by: George Wilson <george@delphix.com>
Reviewed by: Alex Reece <alex@delphix.com>
Approved by: Richard Lowe <richlowe@richlowe.net>
Author: Matthew Ahrens <mahrens@delphix.com>
illumos/illumos-gate@ca0cc3918a
A ZFS feature flags (large blocks) tracks its refcounts as the number of
datasets that have ever used the feature. Several features of this type
are planned to be added (new checksum functions). This code should be made
common infrastructure rather than duplicating the code for each feature.
5925 zfs receive -o origin=
Reviewed by: Prakash Surya <prakash.surya@delphix.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Author: Paul Dagnelie <pcd@delphix.com>
While running 'zfs recv' we noticed that every 128th 8K block required a
read. We were seeing that restore_write() was calling dmu_tx_hold_write()
and the indirect block was not cached. We should prefetch upcoming indirect
blocks to avoid having to go to disk and blocking the restore_write().
Allow an incremental send stream to be received as a clone, even if the
stream does not mark it as a clone.
lzc_send_space when source is a bookmark
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Christopher Siden <christopher.siden@delphix.com>
Reviewed by: Steven Hartland <killing@multiplay.co.uk>
Reviewed by: Bayard Bell <buffer.g.overflow@gmail.com>
Approved by: Albert Lee <trisk@nexenta.com>
Author: Max Grossman <max.grossman@delphix.com>
illumos/illumos-gate@643da460c8
Reviewed by: Prakash Surya <prakash.surya@delphix.com>
Reviewed by: Alex Reece <alex@delphix.com>
Reviewed by: George Wilson <george@delphix.com>
Reviewed by: Will Andrews <will@freebsd.org>
Reviewed by: Simon Klinkert <simon.klinkert@gmail.com>
Approved by: Gordon Ross <gwr@nexenta.com>
Author: Matthew Ahrens <mahrens@delphix.com>
illumos/illumos-gate@732885fca0
Reviewed by: Christopher Siden <christopher.siden@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Bayard Bell <buffer.g.overflow@gmail.com>
Approved by: Albert Lee <trisk@omniti.com>
Author: Matthew Ahrens <mahrens@delphix.com>
illumos/illumos-gate@98110f08fa
5661 ZFS: "compression = on" should use lz4 if feature is enabled
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Josef 'Jeff' Sipek <jeffpc@josefsipek.net>
Reviewed by: Xin LI <delphij@freebsd.org>
Approved by: Robert Mustacchi <rm@joyent.com>
Author: Justin T. Gibbs <justing@spectralogic.com>
illumos/illumos-gate@db1741f555
- Rather than assuming that a process is listening on 127.0.0.1:22, use
nc(1) to find an available port and bind to it for the duration of the
test.
MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division
../common/libzfs_import.c, line 1077, function zpool_open_func
illumos/illumos-gate@bd0f709169
Author: Andrew Stormont <andyjstormont@gmail.com>
MFC after: 13 days
0. For spin events report time spent spinning, not a loop count.
While loop count is much easier and cheaper to obtain it is hard
to reason about the reported numbers, espcially for adaptive locks
where both spinning and sleeping can happen.
So, it's better to compare apples and apples.
1. Teach lockstat about FreeBSD rw locks.
This is done in part by changing the corresponding probes
and in part by changing what probes lockstat should expect.
2. Teach lockstat that rw locks are adaptive and can spin on FreeBSD.
3. Report lock acquisition events for successful rw try-lock operations.
4. Teach lockstat about FreeBSD sx locks.
Reporting of events for those locks completely mirrors
rw locks.
5. Report spin and block events before acquisition event.
This is behavior documented for the upstream, so it makes sense to stick
to it. Note that because of FreeBSD adaptive lock implementations
both the spin and block events may be reported for the same acquisition
while the upstream reports only one of them.
Differential Revision: https://reviews.freebsd.org/D2727
Reviewed by: markj
MFC after: 17 days
Relnotes: yes
Sponsored by: ClusterHQ
The "depends_on module" pragma can be used to declare a dependency on a
DTrace module, which for kernel probes corresponds to a KLD. Such
dependencies cannot be checked if the KLD is compiled into the kernel.
Therefore, allow a module dependency to be satisfied if either a kernel
module or a KLD with the specified name is loaded.
Differential Revision: https://reviews.freebsd.org/D2653
Reviewed by: gnn, rpaulo
Reported by: gnn
return an error if one of the depends_on directives in a library is not
satisfied. In this case, libdtrace is supposed to ignore the library and
carry on. However, the remainder of the library may still be buffered by
the lexer, causing libdtrace to erroneously continue processing it on the
next call to yyparse(). Fix this by explicitly flushing the input buffer
each time the compiler state is reset.
MFC after: 3 weeks
list of pending dynamic type definitions, a match on the type name is not
sufficient - we need to compare the type encodings as well. For example,
bitfields have their own distinct type definitions which share the name of
the underlying integer type, and these types aren't generally
interchangeable.
This bug was causing the following libdtrace error when attempting to trace
the th_flags member of a struct tcphdr:
cg: bad field: off 104 type <32877> bits 539620016
Reported by: rwatson
MFC after: 3 weeks
The format of these pages is somewhat experimental, so they may be subject
to further tweaking.
Differential Revision: https://reviews.freebsd.org/D2170
Reviewed by: bcr, rpaulo
MFC after: 2 weeks
Passing "-x lazyload" to dtrace -G during compilation causes dtrace(1) to
not link drti.o into the output object file, so the USDT probes are not created
during process startup. Instead, dtrace(1) will automatically discover and
create probes on the process' behalf when attaching.
Differential Revision: https://reviews.freebsd.org/D2203
Reviewed by: rpaulo
MFC after: 1 month
ctf_add_type() first performs a by-name lookup of the type in the
destination container. If this lookup returns a forward declaration for an
enum, struct, or union, reset dst_type back to CTF_ERR, indicating that the
source type is not in fact present in the destination container. This
ensures that ctf_add_type() will also search the destination container's
dynamic type list for the source type.
Without this change, a pair of mutually recursive struct definitions could
cause infinite recursion in ctf_add_type() if the destination container
only contained forward declarations for the struct types: ctf_add_type()
recursively calls itself on each struct member's type, and the forward
declarations meant that the dynamic type list search would be skipped.
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
indicates the range of type indices which have been committed to the
container by ctf_update(). However, the top bit of the dtd_type field is
not part of the type index; rather, it is a flag used to indicate that the
corresponding CTF container is a parent. This is why the maximum CTF type
index is 2^15 - 1 rather than 2^16 - 1. Therefore, this flag must be masked
off (using the CTF_TYPE_TO_INDEX macro) when comparing a type index with the
ctf_dtoldid field of a container.
This bug was causing libctf to erroneously free committed type definitions
in ctf_discard(). libdtrace holds some references to such types, resulting
in a use-after-free.
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
we end up skipping a dynamic type because it has already been committed to
the container, we would previously either set the loop variable to an
uninitialized local variable, or set it to itself, resulting in an infinite
loop.
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
