598 Commits

Author SHA1 Message Date
delphij
15c18a3cba MFV r286224: 5695 dmu_sync'ed holes do not retain birth time
(userland portion that was not merged in r286677)

Update zdb to also print ltime, type, and level information
for these new style holes. Previously, only the logical birth
time would be printed.
2015-09-14 06:10:49 +00:00
delphij
0187024b8d MFV r277429:
Document -S option when zfs inherit fails on quota and
in manual pages.

Illumos ZFS issues:

    5410 Document -S option to zfs inherit
    https://illumos.org/issues/5410

    5412 Mention -S option when zfs inherit fails on quota
    https://illumos.org/issues/5412

illumos/illumos-gate@5ff8cfa92e

MFC after:	2 weeks
2015-09-14 06:00:50 +00:00
delphij
e9ff0b2b14 Fix build (r287703). Lesson learned: no matter how a change looks like an
innocent one, always do a build test first.

Pointy hat to:	delphij
2015-09-12 09:28:02 +00:00
allanjude
97c6fe6d1e Remove duplicate defines introduced in initial ZFS import (r168404)
This change reduces compiler warnings by removing duplicate defines

Line numbers are from r168404 (and r284648)
#define lbolt: lines 384 and 459 (531 and 648) (original was renamed later)
#define lbolt64: lines 385 and 460 (532 and 649) (original was renamed later)
#define gethrestime_sec: lines 390 and 465 (540 and 653)
uint64_t physmem: lines 402 and 463 (561 and 651)

Reviewed by:	smh, delphij
Approved by:	bapt (mentor)
Sponsored by:	ScaleEngine Inc.
Differential Revision:	https://reviews.freebsd.org/D2878
2015-08-31 22:36:17 +00:00
markj
251669654a Re-apply r274569. It was reverted in r276848 since that appeared to fix
some ctfmerge crashes that started to occur on i386 weeks after r274569 was
committed. Some later investigation indicated that the crashes were caused
by malformed CTF info that led to a stack overflow. The issue with CTF
info in i386 kernels seems to have been resolved by r261246, which updated
libdwarf and libelf.

r274569 fixes a bug which caused duplicate types to appear in the kernel's
CTF info. This duplication generally does not cause problems when using
DTrace, but makes it easier to hit the limit of 2^15 - 1 distinct type
definitions in a CTF container.

MFC after:	2 weeks
2015-08-28 00:38:18 +00:00
delphij
2c7c53008a Plug a memory leak.
MFC after:	2 weeks
2015-08-13 18:45:52 +00:00
mav
95e93478d4 MFV 286707: 5959 clean up per-dataset feature count code
Reviewed by: Toomas Soome <tsoome@me.com>
Reviewed by: George Wilson <george@delphix.com>
Reviewed by: Alex Reece <alex@delphix.com>
Approved by: Richard Lowe <richlowe@richlowe.net>
Author: Matthew Ahrens <mahrens@delphix.com>

illumos/illumos-gate@ca0cc3918a

A ZFS feature flags (large blocks) tracks its refcounts as the number of
datasets that have ever used the feature. Several features of this type
are planned to be added (new checksum functions). This code should be made
common infrastructure rather than duplicating the code for each feature.
2015-08-12 23:59:17 +00:00
mav
99cadf9eed MFV r286704: 5960 zfs recv should prefetch indirect blocks
5925 zfs receive -o origin=

Reviewed by: Prakash Surya <prakash.surya@delphix.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Author: Paul Dagnelie <pcd@delphix.com>

While running 'zfs recv' we noticed that every 128th 8K block required a
read. We were seeing that restore_write() was calling dmu_tx_hold_write()
and the indirect block was not cached. We should prefetch upcoming indirect
blocks to avoid having to go to disk and blocking the restore_write().

Allow an incremental send stream to be received as a clone, even if the
stream does not mark it as a clone.
2015-08-12 22:41:06 +00:00
mav
53d48e6cc1 MFV r286682: 5765 add support for estimating send stream size with
lzc_send_space when source is a bookmark

Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Christopher Siden <christopher.siden@delphix.com>
Reviewed by: Steven Hartland <killing@multiplay.co.uk>
Reviewed by: Bayard Bell <buffer.g.overflow@gmail.com>
Approved by: Albert Lee <trisk@nexenta.com>
Author: Max Grossman <max.grossman@delphix.com>

illumos/illumos-gate@643da460c8
2015-08-12 18:23:08 +00:00
mav
7473557e25 MFV 286602: 5810 zdb should print details of bpobj
Reviewed by: Prakash Surya <prakash.surya@delphix.com>
Reviewed by: Alex Reece <alex@delphix.com>
Reviewed by: George Wilson <george@delphix.com>
Reviewed by: Will Andrews <will@freebsd.org>
Reviewed by: Simon Klinkert <simon.klinkert@gmail.com>
Approved by: Gordon Ross <gwr@nexenta.com>
Author: Matthew Ahrens <mahrens@delphix.com>

illumos/illumos-gate@732885fca0
2015-08-10 21:32:40 +00:00
mav
8fa4822e3d MFV 286586: 5746 more checksumming in zfs send
Reviewed by: Christopher Siden <christopher.siden@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Bayard Bell <buffer.g.overflow@gmail.com>
Approved by: Albert Lee <trisk@omniti.com>
Author: Matthew Ahrens <mahrens@delphix.com>

illumos/illumos-gate@98110f08fa
2015-08-10 19:32:58 +00:00
mav
3a572fa507 MFV 286546:
5661 ZFS: "compression = on" should use lz4 if feature is enabled

Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Josef 'Jeff' Sipek <jeffpc@josefsipek.net>
Reviewed by: Xin LI <delphij@freebsd.org>
Approved by: Robert Mustacchi <rm@joyent.com>
Author: Justin T. Gibbs <justing@spectralogic.com>

illumos/illumos-gate@db1741f555
2015-08-09 20:02:16 +00:00
markj
a9f0d5f01d Don't hardcode the module or function component of lockstat probes.
MFC after:	1 week
2015-08-02 00:24:21 +00:00
markj
0de52e9e95 - Remove hardcoded paths for the perl executable.
- Rather than assuming that a process is listening on 127.0.0.1:22, use
  nc(1) to find an available port and bind to it for the duration of the
  test.

MFC after:	1 week
Sponsored by:	EMC / Isilon Storage Division
2015-08-02 00:23:18 +00:00
markj
f6c32e46b9 Perform bounds checking when constructing a format string.
This was detected by the FORTIFY_SOURCE build.

PR:		201657
Reported by:	pfg
MFC after:	2 weeks
2015-08-02 00:18:48 +00:00
gjb
ce02aeb9eb Fix a rendering issue in the zfs(8) manual.
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2015-07-30 19:34:24 +00:00
markj
2092d6ebaa Consistently use a reader/writer flag for lockstat probes in rwlock(9) and
sx(9), rather than using the probe function name to determine whether a
given lock is a read lock or a write lock. Update lockstat(1) accordingly.
2015-07-19 22:24:33 +00:00
br
70d4d5a42d First cut of DTrace for AArch64.
Reviewed by:	andrew, emaste
Sponsored by:	ARM Limited
Differential Revision:	https://reviews.freebsd.org/D2738
2015-07-01 15:51:11 +00:00
avg
34e109f495 zfs clone should not mount the clone if canmount == noauto
Creation of a new filesystem does not imply an intent to mount it.

Since canmount property is not inherited and its default value is 'on',
the only scenario where this matters is zfs clone -o canmount=noauto.
zfs create -o canmount=noauto already does not mount the new filesystem.

Also see:
https://www.illumos.org/issues/5984
https://reviews.csiden.org/r/228/
dd0e0e69f5
https://github.com/zfsonlinux/zfs/issues/2241

Reviewed by:	mahrens
MFC after:	8 days
Sponsored by:	ClusterHQ
2015-06-12 11:21:35 +00:00
avg
18e52c031c MFV r284042: 1778 Assertion failed: rn->rn_nozpool == B_FALSE, file
../common/libzfs_import.c, line 1077, function zpool_open_func

illumos/illumos-gate@bd0f709169

Author:		Andrew Stormont <andyjstormont@gmail.com>
MFC after:	13 days
2015-06-12 11:16:43 +00:00
avg
35511df052 MFV r284030: 5818 zfs {ref}compressratio is incorrect with 4k sector size
illumos/illumos-gate@81cd5c555f

Author:	Matthew Ahrens <mahrens@delphix.com>
MFC after:	17 days
2015-06-12 10:57:05 +00:00
avg
2d008d4579 several lockstat improvements
0. For spin events report time spent spinning, not a loop count.
While loop count is much easier and cheaper to obtain it is hard
to reason about the reported numbers, espcially for adaptive locks
where both spinning and sleeping can happen.
So, it's better to compare apples and apples.

1. Teach lockstat about FreeBSD rw locks.
This is done in part by changing the corresponding probes
and in part by changing what probes lockstat should expect.

2. Teach lockstat that rw locks are adaptive and can spin on FreeBSD.

3. Report lock acquisition events for successful rw try-lock operations.

4. Teach lockstat about FreeBSD sx locks.
Reporting of events for those locks completely mirrors
rw locks.

5. Report spin and block events before acquisition event.
This is behavior documented for the upstream, so it makes sense to stick
to it.  Note that because of FreeBSD adaptive lock implementations
both the spin and block events may be reported for the same acquisition
while the upstream reports only one of them.

Differential Revision:	https://reviews.freebsd.org/D2727
Reviewed by:	markj
MFC after:	17 days
Relnotes:	yes
Sponsored by:	ClusterHQ
2015-06-12 10:01:24 +00:00
araujo
aca9f1ba41 At revision r247852 accidentally was removed from print the variable obj.
While here, declare some global variables as static to silence clang
warnings.

Differential Revision:	D2722
Reviewed by:		delphij, mahrens
2015-06-12 02:16:14 +00:00
markj
07f201748f libdtrace: allow D libraries to declare dependencies on kernel modules
The "depends_on module" pragma can be used to declare a dependency on a
DTrace module, which for kernel probes corresponds to a KLD. Such
dependencies cannot be checked if the KLD is compiled into the kernel.
Therefore, allow a module dependency to be satisfied if either a kernel
module or a KLD with the specified name is loaded.

Differential Revision:	https://reviews.freebsd.org/D2653
Reviewed by:	gnn, rpaulo
Reported by:	gnn
2015-06-06 16:47:45 +00:00
markj
e9847d6bea lockstat(1): document the -V option.
MFC after:	3 days
2015-05-25 01:18:46 +00:00
markj
7037e8b851 Fix a typo that snuck in with r283024, and remove the EXFAIL annotation from
a test which now passes as a result of that change.
2015-05-17 23:09:58 +00:00
markj
78dbee3100 Respect the libdir option when linking drti.o, rather than hardcoding the
default path of /usr/lib(32)/dtrace.

MFC after:	3 weeks
2015-05-17 23:08:01 +00:00
markj
6274bab2e0 As dtrace(1) processes D libraries under /usr/lib/dtrace, the compiler may
return an error if one of the depends_on directives in a library is not
satisfied. In this case, libdtrace is supposed to ignore the library and
carry on. However, the remainder of the library may still be buffered by
the lexer, causing libdtrace to erroneously continue processing it on the
next call to yyparse(). Fix this by explicitly flushing the input buffer
each time the compiler state is reset.

MFC after:	3 weeks
2015-05-17 03:59:08 +00:00
markj
48547f12fe When in lazyload mode, write the DOF to a temporary file and rename it
rather than writing directly to the output file.

CID:	1147172
2015-05-17 03:50:42 +00:00
markj
2c4686751c ctf_add_type(): when looking up an integer or floating point type in the
list of pending dynamic type definitions, a match on the type name is not
sufficient - we need to compare the type encodings as well. For example,
bitfields have their own distinct type definitions which share the name of
the underlying integer type, and these types aren't generally
interchangeable.

This bug was causing the following libdtrace error when attempting to trace
the th_flags member of a struct tcphdr:
  cg: bad field: off 104 type <32877> bits 539620016

Reported by:	rwatson
MFC after:	3 weeks
2015-05-10 21:39:24 +00:00
avg
311ef80759 dump_nvlist: handle DATA_TYPE_BOOLEAN_ARRAY
To do:	upstream (https://www.illumos.org/issues/5752)
MFC after:	10 days
2015-04-28 06:33:42 +00:00
markj
78b4348c8c Add manual pages for the io, ip, proc, sched, tcp and udp DTrace providers.
The format of these pages is somewhat experimental, so they may be subject
to further tweaking.

Differential Revision:	https://reviews.freebsd.org/D2170
Reviewed by:	bcr, rpaulo
MFC after:	2 weeks
2015-04-18 21:00:36 +00:00
markj
1d6ffde4f4 libdtrace: add support for lazyload mode.
Passing "-x lazyload" to dtrace -G during compilation causes dtrace(1) to
not link drti.o into the output object file, so the USDT probes are not created
during process startup. Instead, dtrace(1) will automatically discover and
create probes on the process' behalf when attaching.

Differential Revision:	https://reviews.freebsd.org/D2203
Reviewed by:		rpaulo
MFC after:		1 month
2015-04-08 02:36:37 +00:00
markj
b940cb0c6b Fix a misparenthesization that could cause a crash if TERM is not set.
Reported by:	Coverity (internal)
MFC after:	3 days
Sponsored by:	EMC / Isilon Storage Division
2015-03-31 02:49:05 +00:00
markj
6968ab0dde Fix ping(8) and ping6(8) usage in a couple of ip provider tests, and
update expected test output to reflect differences in default TTL and
payload length.

MFC after:	1 week
2015-03-30 04:06:36 +00:00
markj
2be5269c48 Fix ping(8) usage in funcs/tst.system.d so that the test actually completes.
MFC after:	1 week
2015-03-30 04:03:01 +00:00
markj
09331b0ed0 tcp provider tests: sleep for a bit after closing the socket so that
libdtrace has a chance to capture the traced data.

MFC after:	1 week
2015-03-28 01:11:18 +00:00
markj
bfd93bccca Add a missing format string argument.
PR:		197391
MFC after:	3 days
2015-03-15 21:57:44 +00:00
markj
1c0a61aa61 When copying a type from a source CTF container to a destination container,
ctf_add_type() first performs a by-name lookup of the type in the
destination container. If this lookup returns a forward declaration for an
enum, struct, or union, reset dst_type back to CTF_ERR, indicating that the
source type is not in fact present in the destination container. This
ensures that ctf_add_type() will also search the destination container's
dynamic type list for the source type.

Without this change, a pair of mutually recursive struct definitions could
cause infinite recursion in ctf_add_type() if the destination container
only contained forward declarations for the struct types: ctf_add_type()
recursively calls itself on each struct member's type, and the forward
declarations meant that the dynamic type list search would be skipped.

MFC after:	2 weeks
Sponsored by:	EMC / Isilon Storage Division
2015-03-11 00:01:39 +00:00
markj
564efa5c09 CTF containers use the ctf_dtoldid field as a threshold type index which
indicates the range of type indices which have been committed to the
container by ctf_update(). However, the top bit of the dtd_type field is
not part of the type index; rather, it is a flag used to indicate that the
corresponding CTF container is a parent. This is why the maximum CTF type
index is 2^15 - 1 rather than 2^16 - 1. Therefore, this flag must be masked
off (using the CTF_TYPE_TO_INDEX macro) when comparing a type index with the
ctf_dtoldid field of a container.

This bug was causing libctf to erroneously free committed type definitions
in ctf_discard(). libdtrace holds some references to such types, resulting
in a use-after-free.

MFC after:	2 weeks
Sponsored by:	EMC / Isilon Storage Division
2015-03-10 21:08:58 +00:00
markj
c50bfa1dd2 ctf_discard(): fetch the next list element before restarting the loop. If
we end up skipping a dynamic type because it has already been committed to
the container, we would previously either set the loop variable to an
uninitialized local variable, or set it to itself, resulting in an infinite
loop.

MFC after:	2 weeks
Sponsored by:	EMC / Isilon Storage Division
2015-03-10 20:52:03 +00:00
markj
6bcf6bdf56 MFV r279822:
This merge is effectively a no-op since parts of it are already present
in FreeBSD, and the rest is incorrect since gelf_newehdr(3) and
gelf_newphdr(3) return pointers on FreeBSD rather than integers.

Illumos issue:
    5589 improper use of NULL in tools/ctf

MFC after:	3 days
2015-03-10 20:43:16 +00:00
gnn
73a716f42d Summary: Remove the need for `mpid which is Illumos/Solaris specific. 2015-03-08 21:51:37 +00:00
markj
515a3333a1 Use the -shared option to create a shared library.
MFC after:	1 week
2015-02-28 23:14:35 +00:00
markj
7a6db9bb6f Remove a leading tab that causes a make(1) error when running the test.
MFC after:	1 week
2015-02-28 23:11:18 +00:00
markj
eed96d83d4 Only kill sleep processes that were forked from the test script.
MFC after:	1 week
2015-02-28 23:10:26 +00:00
markj
7907d899e1 Stop hard-coding an incorrect path to rm(1).
MFC after:	1 week
2015-02-28 23:09:01 +00:00
delphij
139773cf3f Set altroot if the user have specified it via -o altroot.
MFC after:	2 weeks
2015-02-27 19:13:13 +00:00
markj
766a186dd8 Remove drti.o's dependency on libelf. This makes it possible to add DTrace
probes to userland programs and libraries without also needing to link
libelf.

dtrace -G places the __SUNW_dof symbol at the beginning of the DOF (DTrace
probe and provider metdata) section in the generated object file; drti.o
now just uses this symbol to locate the section. A complication occurs
when multiple dtrace-generated object files are linked together, since the
__SUNW_dof symbol defined in each file is global. This is handled by
using objcopy(1) to convert __SUNW_dof to a local symbol once drti.o has
been linked with the generated object file. Upstream, this is done using a
linker feature not present in GNU ld.

Differential Revision:	https://reviews.freebsd.org/D1757
Reviewed by:	rpaulo
MFC after:	1 month
Relnotes:	yes
2015-02-18 03:54:54 +00:00
markj
81644b962a Tweak the fds test program so that it actually compiles. Also use 0 instead
of -1 for the bogus ioctl command so that dmesg doesn't get spammed with
sign extension warnings when the test program runs.

MFC after:	1 week
2015-02-14 00:03:43 +00:00