Commit Graph

743 Commits

Author SHA1 Message Date
Bryan Drewery
aa92269e46 Add more SUBDIR_PARALLEL.
MFC after:	3 weeks
Sponsored by:	EMC / Isilon Storage Division
2015-10-15 22:55:08 +00:00
Alexander Motin
c70e61feed MFV r289312: 2605 want to resume interrupted zfs send
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Paul Dagnelie <pcd@delphix.com>
Reviewed by: Richard Elling <Richard.Elling@RichardElling.com>
Reviewed by: Xin Li <delphij@freebsd.org>
Reviewed by: Arne Jansen <sensille@gmx.net>
Approved by: Dan McDonald <danmcd@omniti.com>
Author: Matthew Ahrens <mahrens@delphix.com>

illumos/illumos-gate@9c3fd1216f

For more info, see:
 - slides http://www.slideshare.net/MatthewAhrens/openzfs-send-and-receive
 - video https://www.youtube.com/watch?v=iY44jPMvxog
 - manpage changes (for zfs resume -s and zfs send -t)
 - upcoming talk at the OpenZFS Developer Summit

The TL;DR is:
Use "zfs receive -s" to save the partially received state on failure.
On failure, get the receive token with "zfs get receive_resume_token <fs>"
Resume the send with "zfs send -t <token_value>"

Relnotes:	yes
2015-10-15 08:47:32 +00:00
Alexander Motin
2bfd7091a5 MFV r289311: 5764 "zfs send -nv" directs output to stderr
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Paul Dagnelie <paul.dagnelie@delphix.com>
Reviewed by: Basil Crow <basil.crow@delphix.com>
Reviewed by: Steven Hartland <killing@multiplay.co.uk>
Reviewed by: Bayard Bell <buffer.g.overflow@gmail.com>
Approved by: Dan McDonald <danmcd@omniti.com>
Author: Manoj Joseph <manoj.joseph@delphix.com>

illumos/illumos-gate@dc5f28a3c3
2015-10-14 11:52:58 +00:00
Enji Cooper
b2d48be1bc Refactor the test/ Makefiles after recent changes to bsd.test.mk (r289158) and
netbsd-tests.test.mk (r289151)

- Eliminate explicit OBJTOP/SRCTOP setting
- Convert all ad hoc NetBSD test integration over to netbsd-tests.test.mk
- Remove unnecessary TESTSDIR setting
- Use SRCTOP where possible for clarity

MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Divison
2015-10-12 08:16:03 +00:00
Mark Johnston
3142b37664 Have lockstat(1) trace locks by name rather than by address.
Previously, lockstat(1) would use a lock's address as its identifier when
consuming data describing lock contention and hold events. After collecting
the requested data, it would use ksyms(4) to resolve lock addresses to
names. Of course, this doesn't work too well for locks contained in
dynamically-allocated memory. This change modifies lockstat(1) to trace the
lock names obtained from the base struct lock_object instead, leading to
output that is generally much more useful.

This change also removes the -c option, which is used to coalesce data for
locks in an array. It's not possible to support this option without also
tracing lock addresses, and since lock arrays in which the lock names are
distinct are not very common in FreeBSD, it's simpler to just remove the
option.

Reviewed by:	avg (earlier revision)
Differential Revision:	https://reviews.freebsd.org/D3661
2015-09-30 05:46:56 +00:00
Mark Johnston
b22ef02080 Update DTrace test makefiles after r288415. 2015-09-30 05:25:39 +00:00
Mark Johnston
f7c3db2537 MFV r288408:
6266 harden dtrace_difo_chunksize() with respect to malicious DIF

illumos/illumos-gate@395c7a3dcf

Reviewed by: Alex Wilson <alex.wilson@joyent.com>
Reviewed by: Dan McDonald <danmcd@omniti.com>
Approved by: Garrett D'Amore <garrett@damore.org>
Author: Bryan Cantrill <bryan@joyent.com>

MFC after:	1 week
2015-09-30 05:24:22 +00:00
Mark Johnston
c5af5ada34 Stop hard-coding a 32-bit data model for USDT tests, and just use the native
model. This was causing many of the tests to fail on amd64 since USDT
support for 32-bit programs is currently non-functional.

MFC after:	1 week
2015-09-30 04:48:56 +00:00
Andriy Gapon
9b977fcea2 define aok in libnvpair which is linked to all zfs libraries that need aok
This removes the circular dependency of libnvpair on libzfs / libzpool.

PR:		199811
Obtained from:	bapt
MFC after:	23 days
2015-09-28 15:25:36 +00:00
Andriy Gapon
9875367b9a remove unused sgsmsg utility (originally imported from opensolaris)
MFC after:	25 days
2015-09-28 12:38:57 +00:00
Andriy Gapon
65b71c48d2 remove an extra copy of avl.c from illumos contrib code
MFC after:	20 days
2015-09-28 12:23:10 +00:00
Bryan Drewery
54c7d75a06 META_MODE: Remove DEP_RELDIR from Makefile.depend files.
This has not been needed since r284171 in projects/bmake.

Sponsored by:	EMC / Isilon Storage Division
2015-09-25 19:26:08 +00:00
Bryan Drewery
144cd65bca Use FILES for installing the dtrace scripts.
The BSD.usr.dist mtree always creates /usr/lib/dtrace so there is no
need to check if it exists.

The FILES mechanism already supports LIBRARIES_ONLY.

Sponsored by:	EMC / Isilon Storage Division
2015-09-18 21:18:44 +00:00
Mark Johnston
c50c331896 Remove an unneeded typedef of ip6_t from the DTrace ip provider library.
It causes an error when ipfilter is enabled, since ipl.ko contains an
identical typedef.

PR:		203092
MFC after:	1 week
2015-09-15 05:16:26 +00:00
Xin LI
cfb9fa4ec9 MFV r286224: 5695 dmu_sync'ed holes do not retain birth time
(userland portion that was not merged in r286677)

Update zdb to also print ltime, type, and level information
for these new style holes. Previously, only the logical birth
time would be printed.
2015-09-14 06:10:49 +00:00
Xin LI
44ec48bc55 MFV r277429:
Document -S option when zfs inherit fails on quota and
in manual pages.

Illumos ZFS issues:

    5410 Document -S option to zfs inherit
    https://illumos.org/issues/5410

    5412 Mention -S option when zfs inherit fails on quota
    https://illumos.org/issues/5412

illumos/illumos-gate@5ff8cfa92e

MFC after:	2 weeks
2015-09-14 06:00:50 +00:00
George V. Neville-Neil
5d06879adb dd DTrace probe points, translators and a corresponding script
to provide the TCPDEBUG functionality with pure DTrace.

Reviewed by:	rwatson
MFC after:	2 weeks
Sponsored by:	Limelight Networks
Differential Revision:	D3530
2015-09-13 15:50:55 +00:00
Xin LI
530ecabd3d Fix build (r287703). Lesson learned: no matter how a change looks like an
innocent one, always do a build test first.

Pointy hat to:	delphij
2015-09-12 09:28:02 +00:00
Allan Jude
73457dc357 Remove duplicate defines introduced in initial ZFS import (r168404)
This change reduces compiler warnings by removing duplicate defines

Line numbers are from r168404 (and r284648)
#define lbolt: lines 384 and 459 (531 and 648) (original was renamed later)
#define lbolt64: lines 385 and 460 (532 and 649) (original was renamed later)
#define gethrestime_sec: lines 390 and 465 (540 and 653)
uint64_t physmem: lines 402 and 463 (561 and 651)

Reviewed by:	smh, delphij
Approved by:	bapt (mentor)
Sponsored by:	ScaleEngine Inc.
Differential Revision:	https://reviews.freebsd.org/D2878
2015-08-31 22:36:17 +00:00
Mark Johnston
bd81e07d27 Re-apply r274569. It was reverted in r276848 since that appeared to fix
some ctfmerge crashes that started to occur on i386 weeks after r274569 was
committed. Some later investigation indicated that the crashes were caused
by malformed CTF info that led to a stack overflow. The issue with CTF
info in i386 kernels seems to have been resolved by r261246, which updated
libdwarf and libelf.

r274569 fixes a bug which caused duplicate types to appear in the kernel's
CTF info. This duplication generally does not cause problems when using
DTrace, but makes it easier to hit the limit of 2^15 - 1 distinct type
definitions in a CTF container.

MFC after:	2 weeks
2015-08-28 00:38:18 +00:00
Warner Losh
c404eb64d6 Sparc64 is the odd-man out, so form the if that way rather than
listing everybody else.
2015-08-23 23:12:30 +00:00
Xin LI
fb90888521 Plug a memory leak.
MFC after:	2 weeks
2015-08-13 18:45:52 +00:00
Alexander Motin
0d0def87fe MFV 286707: 5959 clean up per-dataset feature count code
Reviewed by: Toomas Soome <tsoome@me.com>
Reviewed by: George Wilson <george@delphix.com>
Reviewed by: Alex Reece <alex@delphix.com>
Approved by: Richard Lowe <richlowe@richlowe.net>
Author: Matthew Ahrens <mahrens@delphix.com>

illumos/illumos-gate@ca0cc3918a

A ZFS feature flags (large blocks) tracks its refcounts as the number of
datasets that have ever used the feature. Several features of this type
are planned to be added (new checksum functions). This code should be made
common infrastructure rather than duplicating the code for each feature.
2015-08-12 23:59:17 +00:00
Alexander Motin
b696497df0 MFV r286704: 5960 zfs recv should prefetch indirect blocks
5925 zfs receive -o origin=

Reviewed by: Prakash Surya <prakash.surya@delphix.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Author: Paul Dagnelie <pcd@delphix.com>

While running 'zfs recv' we noticed that every 128th 8K block required a
read. We were seeing that restore_write() was calling dmu_tx_hold_write()
and the indirect block was not cached. We should prefetch upcoming indirect
blocks to avoid having to go to disk and blocking the restore_write().

Allow an incremental send stream to be received as a clone, even if the
stream does not mark it as a clone.
2015-08-12 22:41:06 +00:00
Alexander Motin
ebf527de10 MFV r286682: 5765 add support for estimating send stream size with
lzc_send_space when source is a bookmark

Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Christopher Siden <christopher.siden@delphix.com>
Reviewed by: Steven Hartland <killing@multiplay.co.uk>
Reviewed by: Bayard Bell <buffer.g.overflow@gmail.com>
Approved by: Albert Lee <trisk@nexenta.com>
Author: Max Grossman <max.grossman@delphix.com>

illumos/illumos-gate@643da460c8
2015-08-12 18:23:08 +00:00
Alexander Motin
799f47828d MFV 286602: 5810 zdb should print details of bpobj
Reviewed by: Prakash Surya <prakash.surya@delphix.com>
Reviewed by: Alex Reece <alex@delphix.com>
Reviewed by: George Wilson <george@delphix.com>
Reviewed by: Will Andrews <will@freebsd.org>
Reviewed by: Simon Klinkert <simon.klinkert@gmail.com>
Approved by: Gordon Ross <gwr@nexenta.com>
Author: Matthew Ahrens <mahrens@delphix.com>

illumos/illumos-gate@732885fca0
2015-08-10 21:32:40 +00:00
Alexander Motin
57f7c5acf5 MFV 286586: 5746 more checksumming in zfs send
Reviewed by: Christopher Siden <christopher.siden@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Bayard Bell <buffer.g.overflow@gmail.com>
Approved by: Albert Lee <trisk@omniti.com>
Author: Matthew Ahrens <mahrens@delphix.com>

illumos/illumos-gate@98110f08fa
2015-08-10 19:32:58 +00:00
Alexander Motin
4ff9527edc MFV 286546:
5661 ZFS: "compression = on" should use lz4 if feature is enabled

Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Josef 'Jeff' Sipek <jeffpc@josefsipek.net>
Reviewed by: Xin LI <delphij@freebsd.org>
Approved by: Robert Mustacchi <rm@joyent.com>
Author: Justin T. Gibbs <justing@spectralogic.com>

illumos/illumos-gate@db1741f555
2015-08-09 20:02:16 +00:00
Mark Johnston
7e518a6648 - Use an explicit "depends_on module kernel" guard in DTrace libraries that
reference types defined in the kernel. Otherwise dtrace(1) expects to find
  CTF definitions for all referenced types, which is not very reasonable
  when it is being used in a build environment. This was previously worked
  around by adding "-x nolibs" to dtrace -h or -G invocations, but as of
  r283025, dtrace(1) actually handles dependencies properly, so this is no
  longer necessary.
- Remove "pragma ident" directives from DTrace libraries, as they're being
  phased out upstream as well.

Submitted by:	Krister Johansen <Krister.Johansen@isilon.com> [1]
MFC after:	1 week
Sponsored by:	EMC / Isilon Storage Division
2015-08-07 19:56:22 +00:00
Mark Johnston
d912066c36 Add a src.conf option to build and install the DTrace test suite.
Reviewed by:	gnn, ngie
Differential Revision:	https://reviews.freebsd.org/D3195
2015-08-02 00:37:33 +00:00
Mark Johnston
88f578841f Don't hardcode the module or function component of lockstat probes.
MFC after:	1 week
2015-08-02 00:24:21 +00:00
Mark Johnston
6b9db41be6 - Remove hardcoded paths for the perl executable.
- Rather than assuming that a process is listening on 127.0.0.1:22, use
  nc(1) to find an available port and bind to it for the duration of the
  test.

MFC after:	1 week
Sponsored by:	EMC / Isilon Storage Division
2015-08-02 00:23:18 +00:00
Mark Johnston
61ab25cd3d Perform bounds checking when constructing a format string.
This was detected by the FORTIFY_SOURCE build.

PR:		201657
Reported by:	pfg
MFC after:	2 weeks
2015-08-02 00:18:48 +00:00
Glen Barber
f5827b16eb Fix a rendering issue in the zfs(8) manual.
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2015-07-30 19:34:24 +00:00
Mark Johnston
de2c95cc00 Consistently use a reader/writer flag for lockstat probes in rwlock(9) and
sx(9), rather than using the probe function name to determine whether a
given lock is a read lock or a write lock. Update lockstat(1) accordingly.
2015-07-19 22:24:33 +00:00
Mariusz Zaborski
306a82f8f4 Rename zfs nvpair files to not colidate with our nvlist.
PR:		201356
Approved by:	pjd (mentor)
2015-07-09 21:53:40 +00:00
Ruslan Bukin
b78ee15e9f First cut of DTrace for AArch64.
Reviewed by:	andrew, emaste
Sponsored by:	ARM Limited
Differential Revision:	https://reviews.freebsd.org/D2738
2015-07-01 15:51:11 +00:00
Baptiste Daroussin
18b2ee82db Revert r284417 it is not necessary anymore 2015-06-15 19:28:07 +00:00
Baptiste Daroussin
4232f82668 Enforce overwritting SHLIBDIR
Since METAMODE has been added, sys.mk loads bsd.mkopt.mk which ends load loading
bsd.own.mk which then defines SHLIBDIR before all the Makefile.inc everywhere.

This makes /lib being populated again.

Reported by:	many
2015-06-15 15:34:20 +00:00
Baptiste Daroussin
13500839f2 Fix circular dependency between libzfs and libzfs_core
libzfs_core is the wrapper around kernel ioctls, the ioctl compat code belongs
to it
2015-06-15 10:48:48 +00:00
Simon J. Gerraty
ccfb965433 Add META_MODE support.
Off by default, build behaves normally.
WITH_META_MODE we get auto objdir creation, the ability to
start build from anywhere in the tree.

Still need to add real targets under targets/ to build packages.

Differential Revision:       D2796
Reviewed by: brooks imp
2015-06-13 19:20:56 +00:00
Andriy Gapon
a565264d3c zfs clone should not mount the clone if canmount == noauto
Creation of a new filesystem does not imply an intent to mount it.

Since canmount property is not inherited and its default value is 'on',
the only scenario where this matters is zfs clone -o canmount=noauto.
zfs create -o canmount=noauto already does not mount the new filesystem.

Also see:
https://www.illumos.org/issues/5984
https://reviews.csiden.org/r/228/
dd0e0e69f5
https://github.com/zfsonlinux/zfs/issues/2241

Reviewed by:	mahrens
MFC after:	8 days
Sponsored by:	ClusterHQ
2015-06-12 11:21:35 +00:00
Andriy Gapon
01628dbd09 MFV r284042: 1778 Assertion failed: rn->rn_nozpool == B_FALSE, file
../common/libzfs_import.c, line 1077, function zpool_open_func

illumos/illumos-gate@bd0f709169

Author:		Andrew Stormont <andyjstormont@gmail.com>
MFC after:	13 days
2015-06-12 11:16:43 +00:00
Andriy Gapon
ff7e06fbf4 MFV r284030: 5818 zfs {ref}compressratio is incorrect with 4k sector size
illumos/illumos-gate@81cd5c555f

Author:	Matthew Ahrens <mahrens@delphix.com>
MFC after:	17 days
2015-06-12 10:57:05 +00:00
Andriy Gapon
076dd8eb2e several lockstat improvements
0. For spin events report time spent spinning, not a loop count.
While loop count is much easier and cheaper to obtain it is hard
to reason about the reported numbers, espcially for adaptive locks
where both spinning and sleeping can happen.
So, it's better to compare apples and apples.

1. Teach lockstat about FreeBSD rw locks.
This is done in part by changing the corresponding probes
and in part by changing what probes lockstat should expect.

2. Teach lockstat that rw locks are adaptive and can spin on FreeBSD.

3. Report lock acquisition events for successful rw try-lock operations.

4. Teach lockstat about FreeBSD sx locks.
Reporting of events for those locks completely mirrors
rw locks.

5. Report spin and block events before acquisition event.
This is behavior documented for the upstream, so it makes sense to stick
to it.  Note that because of FreeBSD adaptive lock implementations
both the spin and block events may be reported for the same acquisition
while the upstream reports only one of them.

Differential Revision:	https://reviews.freebsd.org/D2727
Reviewed by:	markj
MFC after:	17 days
Relnotes:	yes
Sponsored by:	ClusterHQ
2015-06-12 10:01:24 +00:00
Marcelo Araujo
217ff3d044 At revision r247852 accidentally was removed from print the variable obj.
While here, declare some global variables as static to silence clang
warnings.

Differential Revision:	D2722
Reviewed by:		delphij, mahrens
2015-06-12 02:16:14 +00:00
Simon J. Gerraty
44d314f704 dirdeps.mk now sets DEP_RELDIR 2015-06-08 23:35:17 +00:00
Mark Johnston
8436cb81cd libdtrace: allow D libraries to declare dependencies on kernel modules
The "depends_on module" pragma can be used to declare a dependency on a
DTrace module, which for kernel probes corresponds to a KLD. Such
dependencies cannot be checked if the KLD is compiled into the kernel.
Therefore, allow a module dependency to be satisfied if either a kernel
module or a KLD with the specified name is loaded.

Differential Revision:	https://reviews.freebsd.org/D2653
Reviewed by:	gnn, rpaulo
Reported by:	gnn
2015-06-06 16:47:45 +00:00
Simon J. Gerraty
98e0ffaefb Merge sync of head 2015-05-27 01:19:58 +00:00
Mark Johnston
9b055b7404 lockstat(1): document the -V option.
MFC after:	3 days
2015-05-25 01:18:46 +00:00