Commit Graph

763 Commits

Author SHA1 Message Date
Bryan Drewery
60648601bf Reduce overlinking of libdtrace consumers with libctf, libelf, libproc.
The proper place for this list is _DP_dtrace.

Due to removing the LDADD_dtrace, more LIBADD are needed in
cddl/usr.sbin/dtrace to prevent underlinking.

This fixes overlinking in cddl/usr.sbin/lockstat and
cddl/usr.sbin/plockstat.

Sponsored by:	EMC / Isilon Storage Division
2015-12-02 00:49:49 +00:00
Bryan Drewery
a9dfc956d6 Fully connect cddl/usr.sbin/plockstat.
There seems to be no reason to keep this so private.

Also add missing optional MK_CDDL files.

Sponsored by:	EMC / Isilon Storage Division
2015-12-02 00:49:45 +00:00
Bryan Drewery
b1f92fa229 META MODE: Update dependencies with 'the-lot' and add missing directories.
This is not properly respecting WITHOUT or ARCH dependencies in target/.
Doing so requires a massive effort to rework targets/ to do so.  A
better approach will be to either include the SUBDIR Makefiles directly
and map to DIRDEPS or just dynamically lookup the SUBDIR.  These lose
the benefit of having a userland/lib, userland/libexec, etc, though and
results in a massive package.  The current implementation of targets/ is
very unmaintainable.

Currently rescue/rescue and sys/modules are still not connected.

Sponsored by:	EMC / Isilon Storage Division
2015-12-01 05:23:19 +00:00
Bryan Drewery
a0eb270b12 META MODE: Fix userland/cddl.
MK_CTF controls whether ctfmerge is ran against the binaries, not
whether CTF tools should be built.

Sponsored by:	EMC / Isilon Storage Division
2015-12-01 05:22:10 +00:00
Bryan Drewery
b16fc49e73 Rename Makefile.inc1 to dtrace.test.mk to avoid clash with top-level Makefile.inc1.
Discussed with:	ngie, markj
Sponsored by:	EMC / Isilon Storage Division
2015-11-25 20:43:45 +00:00
Bryan Drewery
7b3ea376a2 META MODE: Prefer INSTALL=tools/install.sh to lessen the need for xinstall.host.
This both avoids some dependencies on xinstall.host and allows
bootstrapping on older releases to work due to lack of at least 'install -l'
support.

Sponsored by:	EMC / Isilon Storage Division
2015-11-25 19:10:28 +00:00
Steven Hartland
7386fd9110 Fix zfs(8) set options
Fix zfs(8) not formatting due to wrong macro (Oc) in the syntax for the new
zfs set multiple dataset properties option.

PR:		204631
Submitted by:	Thomas Eberhardt
Sponsored by:	Multiplay
2015-11-19 12:03:11 +00:00
Enji Cooper
7c49f4218a Reduce the Makefile snippet complexity a bit
- Set BINDIR to TESTSDIR globally (and subsequently, remove all
  `${FILESGROUP}DIR` setting because BINDIR is set to `TESTSDIR`)
- Set MAN to "" globally, instead of per-PROG

MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division
2015-11-09 02:29:38 +00:00
Bryan Drewery
ca8d93fc70 Remove unneeded _RECURSING_PROGS check.
It is definitely not needed after r288158, and is a private variable as well
that should not be checked here.

MFC after:	1 week
Sponsored by:	EMC / Isilon Storage Division
2015-10-30 21:23:14 +00:00
Allan Jude
cd684494e5 Allow 'zfs holds -r' to recurse over a file system or volume to find holds
Previously, the parameters of 'zfs holds' could only be snapshots

Add -d <depth> flag to limit depth of recursion
Add -p flag to print literal values, rather than interpreted values
Add -H flag to suppress header output and use tabs rather than whitespace

Reviewed by:	mahrens, smh, dteske
Approved by:	bapt (mentor)
MFC after:	3 weeks
Relnotes:	yes
Sponsored by:	ScaleEngine Inc.
Differential Revision:	https://reviews.freebsd.org/D3994
2015-10-26 17:07:22 +00:00
Mark Johnston
a4dc509f72 DWARF emitted by clang 3.7 encodes array sizes using the DW_AT_count
attribute rather than DW_AT_upper_bound. Teach ctfconvert about this so that
array type sizes are encoded correctly.

PR:		203772
MFC after:	1 week
2015-10-24 03:14:36 +00:00
Alexander Motin
6b513e2853 MFV r289561: 6328 Fix cstyle errors in zfs codebase
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Alex Reece <alex@delphix.com>
Reviewed by: Richard Elling <Richard.Elling@RichardElling.com>
Reviewed by: Jorgen Lundman <lundman@lundman.net>
Approved by: Robert Mustacchi <rm@joyent.com>
Author: Paul Dagnelie <pcd@delphix.com>

illumos/illumos-gate@9a686fbc18
2015-10-19 08:25:37 +00:00
Alexander Motin
ebe70171e0 MFV r289535: 5767 fix several problems with zfs test suite
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Christopher Siden <christopher.siden@delphix.com>
Approved by: Gordon Ross <gwr@nexenta.com>
Author: John Wren Kennedy <john.kennedy@delphix.com>

illumos/illumos-gate@52244c0958

In fact, only unrelated part of that commit is applicable:
8. zpool list -v doesn't print spares
It also doesn't correctly identify log devices.
2015-10-18 19:05:56 +00:00
Alexander Motin
7076459d74 MFV r289530: 5847 libzfs_diff should check zfs_prop_get() return
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Albert Lee <trisk@omniti.com>
Approved by: Dan McDonald <danmcd@omniti.com>
Author: Alexander Eremin <a.eremin@nexenta.com>

illumos/illumos-gate@8430278980
2015-10-18 18:32:22 +00:00
Alexander Motin
d82754bbbc Reduce diff from upstream.
Should be no functional change.
2015-10-18 18:25:00 +00:00
Alexander Motin
62ed65eb78 MFV r289526:
5561 support root pools on EFI/GPT partitioned disks
5125 update zpool/libzfs to manage bootable whole disk pools (EFI/GPT labeled disks)

Reviewed by: Jean McCormack <jean.mccormack@nexenta.com>
Reviewed by: Josef 'Jeff' Sipek <josef.sipek@nexenta.com>
Approved by: Dan McDonald <danmcd@omniti.com>
Author: Hans Rosenfeld <hans.rosenfeld@nexenta.com>

illumos/illumos-gate@1a902ef862

This is NOP changes for FreeBSD.
2015-10-18 18:08:33 +00:00
Alexander Motin
c0a598da45 MFC r289498: 6298 zfs_create_008_neg and zpool_create_023_neg need to be
updated for large block support.

Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: John Kennedy <john.kennedy@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author: Joe Stein <joe.stein@delphix.com>

illumos/illumos-gate@e9316f7696
2015-10-18 11:44:31 +00:00
Alexander Motin
9a975c8c7b MFV r247180: Update vendor/illumos/dist and vendor-sys/illumos/dist
to illumos-gate 13967:92bec6d87f59

Illumos ZFS issues:
  3557 dumpvp_size is not updated correctly when a dump zvol's size is
       changed
  3558 setting the volsize on a dump device does not return back ENOSPC
  3559 setting a volsize larger than the space available sometimes succeeds
2015-10-18 11:36:43 +00:00
Alexander Motin
05b1dc3f1f MFV r289493: 5745 zfs set allows only one dataset property to be set at a time
Reviewed by: Christopher Siden <christopher.siden@delphix.com>
Reviewed by: George Wilson <george@delphix.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Bayard Bell <buffer.g.overflow@gmail.com>
Reviewed by: Richard PALO <richard@NetBSD.org>
Reviewed by: Steven Hartland <killing@multiplay.co.uk>
Approved by: Rich Lowe <richlowe@richlowe.net>
Author: Chris Williamson <chris.williamson@delphix.com>

illumos/illumos-gate@30925561c2
2015-10-18 11:21:08 +00:00
Alexander Motin
43f774f296 MFV r289310:
4185 add new cryptographic checksums to ZFS: SHA-512, Skein, Edon-R

Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Prakash Surya <prakash.surya@delphix.com>
Reviewed by: Saso Kiselkov <saso.kiselkov@nexenta.com>
Reviewed by: Richard Lowe <richlowe@richlowe.net>
Approved by: Garrett D'Amore <garrett@damore.org>
Author: Matthew Ahrens <mahrens@delphix.com>

illumos/illumos-gate@45818ee124

This is only a partial merge of respective ZFS infrastructure changes.
At this moment FreeBSD kernel has no those crypto algorithms, so the
parts of the code to enable them are commented out.  When they are
implemented, it will be trivial to plug them in.
2015-10-16 14:45:21 +00:00
Bryan Drewery
aa92269e46 Add more SUBDIR_PARALLEL.
MFC after:	3 weeks
Sponsored by:	EMC / Isilon Storage Division
2015-10-15 22:55:08 +00:00
Alexander Motin
c70e61feed MFV r289312: 2605 want to resume interrupted zfs send
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Paul Dagnelie <pcd@delphix.com>
Reviewed by: Richard Elling <Richard.Elling@RichardElling.com>
Reviewed by: Xin Li <delphij@freebsd.org>
Reviewed by: Arne Jansen <sensille@gmx.net>
Approved by: Dan McDonald <danmcd@omniti.com>
Author: Matthew Ahrens <mahrens@delphix.com>

illumos/illumos-gate@9c3fd1216f

For more info, see:
 - slides http://www.slideshare.net/MatthewAhrens/openzfs-send-and-receive
 - video https://www.youtube.com/watch?v=iY44jPMvxog
 - manpage changes (for zfs resume -s and zfs send -t)
 - upcoming talk at the OpenZFS Developer Summit

The TL;DR is:
Use "zfs receive -s" to save the partially received state on failure.
On failure, get the receive token with "zfs get receive_resume_token <fs>"
Resume the send with "zfs send -t <token_value>"

Relnotes:	yes
2015-10-15 08:47:32 +00:00
Alexander Motin
2bfd7091a5 MFV r289311: 5764 "zfs send -nv" directs output to stderr
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Paul Dagnelie <paul.dagnelie@delphix.com>
Reviewed by: Basil Crow <basil.crow@delphix.com>
Reviewed by: Steven Hartland <killing@multiplay.co.uk>
Reviewed by: Bayard Bell <buffer.g.overflow@gmail.com>
Approved by: Dan McDonald <danmcd@omniti.com>
Author: Manoj Joseph <manoj.joseph@delphix.com>

illumos/illumos-gate@dc5f28a3c3
2015-10-14 11:52:58 +00:00
Enji Cooper
b2d48be1bc Refactor the test/ Makefiles after recent changes to bsd.test.mk (r289158) and
netbsd-tests.test.mk (r289151)

- Eliminate explicit OBJTOP/SRCTOP setting
- Convert all ad hoc NetBSD test integration over to netbsd-tests.test.mk
- Remove unnecessary TESTSDIR setting
- Use SRCTOP where possible for clarity

MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Divison
2015-10-12 08:16:03 +00:00
Mark Johnston
3142b37664 Have lockstat(1) trace locks by name rather than by address.
Previously, lockstat(1) would use a lock's address as its identifier when
consuming data describing lock contention and hold events. After collecting
the requested data, it would use ksyms(4) to resolve lock addresses to
names. Of course, this doesn't work too well for locks contained in
dynamically-allocated memory. This change modifies lockstat(1) to trace the
lock names obtained from the base struct lock_object instead, leading to
output that is generally much more useful.

This change also removes the -c option, which is used to coalesce data for
locks in an array. It's not possible to support this option without also
tracing lock addresses, and since lock arrays in which the lock names are
distinct are not very common in FreeBSD, it's simpler to just remove the
option.

Reviewed by:	avg (earlier revision)
Differential Revision:	https://reviews.freebsd.org/D3661
2015-09-30 05:46:56 +00:00
Mark Johnston
b22ef02080 Update DTrace test makefiles after r288415. 2015-09-30 05:25:39 +00:00
Mark Johnston
f7c3db2537 MFV r288408:
6266 harden dtrace_difo_chunksize() with respect to malicious DIF

illumos/illumos-gate@395c7a3dcf

Reviewed by: Alex Wilson <alex.wilson@joyent.com>
Reviewed by: Dan McDonald <danmcd@omniti.com>
Approved by: Garrett D'Amore <garrett@damore.org>
Author: Bryan Cantrill <bryan@joyent.com>

MFC after:	1 week
2015-09-30 05:24:22 +00:00
Mark Johnston
c5af5ada34 Stop hard-coding a 32-bit data model for USDT tests, and just use the native
model. This was causing many of the tests to fail on amd64 since USDT
support for 32-bit programs is currently non-functional.

MFC after:	1 week
2015-09-30 04:48:56 +00:00
Andriy Gapon
9b977fcea2 define aok in libnvpair which is linked to all zfs libraries that need aok
This removes the circular dependency of libnvpair on libzfs / libzpool.

PR:		199811
Obtained from:	bapt
MFC after:	23 days
2015-09-28 15:25:36 +00:00
Andriy Gapon
9875367b9a remove unused sgsmsg utility (originally imported from opensolaris)
MFC after:	25 days
2015-09-28 12:38:57 +00:00
Andriy Gapon
65b71c48d2 remove an extra copy of avl.c from illumos contrib code
MFC after:	20 days
2015-09-28 12:23:10 +00:00
Bryan Drewery
54c7d75a06 META_MODE: Remove DEP_RELDIR from Makefile.depend files.
This has not been needed since r284171 in projects/bmake.

Sponsored by:	EMC / Isilon Storage Division
2015-09-25 19:26:08 +00:00
Bryan Drewery
144cd65bca Use FILES for installing the dtrace scripts.
The BSD.usr.dist mtree always creates /usr/lib/dtrace so there is no
need to check if it exists.

The FILES mechanism already supports LIBRARIES_ONLY.

Sponsored by:	EMC / Isilon Storage Division
2015-09-18 21:18:44 +00:00
Mark Johnston
c50c331896 Remove an unneeded typedef of ip6_t from the DTrace ip provider library.
It causes an error when ipfilter is enabled, since ipl.ko contains an
identical typedef.

PR:		203092
MFC after:	1 week
2015-09-15 05:16:26 +00:00
Xin LI
cfb9fa4ec9 MFV r286224: 5695 dmu_sync'ed holes do not retain birth time
(userland portion that was not merged in r286677)

Update zdb to also print ltime, type, and level information
for these new style holes. Previously, only the logical birth
time would be printed.
2015-09-14 06:10:49 +00:00
Xin LI
44ec48bc55 MFV r277429:
Document -S option when zfs inherit fails on quota and
in manual pages.

Illumos ZFS issues:

    5410 Document -S option to zfs inherit
    https://illumos.org/issues/5410

    5412 Mention -S option when zfs inherit fails on quota
    https://illumos.org/issues/5412

illumos/illumos-gate@5ff8cfa92e

MFC after:	2 weeks
2015-09-14 06:00:50 +00:00
George V. Neville-Neil
5d06879adb dd DTrace probe points, translators and a corresponding script
to provide the TCPDEBUG functionality with pure DTrace.

Reviewed by:	rwatson
MFC after:	2 weeks
Sponsored by:	Limelight Networks
Differential Revision:	D3530
2015-09-13 15:50:55 +00:00
Xin LI
530ecabd3d Fix build (r287703). Lesson learned: no matter how a change looks like an
innocent one, always do a build test first.

Pointy hat to:	delphij
2015-09-12 09:28:02 +00:00
Allan Jude
73457dc357 Remove duplicate defines introduced in initial ZFS import (r168404)
This change reduces compiler warnings by removing duplicate defines

Line numbers are from r168404 (and r284648)
#define lbolt: lines 384 and 459 (531 and 648) (original was renamed later)
#define lbolt64: lines 385 and 460 (532 and 649) (original was renamed later)
#define gethrestime_sec: lines 390 and 465 (540 and 653)
uint64_t physmem: lines 402 and 463 (561 and 651)

Reviewed by:	smh, delphij
Approved by:	bapt (mentor)
Sponsored by:	ScaleEngine Inc.
Differential Revision:	https://reviews.freebsd.org/D2878
2015-08-31 22:36:17 +00:00
Mark Johnston
bd81e07d27 Re-apply r274569. It was reverted in r276848 since that appeared to fix
some ctfmerge crashes that started to occur on i386 weeks after r274569 was
committed. Some later investigation indicated that the crashes were caused
by malformed CTF info that led to a stack overflow. The issue with CTF
info in i386 kernels seems to have been resolved by r261246, which updated
libdwarf and libelf.

r274569 fixes a bug which caused duplicate types to appear in the kernel's
CTF info. This duplication generally does not cause problems when using
DTrace, but makes it easier to hit the limit of 2^15 - 1 distinct type
definitions in a CTF container.

MFC after:	2 weeks
2015-08-28 00:38:18 +00:00
Warner Losh
c404eb64d6 Sparc64 is the odd-man out, so form the if that way rather than
listing everybody else.
2015-08-23 23:12:30 +00:00
Xin LI
fb90888521 Plug a memory leak.
MFC after:	2 weeks
2015-08-13 18:45:52 +00:00
Alexander Motin
0d0def87fe MFV 286707: 5959 clean up per-dataset feature count code
Reviewed by: Toomas Soome <tsoome@me.com>
Reviewed by: George Wilson <george@delphix.com>
Reviewed by: Alex Reece <alex@delphix.com>
Approved by: Richard Lowe <richlowe@richlowe.net>
Author: Matthew Ahrens <mahrens@delphix.com>

illumos/illumos-gate@ca0cc3918a

A ZFS feature flags (large blocks) tracks its refcounts as the number of
datasets that have ever used the feature. Several features of this type
are planned to be added (new checksum functions). This code should be made
common infrastructure rather than duplicating the code for each feature.
2015-08-12 23:59:17 +00:00
Alexander Motin
b696497df0 MFV r286704: 5960 zfs recv should prefetch indirect blocks
5925 zfs receive -o origin=

Reviewed by: Prakash Surya <prakash.surya@delphix.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Author: Paul Dagnelie <pcd@delphix.com>

While running 'zfs recv' we noticed that every 128th 8K block required a
read. We were seeing that restore_write() was calling dmu_tx_hold_write()
and the indirect block was not cached. We should prefetch upcoming indirect
blocks to avoid having to go to disk and blocking the restore_write().

Allow an incremental send stream to be received as a clone, even if the
stream does not mark it as a clone.
2015-08-12 22:41:06 +00:00
Alexander Motin
ebf527de10 MFV r286682: 5765 add support for estimating send stream size with
lzc_send_space when source is a bookmark

Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Christopher Siden <christopher.siden@delphix.com>
Reviewed by: Steven Hartland <killing@multiplay.co.uk>
Reviewed by: Bayard Bell <buffer.g.overflow@gmail.com>
Approved by: Albert Lee <trisk@nexenta.com>
Author: Max Grossman <max.grossman@delphix.com>

illumos/illumos-gate@643da460c8
2015-08-12 18:23:08 +00:00
Alexander Motin
799f47828d MFV 286602: 5810 zdb should print details of bpobj
Reviewed by: Prakash Surya <prakash.surya@delphix.com>
Reviewed by: Alex Reece <alex@delphix.com>
Reviewed by: George Wilson <george@delphix.com>
Reviewed by: Will Andrews <will@freebsd.org>
Reviewed by: Simon Klinkert <simon.klinkert@gmail.com>
Approved by: Gordon Ross <gwr@nexenta.com>
Author: Matthew Ahrens <mahrens@delphix.com>

illumos/illumos-gate@732885fca0
2015-08-10 21:32:40 +00:00
Alexander Motin
57f7c5acf5 MFV 286586: 5746 more checksumming in zfs send
Reviewed by: Christopher Siden <christopher.siden@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Bayard Bell <buffer.g.overflow@gmail.com>
Approved by: Albert Lee <trisk@omniti.com>
Author: Matthew Ahrens <mahrens@delphix.com>

illumos/illumos-gate@98110f08fa
2015-08-10 19:32:58 +00:00
Alexander Motin
4ff9527edc MFV 286546:
5661 ZFS: "compression = on" should use lz4 if feature is enabled

Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Josef 'Jeff' Sipek <jeffpc@josefsipek.net>
Reviewed by: Xin LI <delphij@freebsd.org>
Approved by: Robert Mustacchi <rm@joyent.com>
Author: Justin T. Gibbs <justing@spectralogic.com>

illumos/illumos-gate@db1741f555
2015-08-09 20:02:16 +00:00
Mark Johnston
7e518a6648 - Use an explicit "depends_on module kernel" guard in DTrace libraries that
reference types defined in the kernel. Otherwise dtrace(1) expects to find
  CTF definitions for all referenced types, which is not very reasonable
  when it is being used in a build environment. This was previously worked
  around by adding "-x nolibs" to dtrace -h or -G invocations, but as of
  r283025, dtrace(1) actually handles dependencies properly, so this is no
  longer necessary.
- Remove "pragma ident" directives from DTrace libraries, as they're being
  phased out upstream as well.

Submitted by:	Krister Johansen <Krister.Johansen@isilon.com> [1]
MFC after:	1 week
Sponsored by:	EMC / Isilon Storage Division
2015-08-07 19:56:22 +00:00
Mark Johnston
d912066c36 Add a src.conf option to build and install the DTrace test suite.
Reviewed by:	gnn, ngie
Differential Revision:	https://reviews.freebsd.org/D3195
2015-08-02 00:37:33 +00:00