Commit Graph

1045 Commits

Author SHA1 Message Date
Andriy Gapon
8b30f189d1 MFV r322217: 8418 zfs_prop_get_table() call in zfs_validate_name() is a no-op
illumos/illumos-gate@e09ba01dcd
e09ba01dcd

https://www.illumos.org/issues/8418
  The following line in zfs_validate_name() is just a no-op and it
  should be removed:
      108    (void) zfs_prop_get_table();

Reviewed by: Vitaliy Gusev <gusev.vitaliy@icloud.com>
Approved by: Matthew Ahrens <mahrens@delphix.com>
Author: Marcel Telka <marcel@telka.sk>

MFC after:	2 weeks
2017-08-08 10:30:49 +00:00
Ruslan Bukin
ca20f8ec29 o Replace __riscv__ with __riscv
o Replace __riscv64 with (__riscv && __riscv_xlen == 64)

This is required to support new GCC 7.1 compiler.
This is compatible with current GCC 6.1 compiler.

RISC-V is extensible ISA and the idea here is to have built-in define
per each extension, so together with __riscv we will have some subset
of these as well (depending on -march string passed to compiler):

__riscv_compressed
__riscv_atomic
__riscv_mul
__riscv_div
__riscv_muldiv
__riscv_fdiv
__riscv_fsqrt
__riscv_float_abi_soft
__riscv_float_abi_single
__riscv_float_abi_double
__riscv_cmodel_medlow
__riscv_cmodel_medany
__riscv_cmodel_pic
__riscv_xlen

Reviewed by:	ngie
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D11901
2017-08-07 14:09:57 +00:00
Mark Johnston
22e406c80b Rework and simplify the ksyms(4) implementation.
- Store the symbol table contents in an anonymous swap-backed object. Have
  mmap(/dev/ksyms) map that object, and stop mapping the symbol table into
  the calling process in ksyms_open(). Previously we would cache a pointer
  to the pmap of the opening process, and mmap(/dev/ksyms) would create a
  mapping using the physical address found by a pmap lookup at the initial
  mapping address. However, this assumes that the cached pmap is valid,
  which may not be the case. [1]
- Remove the ksyms ioctl interface. It appears to have been added to work
  around a limitation in libelf that no longer exists; see r321842.
  Moreover, the interface is difficult to support and isn't present in
  illumos. Since ksyms was added specifically to support lockstat(1), it
  is expected that this removal won't have any real impact.
- Simplify ksyms_read() to avoid unnecessary copying.
- Don't call the device handle destructor if we fail to capture a snapshot
  of the kernel's symbol table. devfs will do that for us.

Reported by:	Ilja van Sprundel <ivansprundel@ioactive.com> [1]
Reviewed by:	kib (previous revision)
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D11789
2017-08-03 00:38:13 +00:00
Alexander Motin
f2d3f6918e Add compat shim part missed at r305197.
This fixes compatibility between old kernel and new ZFS tools.
It seems to be tradition to forget it. :(

PR:		221112
MFC after:	3 days
2017-08-02 10:33:47 +00:00
Enji Cooper
4b330699f8 Convert traditional ${MK_TESTS} conditional idiom for including test
directories to SUBDIR.${MK_TESTS} idiom

This is being done to pave the way for future work (and homogenity) in
^/projects/make-check-sandbox .

No functional change intended.

MFC after:	1 weeks
2017-08-02 08:35:51 +00:00
Mark Johnston
3c9308e82b Remove local variables missed in r321842.
X-MFC with:	r321842
2017-08-01 04:52:03 +00:00
Mark Johnston
90ec6fd4ba Let lockstat use ksyms(4)'s mmap interface.
The workaround described in the deleted comment is no longer needed.

MFC after:	1 week
2017-08-01 04:49:54 +00:00
Li-Wen Hsu
ad89d783a6 Add an auxiliary subroutine to generate some events for testing
This test is also timeout on a quiet system because there is nobody triggering
read probefunc while test execution.

Reviewed by:	gnn, markj, ngie
Differential Revision:	https://reviews.freebsd.org/D11731
2017-07-26 12:07:46 +00:00
Li-Wen Hsu
288ebd813a The test case common.funcs.t_dtrace_contrib.tst_basename_d generates a
verifying script which needs being run to complete the test.

While here, add missing shebang.

Reviewed by:	gnn, markj, ngie
Differential Revision:	https://reviews.freebsd.org/D11716
2017-07-25 13:18:28 +00:00
Li-Wen Hsu
637ba06516 Modify glob patterns and expected output to match FreeBSD's implementation.
Reviewed by:	gnn, markj, ngie
Differential Revision:	https://reviews.freebsd.org/D11713
2017-07-25 13:14:02 +00:00
Li-Wen Hsu
5604d0f997 Make this test case accepts basename() in D script returns "" or "."
In Solaris, basename(1) and basename(3) both return "." while being given an
empty string (""), while in BSD (and Linux) basename(1) returns "" and
basename(3) returns "."

While here, also change #!/usr/bin/ksh to #!/usr/bin/env ksh to find ksh in
$PATH

Reviewed by:	gnn, markj (earlier version), ngie (earlier version)
Differential Revision:	https://reviews.freebsd.org/D11707
2017-07-25 13:11:20 +00:00
Li-Wen Hsu
4ca0dfa6b0 Explicitly set dynamic variable buffer size.
We added too many variable assignments in BEGIN block, which will run out of
default auto-configured variable buffer space.  The test VM has 4G RAM which
should be enough for most cases so it's reasonable to increase limitation to
these case.

Reviewed by:	gnn
Differential Revision:	https://reviews.freebsd.org/D11676
2017-07-25 13:07:06 +00:00
Li-Wen Hsu
23833df483 Explicitly set dynamic variable buffer size.
We added too many variable assignments in BEGIN block, which will run out of
default auto-configured variable buffer space.  The test VM has 4G RAM which
should be enough for most cases so it's reasonable to increase limitation to
these case.

Reviewed by:	gnn, markj, ngie
Differential Revision:	https://reviews.freebsd.org/D11674
2017-07-25 13:04:24 +00:00
Li-Wen Hsu
070a148127 Add an auxiliary subroutine to generate read(2) event while testing.
Reviewed by:	gnn, ngie
Differential Revision:	https://reviews.freebsd.org/D11673
2017-07-25 13:01:10 +00:00
Li-Wen Hsu
d83c70758a Add a simple script which calls open(2) and others to generate events for
testing.

This test times-out on a quiet system because there is nobody triggers
syscall::open:entry or syscall::: probe while test execution.

Reviewed by:	gnn, markj (earlier version)
Differential Revision:	https://reviews.freebsd.org/D11671
2017-07-25 12:58:03 +00:00
Li-Wen Hsu
b9de3393dd Add a simple program which calls sigtimedwait(2) to generate events for testing
This test timeout on a quiet system because there is nobody triggers
'syscall::*wait*:entry' probe while test execution.

Reviewed by:	gnn, markj, ngie
Differential Revision:	https://reviews.freebsd.org/D11668
2017-07-25 12:52:32 +00:00
Enji Cooper
31ed01a2de Fix whitespace on a line in fix(..) accidentally missed in r321424
MFC after:	1 month
MFC with:	r321424
2017-07-24 17:29:56 +00:00
Enji Cooper
f3305cae02 Style cleanup: delete spurious trailing whitespace
MFC after:	1 month
2017-07-24 17:27:21 +00:00
Enji Cooper
aa52ad5489 Don't use incorrect hardcoded path to ksh -- use /usr/bin/env
to find ksh instead

MFC after:	1 month
2017-07-23 17:57:00 +00:00
Alan Somers
19d04786c7 zfsd(8): Remove pidfile on shutdown
MFC after:	3 weeks
Sponsored by:	Spectra Logic Corp
2017-06-20 19:45:02 +00:00
Andriy Gapon
f9cdbaba8d MFV r318946: 8021 ARC buf data scatter-ization
illumos/illumos-gate@770499e185
770499e185

https://www.illumos.org/issues/8021
  The ARC buf data project (known simply as "ABD" since its genesis in the ZoL
  community) changes the way the ARC allocates `b_pdata` memory from using linear
  `void *` buffers to using scatter/gather lists of fixed-size 1KB chunks. This
  improves ZFS's performance by helping to defragment the address space occupied
  by the ARC, in particular for cases where compressed ARC is enabled. It could
  also ease future work to allocate pages directly from `segkpm` for minimal-
  overhead memory allocations, bypassing the `kmem` subsystem.
  This is essentially the same change as the one which recently landed in ZFS on
  Linux, although they made some platform-specific changes while adapting this
  work to their codebase:
  1. Implemented the equivalent of the `segkpm` suggestion for future work
  mentioned above to bypass issues that they've had with the Linux kernel memory
  allocator.
  2. Changed the internal representation of the ABD's scatter/gather list so it
  could be used to pass I/O directly into Linux block device drivers. (This
  feature is not available in the illumos block device interface yet.)

FreeBSD notes:
- the actual (default) chunk size is 4KB (despite the text above saying 1KB)
- we can try to reimplement ABDs, so that they are not permanently
  mapped into the KVA unless explicitly requested, especially on
  platforms with scarce KVA
- we can try to use unmapped I/O and avoid intermediate allocation of a
  linear, virtual memory mapped buffer
- we can try to avoid extra data copying by referring to chunks / pages
  in the original ABD

Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Paul Dagnelie <pcd@delphix.com>
Reviewed by: John Kennedy <john.kennedy@delphix.com>
Reviewed by: Prakash Surya <prakash.surya@delphix.com>
Reviewed by: Prashanth Sreenivasa <pks@delphix.com>
Reviewed by: Pavel Zakharov <pavel.zakharov@delphix.com>
Reviewed by: Chris Williamson <chris.williamson@delphix.com>
Approved by: Richard Lowe <richlowe@richlowe.net>
Author: Dan Kimmel <dan.kimmel@delphix.com>

MFC after:	3 weeks
2017-06-20 17:39:24 +00:00
Bryan Drewery
c99b67a794 Utilize SYSROOT from r320119 in places where DESTDIR may be wanting WORLDTMP.
Since buildenv exports SYSROOT all of these uses will now look in
WORLDTMP by default.

sys/boot/efi/loader/Makefile
        A LIBSTAND hack is no longer required for buildenv.

MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-06-19 20:47:24 +00:00
Andriy Gapon
b8d341fe26 MFV r319945,r319946: 8264 want support for promoting datasets in libzfs_core
illumos/illumos-gate@a4b8c9aa65
a4b8c9aa65

https://www.illumos.org/issues/8264
  Oddly there is a lzc_clone function, but no lzc_promote function.

Reviewed by: Andriy Gapon <avg@FreeBSD.org>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Dan McDonald <danmcd@kebe.com>
Approved by: Dan McDonald <danmcd@kebe.com>
Author: Andrew Stormont <astormont@racktopsystems.com>
MFC after:	1 week
2017-06-14 16:31:36 +00:00
Mark Johnston
c645060dbb Override the locale so that file lists get a consistent sort order.
Reported by:	avg
MFC after:	1 week
2017-06-10 14:47:01 +00:00
Andriy Gapon
6bd205a529 follow up to r319746: add the new test files to the make file
Reported by:	markj
MFC after:	2 days
X-MFC with:	r319746
2017-06-10 06:13:52 +00:00
Andriy Gapon
9260925dcd MFV r319740: 8168 NULL pointer dereference in zfs_create()
illumos/illumos-gate@690031d326
690031d326

https://www.illumos.org/issues/8168
  If we manage to export the pool on which we are creating a dataset (filesystem
  or zvol) between entering libzfs`zfs_create() and libzfs`zpool_open() call (for
  which we never check the return value) we end up dereferencing a NULL pointer
  in libzfs`zpool_close().
  This was discovered on ZFS on Linux. The same issue can be reproduced on
  Illumos running in parallel:
    while :; do zpool import -d /tmp testpool ; zpool export testpool ; done
    while :; do zfs create testpool/fs; zfs destroy testpool/fs ; done
  Eventually this will result in several core dumps like this one:
  [root@52-54-00-d3-7a-01 /cores]# mdb core.zfs.4244
  Loading modules: [ libumem.so.1 libc.so.1 libtopo.so.1 libavl.so.1
  libnvpair.so.1 ld.so.1 ]
  > ::stack
  libzfs.so.1`zpool_close+0x17(0, 0, 0, 8047450)
  libzfs.so.1`zfs_create+0x1bb(8090548, 8047e6f, 1, 808cba8)
  zfs_do_create+0x545(2, 8047d74, 80778a0, 801, 0, 3)
  main+0x22c(8047d2c, fef5c6e8, 8047d64, 8055a17, 3, 8047d70)
  _start+0x83(3, 8047e64, 8047e68, 8047e6f, 0, 8047e7b)
  >
  Fix and reproducer (systemtap): https://github.com/zfsonlinux/zfs/pull/6096

Reviewed by: Matt Ahrens <mahrens@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author: loli10K <ezomori.nozomu@gmail.com>
MFC after:	2 weeks
2017-06-09 15:30:41 +00:00
Andriy Gapon
ad2b1a296f MFV r319744,r319745: 8269 dtrace stddev aggregation is normalized incorrectly
illumos/illumos-gate@79809f9cf4
79809f9cf4

https://www.illumos.org/issues/8269
  It seems that currently normalization of stddev aggregation is done
  incorrectly.
  We divide both the sum of values and the sum of their squares by the
  normalization factor. But we should divide the sum of squares by the
  normalization factor squared to scale the original values properly.

FreeBSD note: the actual change was committed in r316853, this commit
adds the test files and record merge information.

Reviewed by: Bryan Cantrill <bryan@joyent.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author: Andriy Gapon <avg@FreeBSD.org>
MFC after:	1 week
Sponsored by:	Panzura
2017-06-09 15:16:39 +00:00
Allan Jude
39b0b876dc New sentences start on new lines, fix two violations
Reviewed by:	bcr
Sponsored by:	BSDCan Dev Summit
2017-06-08 01:39:17 +00:00
Allan Jude
dc379eca14 SHA-512 and Skein have been supported by the boot loader for some time.
Submitted by:	lifanov
Reviewed by:	bcr
Sponsored by:	BSDCan Dev Summit
2017-06-08 01:29:24 +00:00
Andriy Gapon
457711b02b MFV r316922: 5380 receive of a send -p stream doesn't need to try renaming snapshots
illumos/illumos-gate@471a88e499
471a88e499

https://www.illumos.org/issues/5380
  A stream created with zfs send -p -I contains properties of all snapshots of a
  given dataset as opposed to only properties of snapshots in a given range.
  Not only this is suboptimal but the receive code also does not filter
  properties by the range. So, properties of earlier snapshots would be updated
  even though the snapshots themselves are not in the stream (just their
  properties).
  Given that modifying the snapshot properties requires a TXG sync and that the
  snapshots are updated one by one the described behavior may lead to a sever
  performance penalty.

Reviewed by: Paul Dagnelie <pcd@delphix.com>
Reviewed by: Matt Ahrens <mahrens@delphix.com>
Approved by: Dan McDonald <danmcd@omniti.com>
Author: Andriy Gapon <avg@FreeBSD.org>
MFC after:	3 weeks
2017-05-24 22:30:21 +00:00
Andriy Gapon
b6be31c7ca MFC r316908: 7541 zpool import/tryimport ioctl returns ENOMEM because provided buffer is too small for config
illumos/illumos-gate@8b65a70b76
8b65a70b76

https://www.illumos.org/issues/7541
  When calling zpool import, zpool does a few ioctls to ZFS.
  zpool allocates a buffer in userland and passes it to the kernel so that ZFS
  can copy info into it. ZFS will use it to put the nvlist that describes the
  pool configuration.
  If the allocated buffer is too small, ZFS will return ENOMEM and the call will
  have to be redone. This wastes CPU time and slows down the import process. This
  happens very often for the ZFS_IOC_POOL_TRYIMPORT call.

Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Dan Kimmel <dan.kimmel@delphix.com>
Approved by: Dan McDonald <danmcd@omniti.com>
Author: Pavel Zakharov <pavel.zakharov@delphix.com>
MFC after:	2 weeks
2017-05-24 21:32:35 +00:00
Andriy Gapon
5644318970 MFC r316904: 7729 libzfs_core`lzc_rollback() leaks result nvl
illumos/illumos-gate@ac428481f9
ac428481f9

https://www.illumos.org/issues/7729
  libzfs_core`lzc_rollback() doesn't free the result nvl after lzc_ioctl() call.

Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Prakash Surya <prakash.surya@delphix.com>
Approved by: Dan McDonald <danmcd@omniti.com>
Author: Yuri Pankov <yuri.pankov@nexenta.com>

MFC after:	2 weeks
2017-05-24 20:53:01 +00:00
Andriy Gapon
c65389d367 MFV r316860: 7545 zdb should disable reference tracking
illumos/illumos-gate@4dd77f9e38
4dd77f9e38

https://www.illumos.org/issues/7545
  When evicting from the ARC, we manipulate some refcount_t's, e.g. arcs_size.
  When using zdb to examine a large amount of data (e.g. zdb -bb on a large pool
  with small blocks), the ARC may have a large number of entries. If reference
  tracking is enabled, there will be ~1 reference for each block in the ARC. When
  evicting, we decrement the refcount and have to search all the references to
  find the one that we are removing, which is very slow.
  Since zdb is typically used to find problems with the on-disk format, and not
  with the code it is running, we should disable reference tracking in zdb.

Reviewed by: Dan Kimmel <dan.kimmel@delphix.com>
Reviewed by: Steve Gonczi <steve.gonczi@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author: Matthew Ahrens <mahrens@delphix.com>

MFC after:	2 weeks
2017-05-24 20:41:26 +00:00
Konstantin Belousov
6992112349 Commit the 64-bit inode project.
Extend the ino_t, dev_t, nlink_t types to 64-bit ints.  Modify
struct dirent layout to add d_off, increase the size of d_fileno
to 64-bits, increase the size of d_namlen to 16-bits, and change
the required alignment.  Increase struct statfs f_mntfromname[] and
f_mntonname[] array length MNAMELEN to 1024.

ABI breakage is mitigated by providing compatibility using versioned
symbols, ingenious use of the existing padding in structures, and
by employing other tricks.  Unfortunately, not everything can be
fixed, especially outside the base system.  For instance, third-party
APIs which pass struct stat around are broken in backward and
forward incompatible ways.

Kinfo sysctl MIBs ABI is changed in backward-compatible way, but
there is no general mechanism to handle other sysctl MIBS which
return structures where the layout has changed. It was considered
that the breakage is either in the management interfaces, where we
usually allow ABI slip, or is not important.

Struct xvnode changed layout, no compat shims are provided.

For struct xtty, dev_t tty device member was reduced to uint32_t.
It was decided that keeping ABI compat in this case is more useful
than reporting 64-bit dev_t, for the sake of pstat.

Update note: strictly follow the instructions in UPDATING.  Build
and install the new kernel with COMPAT_FREEBSD11 option enabled,
then reboot, and only then install new world.

Credits: The 64-bit inode project, also known as ino64, started life
many years ago as a project by Gleb Kurtsou (gleb).  Kirk McKusick
(mckusick) then picked up and updated the patch, and acted as a
flag-waver.  Feedback, suggestions, and discussions were carried
by Ed Maste (emaste), John Baldwin (jhb), Jilles Tjoelker (jilles),
and Rick Macklem (rmacklem).  Kris Moore (kris) performed an initial
ports investigation followed by an exp-run by Antoine Brodin (antoine).
Essential and all-embracing testing was done by Peter Holm (pho).
The heavy lifting of coordinating all these efforts and bringing the
project to completion were done by Konstantin Belousov (kib).

Sponsored by:	The FreeBSD Foundation (emaste, kib)
Differential revision:	https://reviews.freebsd.org/D10439
2017-05-23 09:29:05 +00:00
Mark Johnston
b4a3f67bd6 Add a little helper program for tst.exitcore.ksh.
sleep(1) is capsicumized, which means that we cannot rely on it to dump
core as required by the test.

MFC after:	1 week
Sponsored by:	Dell EMC Isilon
2017-05-22 20:34:51 +00:00
Alexander Motin
c9bcbf3800 Fix time handling in cv_timedwait_hires().
pthread_cond_timedwait() receives absolute time, not relative.  Passing
wrong time there caused two threads of zdb to spin in a tight loop.

MFC after:	1 week
2017-05-19 05:12:58 +00:00
Mark Johnston
9ebaf5f802 Remove the EXFAIL annotation for tests which pass as of r309596.
Reported by:	bdrewery
Sponsored by:	Dell EMC Isilon
2017-05-19 00:25:09 +00:00
Josh Paetzel
c78abb8b50 MFV 316894
7252 7628 compressed zfs send / receive

illumos/illumos-gate@5602294fda
5602294fda

https://www.illumos.org/issues/7252
  This feature includes code to allow a system with compressed ARC enabled to
  send data in its compressed form straight out of the ARC, and receive data in
  its compressed form directly into the ARC.

https://www.illumos.org/issues/7628
  We should have longer, more readable versions of the ZFS send / recv options.

7628 create long versions of ZFS send / receive options

Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: John Kennedy <john.kennedy@delphix.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Paul Dagnelie <pcd@delphix.com>
Reviewed by: Pavel Zakharov <pavel.zakharov@delphix.com>
Reviewed by: Sebastien Roy <sebastien.roy@delphix.com>
Reviewed by: David Quigley <dpquigl@davequigley.com>
Reviewed by: Thomas Caputi <tcaputi@datto.com>
Approved by: Dan McDonald <danmcd@omniti.com>
Author: Dan Kimmel <dan.kimmel@delphix.com>
2017-04-25 17:57:43 +00:00
Josh Paetzel
ef18459108 MFV 316891
7386 zfs get does not work properly with bookmarks

illumos/illumos-gate@edb901aab9
edb901aab9

https://www.illumos.org/issues/7386
  The zfs get command does not work with the bookmark parameter while it works
  properly with both filesystem and snapshot:
  # zfs get -t all -r creation rpool/test
  NAME               PROPERTY  VALUE                  SOURCE
  rpool/test         creation  Fri Sep 16 15:00 2016  -
  rpool/test@snap    creation  Fri Sep 16 15:00 2016  -
  rpool/test#bkmark  creation  Fri Sep 16 15:00 2016  -
  # zfs get -t all -r creation rpool/test@snap
  NAME             PROPERTY  VALUE                  SOURCE
  rpool/test@snap  creation  Fri Sep 16 15:00 2016  -
  # zfs get -t all -r creation rpool/test#bkmark
  cannot open 'rpool/test#bkmark': invalid dataset name
  #
  The zfs get command should be modified to work properly with bookmarks too.

Reviewed by: Simon Klinkert <simon.klinkert@gmail.com>
Reviewed by: Paul Dagnelie <pcd@delphix.com>
Approved by: Matthew Ahrens <mahrens@delphix.com>
Author: Marcel Telka <marcel@telka.sk>
2017-04-21 19:53:52 +00:00
Alan Somers
07bb15b440 MFV 316855
7900 zdb shouldn't print the path of a znode at verbosity < 5

Reviewed by: Paul Dagnelie <pcd@delphix.com>
Reviewed by: Matt Ahrens <mahrens@delphix.com>
Approved by: Dan McDonald <danmcd@omniti.com>
Author: Alan Somers <asomers@freebsd.org>

illumos/illumos-gate@e548d2fa41
https://www.illumos.org/issues/7900

MFC after:	3 weeks
Sponsored by:	Spectra Logic Corp
2017-04-14 16:30:37 +00:00
Andriy Gapon
5ad79d9b20 dtrace: fix normalization of stddev aggregation
To be upstreamed.

Discussed with:	Bryan Cantrill <bryancantrill@gmail.com>
MFC after:	2 weeks
Sponsored by:	Panzura
2017-04-14 15:31:04 +00:00
Pedro F. Giffuni
9e98194646 MFV r316693:
8046 Let calloc() do the multiplication in libzfs_fru_refresh

5697e03e6e

https://www.illumos.org/issues/8046

Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Paul Dagnelie <pcd@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author:	Pedro Giffuni <pfg@freebsd.org>

MFC after:	3 days
2017-04-10 22:56:38 +00:00
Alexander Motin
3aef5b286a MFV r315290, r315291: 7303 dynamic metaslab selection
illumos/illumos-gate@8363e80ae7
https://github.com/illumos/illumos-gate/commit/8363e80ae72609660f6090766ca8c2c18

https://www.illumos.org/issues/7303

  This change introduces a new weighting algorithm to improve metaslab selection.
  The new weighting algorithm relies on the SPACEMAP_HISTOGRAM feature. As a result,
  the metaslab weight now encodes the type of weighting algorithm used
  (size-based vs segment-based).

  This also introduce a new allocation tracing facility and two new dcmds to help
  debug allocation problems. Each zio now contains a zio_alloc_list_t structure
  that is populated as the zio goes through the allocations stage. Here's an
  example of how to use the tracing facility:

> c5ec000::print zio_t io_alloc_list | ::walk list | ::metaslab_trace
  MSID    DVA    ASIZE      WEIGHT             RESULT               VDEV
     -      0      400           0    NOT_ALLOCATABLE           ztest.0a
     -      0      400           0    NOT_ALLOCATABLE           ztest.0a
     -      0      400           0             ENOSPC           ztest.0a
     -      0      200           0    NOT_ALLOCATABLE           ztest.0a
     -      0      200           0    NOT_ALLOCATABLE           ztest.0a
     -      0      200           0             ENOSPC           ztest.0a
     1      0      400      1 x 8M            17b1a00           ztest.0a

> 1ff2400::print zio_t io_alloc_list | ::walk list | ::metaslab_trace
  MSID    DVA    ASIZE      WEIGHT             RESULT               VDEV
     -      0      200           0    NOT_ALLOCATABLE           mirror-2
     -      0      200           0    NOT_ALLOCATABLE           mirror-0
     1      0      200      1 x 4M            112ae00           mirror-1
     -      1      200           0    NOT_ALLOCATABLE           mirror-2
     -      1      200           0    NOT_ALLOCATABLE           mirror-0
     1      1      200      1 x 4M            112b000           mirror-1
     -      2      200           0    NOT_ALLOCATABLE           mirror-2

  If the metaslab is using segment-based weighting then the WEIGHT column will
  display the number of segments available in the bucket where the allocation
  attempt was made.

Author: George Wilson <george.wilson@delphix.com>
Reviewed by: Alex Reece <alex@delphix.com>
Reviewed by: Chris Siden <christopher.siden@delphix.com>
Reviewed by: Dan Kimmel <dan.kimmel@delphix.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Paul Dagnelie <paul.dagnelie@delphix.com>
Reviewed by: Pavel Zakharov <pavel.zakharov@delphix.com>
Reviewed by: Prakash Surya <prakash.surya@delphix.com>
Reviewed by: Don Brady <don.brady@intel.com>
Approved by: Richard Lowe <richlowe@richlowe.net>
2017-03-24 09:37:00 +00:00
Enji Cooper
acc37ca1c1 cddl: normalize paths using SRCTOP-relative paths or :H when possible
This simplifies make logic/output

While here, remove bogus CFLAGS which look for headers in cddl/lib/libumem.
There aren't any source files there (just Makefiles)

MFC after:	1 month
Sponsored by:	Dell EMC Isilon
2017-03-04 11:30:04 +00:00
Mark Johnston
d935f34b8f Fix memory leaks in error cases in libdtrace.
Submitted by:	Tom Rix <trix@juniper.net>
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D9705
2017-02-23 17:56:24 +00:00
Mark Johnston
74d9553e43 Fix a memory leak in an error case in libctf.
Submitted by:	Tom Rix <trix@juniper.net>
MFC after:	1 week
2017-02-23 17:54:17 +00:00
Mark Johnston
281e4f2dd4 When patching USDT probes, use non-unique names for aliases of weak symbols.
Aliases are normally given names that include a key that's unique for each
input object file. This, for example, ensures that aliases for identically
named local symbols in different object files don't conflict. However, in
some cases the static linker will leave an undefined alias after merging
identical weak symbols, resulting in a link error. A non-unique name allows
the aliases to be merged as well.

PR:		216871
X-MFC With:	r313262
2017-02-10 02:01:32 +00:00
Mark Johnston
35bf9feb41 Search for _DTRACE_VERSION in sys/sdt.h rather than unistd.h.
MFC after:	1 week
2017-02-05 02:45:35 +00:00
Mark Johnston
55c2fd519f Avoid using Sun compiler-specific flags.
MFC after:	1 week
2017-02-05 02:44:48 +00:00
Mark Johnston
273efb05a2 Fix a double free of libelf data buffers in the USDT link code.
libdtrace needs to append to the input object files' string and symbol
tables. Currently it does so by allocating a larger buffer, copying the
existing sections into them, and swapping pointers in the libelf data
descriptors. However, it also frees those buffers when its processing is
complete, which leads to a double free since the elftoolchain libelf
owns them and also frees them in elf_end(3). Instead, free the buffers
originally allocated by libelf.

MFC after:	2 weeks
2017-02-05 02:44:08 +00:00
Mark Johnston
e801af6fba Use PC-relative relocations for USDT probe sites on i386 and amd64.
When recording probe site addresses in the output DOF file, dtrace -G
needs to emit relocations for the .SUNW_dof section in order to obtain
the addresses of functions containing probe sites. DTrace expects the
addresses to be relative to the base address of the final ELF file,
and the amd64 USDT implementation was relying on some unspecified and
incorrect behaviour in the base system GNU ld to achieve this.

This change reimplements the probe site relocation handling to allow
USDT to be used with lld and newer GNU binutils. Specifically, it
makes use of R_X86_64_PC64/R_386_PC32 relocations to obtain the
probe site address relative to the DOF file address, and adds and uses a
new DOF relocation type which computes the final probe site address using
these relative offsets.

Reported by and discussed with:	Rafael Espíndola
MFC after:	1 month
Differential Revision:	https://reviews.freebsd.org/D9374
2017-02-05 02:39:12 +00:00
George V. Neville-Neil
82988b50a1 Add an mbuf to ipinfo_t translator to finish cleanup of mbuf passing to TCP probes.
Reviewed by:	markj
MFC after:	2 weeks
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D9401
2017-02-01 19:33:00 +00:00
Alan Somers
9df99d6ed0 Fix an unchecked return value in zfsd
It's pretty unlikely to actually hit this, but good to check it anyway

Reported by:	Coverity
CID:		1362018
MFC after:	4 weeks
Sponsored by:	Spectra Logic Corp
2017-01-18 22:10:18 +00:00
Andrey V. Elsukov
4349a2145f Convert ipv4_flags and ipv4_offset fields into host byte order.
Also save only high bits in the ipv4_flags, because it is defined
as uint8_t. So now it will show DF and MF flags as 0x40 and 0x20.

Reviewed by:	markj@
MFC after:	1 week
2016-12-29 20:27:54 +00:00
Mark Johnston
3a3e3279e7 Avoid modifying the object string table when patching USDT probes.
dtrace converts pairs of consecutive underscores in a probe name to dashes.
When dtrace -G processes relocations corresponding to USDT probe sites, it
performs this conversion on the corresponding symbol names prior to looking
up the resulting probe names in the USDT provider definition. However, in
so doing it would actually modify the input object's string table, which
breaks the string suffix merging done by recent binutils. Because we don't
care about the symbol name once the probe site is recorded, just perform the
probe lookup using a temporary copy.

Reported by:	hrs, swills
MFC after:	3 weeks
2016-12-20 18:25:41 +00:00
Mark Johnston
a46000e8ff Consistently print D variable indices in decimal when disassembling.
MFC after:	1 week
2016-12-20 05:45:52 +00:00
Mark Johnston
90556839e9 Skip a ustack test that triggers an assertion on INVARIANTS kernels.
Reported by:	ngie
Sponsored by:	Dell EMC Isilon
X-MFC-With:	r309698
2016-12-14 19:01:43 +00:00
Mark Johnston
d78e7e3b58 err.D_PROC_CREATEFAIL.many.d passes, so remove the EFAIL annotation.
MFC after:	1 week
Sponsored by:	Dell EMC Isilon
2016-12-08 18:56:35 +00:00
Mark Johnston
3c606f671e Use the correct path to date(1).
MFC after:	1 week
Sponsored by:	Dell EMC Isilon
2016-12-07 23:38:18 +00:00
Mark Johnston
058f5a9a47 Use the native data model instead of forcing ILP32 in tst.provregex3.ksh.
MFC after:	1 week
Sponsored by:	Dell EMC Isilon
2016-12-07 23:37:51 +00:00
Mark Johnston
6b91eae9d2 Run DTrace test scripts with "tst" set to the test script file name.
MFC after:	1 week
Sponsored by:	Dell EMC Isilon
2016-12-07 23:36:15 +00:00
Mark Johnston
e0d70fc1dc libdtrace: Don't use a read-only handle for enumerating pid probes.
Enumeration of return probes involves disassembling subroutines in the
target process, and ptrace(2) is currently used to read from the target
process. libproc could read from the backing file instead to avoid this
problem, but in the common case libdtrace will have a writeable handle
on the process anyway. In particular, a writeable handle is needed to list
USDT probes, and libdtrace will cache such a handle for processes that it
controls via dtrace -c and -p.
2016-12-06 04:28:56 +00:00
Mark Johnston
b043b5dc6b libproc: Add support for some proc_attach() flags.
This change adds some handling for the equivalent of Solaris' PGRAB_*
flags. In particular, support for PGRAB_RDONLY is needed to avoid a
nasty deadlock: dtrace(1) may otherwise stop the master process for its
pseudo-terminal and end up blocking while writing to standard output.
2016-12-06 04:22:38 +00:00
Mark Johnston
d42df2a447 libproc: Match prefixes when looking up mapped object by name.
When looking up an object by name, allow prefix matches if no direct match
is found. This allows one to, for example, match libc entry probes with:

 # dtrace -n 'pid$target:libc.so::entry' -c ./foo

instead of requiring "libc.so.7" or a glob.

Also remove proc_obj2map() as it currently just duplicates the
functionality of proc_name2map(). It's supposed to take a Solaris
link-map ID as a paramter, but support for this isn't implemented and
isn't required to support DTrace's pid provider.
2016-12-06 04:20:32 +00:00
Pedro F. Giffuni
69718b786d Revert r253678, r253661:
Fix a segfault in ctfmerge(1) due to a bug in GCC.

The change was correct and the bug real, but upstream didn't adopt it
and we want to remain in sync. When/if upstream does something about it
we can bring their version.

The bug in question was fixed in GCC 4.9 which is now the default in
FreeBSD's ports. Our native gcc-4.2, which is still in use in some Tier-2
platforms also has a workaround so no end-user should be harmed by the
revert.
2016-12-03 17:44:43 +00:00
Andriy Gapon
61158a7ce8 MFV r308989: 6428 set canmount=off on unmounted filesystem tries to
unmount children

This is a cosmetic and bookkeeping change as the actual change is
already in FreeBSD.
See r297521, r304520, r308985.
2016-11-24 10:11:09 +00:00
Andriy Gapon
9170c18bb9 revert r304520, set canmount=on is not supposed to mount the filesystem
Not sure where I got the idea that it should.

See https://github.com/openzfs/openzfs/pull/218

Reported by:	mahrens
Pointyhat to:	avg
MFC after:	5 days
2016-11-22 11:44:30 +00:00
Alexander Motin
14b5719f6a After some ZIL changes 6 years ago zil_slog_limit got partially broken
due to zl_itx_list_sz not updated when async itx'es upgraded to sync.
Actually because of other changes about that time zl_itx_list_sz is not
really required to implement the functionality, so this patch removes
some unneeded broken code and variables.

Original idea of zil_slog_limit was to reduce chance of SLOG abuse by
single heavy logger, that increased latency for other (more latency critical)
loggers, by pushing heavy log out into the main pool instead of SLOG. Beside
huge latency increase for heavy writers, this implementation caused double
write of all data, since the log records were explicitly prepared for SLOG.
Since we now have I/O scheduler, I've found it can be much more efficient
to reduce priority of heavy logger SLOG writes from ZIO_PRIORITY_SYNC_WRITE
to ZIO_PRIORITY_ASYNC_WRITE, while still leave them on SLOG.

Existing ZIL implementation had problem with space efficiency when it
has to write large chunks of data into log blocks of limited size. In some
cases efficiency stopped to almost as low as 50%. In case of ZIL stored on
spinning rust, that also reduced log write speed in half, since head had to
uselessly fly over allocated but not written areas. This change improves
the situation by offloading problematic operations from z*_log_write() to
zil_lwb_commit(), which knows real situation of log blocks allocation and
can split large requests into pieces much more efficiently. Also as side
effect it removes one of two data copy operations done by ZIL code WR_COPIED
case.

While there, untangle and unify code of z*_log_write() functions.
Also zfs_log_write() alike to zvol_log_write() can now handle writes crossing
block boundary, that may also improve efficiency if ZPL is made to do that.

Sponsored by:	iXsystems, Inc.
2016-11-17 21:01:27 +00:00
Mark Johnston
e20727061f Define dependencies for some auto-generated source files in libdtrace.
Remove an unneeded beforedepend rule.

Reported by:	emaste, jhb
Reviewed by:	bdrewery
MFC after:	1 week
2016-11-17 18:13:42 +00:00
Mark Johnston
375c8b20dc Remove the DTrace printt and typeref actions.
These are FreeBSD-specific and were added in r178576 to provide the ability
to pretty-print instances of compound types. However, the print action has
long since been augmented to provide this functionality with a simpler
interface.

Discussed with:	gnn
Differential Revision:	https://reviews.freebsd.org/D8478
2016-11-12 19:26:12 +00:00
Andriy Gapon
d5315b02cd MFV r308222: 6051 lzc_receive: allow the caller to read the begin record
illumos/illumos-gate@620f322510
620f322510

https://www.illumos.org/issues/6051
  Currently lzc_receive() requires that its snapname argument is a snapshot name
  (contains '@').
  zfs receive allows to specify just a dataset name and would try to deduce the
  snapshot name from the stream.
  I propose to allow lzc_receive() to do the same.
  That seems to be quite easy to implement, it requires only a small amount of
  logic, it does not require any additional system calls or any additional data
  from the stream.
  The benefit is that the new behavior would allow to keep the snapshot names the
  same between the sender and receiver at zero cost, without a need to pass the
  names out of band.

Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Paul Dagnelie <pcd@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author: Andriy Gapon <avg@icyb.net.ua>
MFC after:	2 weeks
2016-11-03 09:24:27 +00:00
Andriy Gapon
97371ba2a9 zfsbootcfg: a simple tool to set next boot (one time) options for zfsboot
(gpt)zfsboot will read one-time boot directives from a special ZFS pool
area.  The area was previously described as "Boot Block Header", but
currently it is know as Pad2, marked as reserved and is zeroed out on
pool creation.  The new code interprets data in this area, if any, using
the same format as boot.config.  The area is immediately wiped out.
Failure to parse the directives results in a reboot right after the
cleanup.  Otherwise the boot sequence proceeds as usual.

zfsbootcfg writes zfsboot arguments specified on its command line to the
Pad2 area of a disk identified by vfs.zfs.boot.primary_pool and
vfs.zfs.boot.primary_vdev kenv variables that are set by loader during
boot.  Please see the manual page for more.

Thanks to all who reviewed, contributed and made suggestions!  There are
many potential improvements to the feature, please see the review for
details.

Reviewed by:	wblock (docs)
Discussed with:	jhb, tsoome
MFC after:	3 weeks
Relnotes:	yes
Differential Revision: https://reviews.freebsd.org/D7612
2016-10-29 14:09:32 +00:00
Mark Johnston
6a4985f61c Fix tst.args1.c on LP64 platforms.
The untyped probe arguments have a width larger than int on such platforms,
so printing their value without a cast can give unexpected results.

MFC after:	1 week
2016-10-16 19:50:10 +00:00
Mark Johnston
31768cc105 tst.kpriv.ksh fails because DTrace privilege levels are unimplemented.
MFC after:	1 week
2016-10-16 19:46:55 +00:00
George V. Neville-Neil
956c2a73cd Corrected non-portable reuse of va_list in dt_printf()
Submitted by:	Graeme Jenkinson
Reviewed by:	markj
MFC after:	2 weeks
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D8157
2016-10-11 16:12:12 +00:00
Alexander Motin
929d0128f7 MFV r304159: 7277 zdb should be able to print zfs_dbgmsg's
illumos/illumos-gate@29bdd2f916
https://github.com/illumos/illumos-gate/commit/29bdd2f916366ece37c4748bca6b3d61f
57a223b

https://www.illumos.org/issues/7277
  ztest always prints the debug messages (zfs_dbgmsg()) by calling
  zfs_dbgmsg_print(). We should add a flag to zdb to make it do this as well
  before exiting.

Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Igor Kozhukhov <ikozhukhov@gmail.com>
Approved by: Dan McDonald <danmcd@omniti.com>
Author: Pavel Zakharov <pavel.zakharov@delphix.com>
2016-09-03 10:07:46 +00:00
Alexander Motin
5535f02daf MFV r303081: 7163 ztest failures due to excess error injection
illumos/illumos-gate@f34284d835
https://github.com/illumos/illumos-gate/commit/f34284d835bc555f987c1310df46c034c
3101155

https://www.illumos.org/issues/7163
  Running zloop from zfs-precommit hit this assertion:
       *panicstr/s
  0xfffffd7fd7419370: assertion failed for thread 0xfffffd7fe29ed240,
  thread-id 577: parent != NULL, file ../../../uts/common/fs/zfs/dbuf.c, line
  1827
       $c
  libc.so.1`_lwp_kill+0xa()
  libc.so.1`_assfail+0x182(fffffd7ffb1c29fa, fffffd7ffb1cc028, 723)
  libc.so.1`assfail+0x19(fffffd7ffb1c29fa, fffffd7ffb1cc028, 723)
  libzpool.so.1`dbuf_dirty+0xc69(10e3bc10, 3601700)
  libzpool.so.1`dbuf_dirty+0x61e(10c73640, 3601700)
  libzpool.so.1`dbuf_dirty+0x61e(10e28280, 3601700)
  libzpool.so.1`dmu_buf_will_fill+0x64(10e28280, 3601700)
  libzpool.so.1`dmu_write+0x1b6(2c7e640, d, 400000002e000000, 200, 3717b40,
  3601700)
  ztest_replay_write+0x568(4950d0, 3717a80, 0)
  ztest_write+0x125(4950d0, d, 400000002e000000, 200, 413f000)
  ztest_io+0x1bb(4950d0, d, 400000002e000000)
  ztest_dmu_write_parallel+0xaa(4950d0, 6)
  ztest_execute+0x83(1, 420c98, 6)
  ztest_thread+0xf4(6)
  libc.so.1`_thrp_setup+0x8a(fffffd7fe29ed240)
  libc.so.1`_lwp_start()
  This is another manifestation of ECKSUM in ztest:
  The lowest level ancestor that’s in memory is the L8 (topmost). The L7
  ancestor is blkid 0x10:
       ::dbufs -O 0x2c7e640 -o d -l 7 |::dbuf
  addr object lvl blkid holds os
  600be50 d 7 4 1 ztest/ds_6
  719d880 d 7 0 4 ztest/ds_6

Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Paul Dagnelie <pcd@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author: Matthew Ahrens <mahrens@delphix.com>
2016-09-03 08:48:51 +00:00
Alexander Motin
1f0bf00253 MFV r303080: 6451 ztest fails due to checksum errors
illumos/illumos-gate@f9eb9fdf19
https://github.com/illumos/illumos-gate/commit/f9eb9fdf196b6ed476e4ffc69cecd8b0d
a3cb7e7

https://www.illumos.org/issues/6451
  Sometimes ztest fails because zdb detects checksum errors. e.g.:
  Traversing all blocks to verify checksums and verify nothing leaked ...
  zdb_blkptr_cb: Got error 50 reading <71, 47, 0, 8000160> DVA0=<0:1cc2000:
  180000> [L0 other uint64[]] sha256 uncompressed LE contiguou
  s unique single size=100000L/100000P birth=271L/271P fill=1
  cksum=c5a3e27d1ed0f894:843bca3a5473c4bf:f76a19b6830a2e4:91292591613a12bf --
  skipping
  zdb_blkptr_cb: Got error 50 reading <71, 47, 0, 800000180> DVA0=<0:ce16800:
  180000> [L0 other uint64[]] sha256 uncompressed LE contigu
  ous unique single size=100000L/100000P birth=840L/840P fill=1
  cksum=5d018f3d061e17f3:6d1584784587bf63:2805a74a0ce37369:ba68a214806c7e75
  -- skipping
  zdb_blkptr_cb: Got error 50 reading <71, 47, 0, 1000000360> DVA0=<0:10d37400:
  180000> [L0 other uint64[]] sha256 uncompressed LE conti
  guous unique single size=100000L/100000P birth=904L/904P fill=1
  cksum=fa1e11d4138bd14b:86c9488c444473e3:f31e43c72e72e46b:e3446472d1174d
  ba -- skipping
  zdb_blkptr_cb: Got error 50 reading <71, 47, 0, 400000002c0> DVA0=<0:127ef400:
  180000> [L0 other uint64[]] sha256 uncompressed LE cont
  iguous dedup single size=100000L/100000P birth=549L/549P fill=1
  cksum=30e14955ebf13522:66dc2ff8067e6810:4607e750abb9d3b3:6582b8af909fcb
  58 -- skipping
  zdb_blkptr_cb: Got error 50 reading <657, 5, 0, 1c0> DVA0=<0:1a180400:180000>
  [L0 other uint64[]] fletcher4 uncompressed LE contiguou
  s unique single size=100000L/100000P birth=1091L/1091P fill=1 cksum=a6cf1e50:
  29b3bd01c57e5:36779b914035db9a:db61cdcf6bec56f0 -- skippin
  g
  The problem is that ztest_fault_inject() can inject multiple faults into the
  same block. It is designed such that it can inject errors on all leafs of a
  RAID-Z or mirror, but for a given range of offsets, it will only inject errors

Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Prakash Surya <prakash.surya@delphix.com>
Reviewed by: Jorgen Lundman <lundman@lundman.net>
Approved by: Dan McDonald <danmcd@omniti.com>
Author: Matthew Ahrens <mahrens@delphix.com>
2016-09-03 08:47:46 +00:00
Alexander Motin
cdd7c5d9b0 MFV r303079:
7147 ztest: ztest_ddt_repair fails with ztest_pattern_match assertion

illumos/illumos-gate@aab8072633
https://github.com/illumos/illumos-gate/commit/aab80726335c76a7cae32c7300890248d
73a51e3

https://www.illumos.org/issues/7147
  Here's the dbuf we're currently reading:
       966f200::dbuf
  addr object lvl blkid holds os
  966f200 4 0 0 1 ztest/ds_3
       966f200::print dmu_buf_t db_data
  db_data = 0x9ae0400
       0x9ae0400/10J
  0x9ae0400: c1c7ced932020d c1c7ced932020d c1c7ced932020d c1c7ced932020d
  c1c7ced932020d c1c7ced932020d c1c7ced932020d c1c7ced932020d
  c1c7ced932020d c1c7ced932020d
  The pattern we're expecting is actually this: a34ae10b5f2db2. If we attempt to
  read the block on disk we find that it has matches what ztest_ddt_repair()
  would have written:
       ~c1c7ced932020d=J
  ff3e383126cdfdf2
       966f200::print dmu_buf_impl_t db_blkptr | ::blkptr
  DVA0=<0:71d3c00:800>
  [L0 UINT64_OTHER] SHA256 OFF LE contiguous dedup single
  size=400L/400P birth=55L/55P fill=1
  cksum=18486450d3ce8c6d:75a72f4bbf117b0f:2d3a226314eb5650:2eb0fd68648b1af0
     1. zdb -U /rpool/tmp/zpool.cache -R ztest 0:71d3c00:800 | head
        Found vdev type: mirror
  0:71d3c00:800
  0 1 2 3 4 5 6 7 8 9 a b c d e f 0123456789abcdef
  000000: ff3e383126cdfdf2 ff3e383126cdfdf2 ...&18>....&18>.
  000010: ff3e383126cdfdf2 ff3e383126cdfdf2 ...&18>....&18>.
  000020: ff3e383126cdfdf2 ff3e383126cdfdf2 ...&18>....&18>.
  000030: ff3e383126cdfdf2 ff3e383126cdfdf2 ...&18>....&18>.
  000040: ff3e383126cdfdf2 ff3e383126cdfdf2 ...&18>....&18>.
  000050: ff3e383126cdfdf2 ff3e383126cdfdf2 ...&18>....&18>.

Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Prakash Surya <prakash.surya@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author: George Wilson <george.wilson@delphix.com>
2016-09-03 08:46:53 +00:00
Alexander Motin
efa0867fb0 MFV r302991: 6950 ARC should cache compressed data
illumos/illumos-gate@dcbf3bd6a1
dcbf3bd6a1

https://www.illumos.org/issues/6950
  When reading compressed data from disk, the ARC should keep the compressed
  block cached and only decompress it when consumers access the block. The
  uncompressed data should be short-lived allowing the ARC to cache a much larger
  amount of data. The DMU would also maintain a smaller cache of uncompressed
  blocks to minimize the impact of decompressing frequently accessed blocks.

Reviewed by: Prakash Surya <prakash.surya@delphix.com>
Reviewed by: Dan Kimmel <dan.kimmel@delphix.com>
Reviewed by: Matt Ahrens <mahrens@delphix.com>
Reviewed by: Paul Dagnelie <pcd@delphix.com>
Reviewed by: Don Brady <don.brady@intel.com>
Reviewed by: Richard Elling <Richard.Elling@RichardElling.com>
Approved by: Richard Lowe <richlowe@richlowe.net>
Author: George Wilson <george.wilson@delphix.com>
2016-09-03 08:30:51 +00:00
Alexander Motin
3d1e0e0830 MFV r302662: 6447 handful of nvpair cleanups
illumos/illumos-gate@759e89be35
https://github.com/illumos/illumos-gate/commit/759e89be359f2af635e4122d147df56bc
e948773

https://www.illumos.org/issues/6447
  I got a patch from someone who uses nvpair code outside of illumos. It fixes a
  couple of gcc warnings/bugs for him.
     1. silence uninitialized use warnings
     2. add parentheses around assignment used as truth value
     3. fix printf format specifier (ll is for integers only)
     4. strstr, strspn, strcspn, and strcmp are declared in string.h, not
        strings.h.
     5. avoid scanning integer into boolean variable

Reviewed by: Josef 'Jeff' Sipek <jeffpc@josefsipek.net>
Reviewed by: Andy Stormont <astormont@racktopsystems.com>
Reviewed by: Garrett D'Amore <garrett@damore.org>
Approved by: Robert Mustacchi <rm@joyent.com>
Author: Steve Dougherty <sdougherty@barracuda.com>
2016-09-01 15:17:39 +00:00
Alexander Motin
41b9077ef6 MFV r302660: 6314 buffer overflow in dsl_dataset_name
illumos/illumos-gate@9adfa60d48
https://github.com/illumos/illumos-gate/commit/9adfa60d484ce2435f5af77cc99dcd4e6
92b6660

https://www.illumos.org/issues/6314
  Callers of dsl_dataset_name pass a buffer of size ZFS_MAXNAMELEN, but
  dsl_dataset_name copies the datasets' name PLUS the snapshot name to it,
  resulting in a max of 2 * ZFS_MAXNAMELEN + '@'.

Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Prakash Surya <prakash.surya@delphix.com>
Reviewed by: Igor Kozhukhov <ikozhukhov@gmail.com>
Approved by: Dan McDonald <danmcd@omniti.com>
Author: Matthew Ahrens <mahrens@delphix.com>
2016-09-01 15:08:27 +00:00
Alexander Motin
e12a269749 MFV r302659: 6931 lib/libzfs: cleanup gcc warnings
illumos/illumos-gate@88f61dee20
88f61dee20

https://www.illumos.org/issues/6931
  need cleanup:
  CERRWARN += -_gcc=-Wno-switch
  CERRWARN += -_gcc=-Wno-parentheses
  CERRWARN += -_gcc=-Wno-unused-function

Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author: Igor Kozhukhov <ikozhukhov@gmail.com>
2016-09-01 14:57:06 +00:00
Alexander Motin
d61364f631 MFV r302658: 6872 zfs libraries should not allow uninitialized variables
illumos/illumos-gate@f83b46baf9
https://github.com/illumos/illumos-gate/commit/f83b46baf98d276f5f84fa84c8b461f41
2ac1f5e

https://www.illumos.org/issues/6872
  We compile the zfs libraries with -Wno-uninitialized. We should remove
  this. Change makefiles, fix new warnings, fix pbchk errors.

Reviewed by: Dan Kimmel <dan.kimmel@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Prakash Surya <prakash.surya@delphix.com>
Reviewed by: Yuri Pankov <yuri.pankov@nexenta.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author: Paul Dagnelie <pcd@delphix.com>
2016-09-01 14:53:56 +00:00
Alexander Motin
35684d801f MFV r302657: 4521 zfstest is trying to execute evil "zfs unmount -a"
illumos/illumos-gate@8808ac5dae
https://github.com/illumos/illumos-gate/commit/8808ac5dae118369991f158b6ab736cb2
691ecde

https://www.illumos.org/issues/4521
  zfstest is trying to execute evil "zfs unmount -a", which fails (fortunately,
  as it would otherwise leave me with my ~ missing):
  03:44:11.86 cannot unmount '/export/home/yuri': Device busy cannot unmount '/
  export/home': Device busy
  03:44:11.86 ERROR: /usr/sbin/zfs unmount -a exited 1
  This affects, at least, zfs_mount_009_neg and zfs_mount_all_001_pos, both
  failing on that step. The pool containing the /export/home hierarchy is
  included in KEEP variable, but it doesn't seem to affect anything here.

Reviewed by: Andriy Gapon <avg@FreeBSD.org>
Reviewed by: Dan McDonald <danmcd@omniti.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: John Kennedy <john.kennedy@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author: Yuri Pankov <yuri.pankov@nexenta.com>
2016-09-01 14:52:05 +00:00
Alexander Motin
c7eebd2b5d MFV r302655: 6873 zfs_destroy_snaps_nvl leaks errlist
illumos/illumos-gate@4cde22c299
https://github.com/illumos/illumos-gate/commit/4cde22c29999ffb907ca39d2ebd512812
f7e5168

https://www.illumos.org/issues/6873
  lzc_destroy_snaps() returns an nvlist in errlist.
  zfs_destroy_snaps_nvl() should nvlist_free() it before returning.

Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Paul Dagnelie <pcd@delphix.com>
Approved by: Dan McDonald <danmcd@omniti.com>
Author: Chris Williamson <chris.williamson@delphix.com>
2016-09-01 14:47:43 +00:00
Alexander Motin
e736489e04 MFV r302654:
6879 incorrect endianness swap for drr_spill.drr_length in libzfs_sendrecv.c

illumos/illumos-gate@20fea7a474
https://github.com/illumos/illumos-gate/commit/20fea7a47472aceb64d3ed48cc2a3ea26
8bc4795

https://www.illumos.org/issues/6879
  In libzfs_sendrecv, there's a typo:
  case DRR_SPILL:
              if (byteswap) {
                  drr->drr_u.drr_write.drr_length =
                      BSWAP_64(drr->drr_u.drr_spill.drr_length);
              }
  Instead of drr_write.drr_length, we should be assigning the result of the
  byteswap to drr_spill.drr_length.

Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Paul Dagnelie <pcd@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author: Dan Kimmel <dan.kimmel@delphix.com>
2016-09-01 14:46:33 +00:00
Alexander Motin
b7d729391d MFV r302653: 6111 zfs send should ignore datasets created after the ending snapshot
illumos/illumos-gate@4a20c933b1
https://github.com/illumos/illumos-gate/commit/4a20c933b148de8a1c1d3538391c64284
e636653

https://www.illumos.org/issues/6111
  If you create a zfs child folder, zfs send returns an error when a recursive
  incremental send is done between two snapshots made prior to the folder
  creation.
  The problem can be reproduced with the following steps.
  root@zfs:/# zfs create pool/test
  root@zfs:/# zfs snapshot pool/test@snap1
  root@zfs:/# zfs snapshot pool/test@snap2
  root@zfs:/# zfs create pool/test/child
  root@zfs:/# zfs send -R -I pool/test@snap1 pool/test@snap2 > /dev/null
  WARNING: could not send pool/test/child@snap2: does not exist
  WARNING: could not send pool/test/child@snap2: does not exist
  root@zfs:/# echo $?
  1
  root@zfs:/# zfs snapshot -r pool/test@snap3
  root@zfs:/# zfs send -R -I pool/test@snap1 pool/test@snap3 > /dev/null
  root@zfs:/# echo $?
  0
  root@zfs:/# zfs send -R -I pool/test@snap2 pool/test@snap3 > /dev/null
  root@zfs:/# echo $?
  0
  Since pool/test/child was created after snap2, zfs send should not expect snap2
  to be in pool/test/child when doing a recursive send. It should examine the
  compare the creation time of the snapshot and each child folder to decide if
  the folder will be sent. The next incremental send between snap2 and snap3
  would properly create the child folder and snap3 which first appears in the
  child folder.
  The problem is identical if '-i' is used instead of '-I'.

Reviewed by: Alex Aizman alex.aizman@nexenta.com
Reviewed by: Alek Pinchuk alek.pinchuk@nexenta.com
Reviewed by: Roman Strashkin roman.strashkin@nexenta.com
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Paul Dagnelie <pcd@delphix.com>
Approved by: Garrett D'Amore <garrett@damore.org>
Author: Alex Deiter <alex.deiter@nexenta.com>
2016-09-01 14:45:11 +00:00
Alexander Motin
4536fd9bed MFV r302643:
6902 speed up listing of snapshots if requesting name only and sorting by name

This was our change from the beginning, so just reduce the upstream diff.
2016-09-01 13:29:53 +00:00
Alexander Motin
df4ed04000 MFV r302642:
6876 Stack corruption after importing a pool with a too-long name

illumos/illumos-gate@c971037baa
c971037baa

https://www.illumos.org/issues/6876
  Calling dsl_dataset_name on a dataset with a 256 byte buffer is asking for
  trouble. We should check every dataset on import, using a 1024 byte buffer and
  checking each time to see if the dataset's new name is longer than 256 bytes.

Reviewed by: Prakash Surya <prakash.surya@delphix.com>
Reviewed by: Dan Kimmel <dan.kimmel@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Yuri Pankov <yuri.pankov@nexenta.com>
Approved by: Richard Lowe <richlowe@richlowe.net>
Author: Paul Dagnelie <pcd@delphix.com>
2016-09-01 13:14:31 +00:00
Bryan Drewery
1eb4ec9c62 DIRDEPS_BUILD: Add some missing dirctories to the build.
Sponsored by:	EMC / Isilon Storage Division
2016-08-31 19:30:59 +00:00
Bryan Drewery
776d5e11e9 DIRDEPS_BUILD: Update dependencies.
Sponsored by:	EMC / Isilon Storage Division
2016-08-31 19:30:46 +00:00
Mark Johnston
7447ca0eb2 Recursively enumerate anonymous structs and unions in ctf_member_info().
Previously, ctf_member_info() would ignore members belonging to an
anonymous struct or union. This made it impossible to, for example, trace
the m_next field of an mbuf using DTrace.

Reported and tested by:	gallatin
MFC after:	2 weeks
2016-08-30 05:50:42 +00:00
Enji Cooper
47d6e5cb86 Remove redundant declarations and simplify ../ in pathing
- TESTSBASE and LOCALBASE are already defined in bsd.tests.mk
- TESTSDIR is automatically divined as ${TESTSBASE}${RELDIR:H} after
  r289158.
- Replace SRCDIR with SRCTOP

MFC after:	1 week
X-MFC with:	r305019
Sponsored by:	EMC / Isilon Storage Division
2016-08-29 19:10:58 +00:00
Enji Cooper
68aee22c0e Remove unnecessary variable (SRCDIR) replaced by SRCTOP in Makefile.common
MFC after:	1 week
X-MFC with:	r305018
Sponsored by:	EMC / Isilon Storage Division
2016-08-29 19:07:48 +00:00
Enji Cooper
17a7f5c474 Use SRCTOP instead of a homegrown definition for it (SRCDIR)
MFC after:	1 week
Sponsored by:	EMC / Isilon Storage Division
2016-08-29 19:06:40 +00:00
Dimitry Andric
3323aadf23 Fix the zfsd unittest:
* TESTSDIR is supposed to be under cddl/usr.sbin, not cddl/sbin
* DevdCtl::EventBuffer no longer exists, so remove its forward
  declaration

MFC after:	3 days
X-MFC-With:	r305013
2016-08-29 18:55:06 +00:00
Dimitry Andric
895897884f Add an empty virtual destructor to zfsd's Vdev class. This is needed
because the class has virtual functions, and the compiler-generated
default destructor is non-virtual.

Reviewed by:	asomers
MFC after:	3 days
2016-08-29 18:51:49 +00:00
George V. Neville-Neil
11e72c3b25 Unlike Solaris, in FreeBSD p_args can be 0 so check for that
instead of walking down to ar_args blindly.

Reported by:	Amanda Strnad
Reviewed by:	markj, jhb
MFC after:	2 weeks
Sponsored by:	DARPA, AFRL
2016-08-25 23:24:57 +00:00
Andriy Gapon
723af0425e fix bug introduced in r297521, set canmount=on doesn't mount filesystem
There are two cases where changing canmount should result in an action:
- canmount is set to off for a mounted filesystem
- canmount is set to on for an unmounted filesystem
Before r297521 we could unmount and re-mount a filesystem when that was
not necessary, but after r297521 we only handled the first of the above
cases.

MFC after:	5 days
2016-08-20 09:12:01 +00:00
Mark Johnston
be9cb745bc Add a SIGINFO handler for dtrace(1).
Have it print the contents of aggregations, if any. Otherwise, one needs to
kill the running script to view the collected data, or add code to
periodically print it.

Discussed with:	gnn
MFC after:	1 month
2016-08-18 17:27:58 +00:00
Mark Johnston
5968c00154 Regenerate DTrace tests. 2016-08-16 02:34:25 +00:00
Mark Johnston
650f66ac9a MFV r304057:
7085 add support for "if" and "else" statements in dtrace

illumos/illumos-gate@c3bd3abd88

Add syntactic sugar to dtrace: "if" and "else" statements. The sugar is
baked down to standard dtrace features by adding additional clauses with
the appropriate predicates.

Reviewed by: Adam Leventhal <ahl@delphix.com>
Reviewed by: Sebastien Roy <sebastien.roy@delphix.com>
Reviewed by: Paul Dagnelie <pcd@delphix.com>
Reviewed by: Bryan Cantrill <bryan@joyent.com>
Approved by: Richard Lowe <richlowe@richlowe.net>
Author: Matthew Ahrens <mahrens@delphix.com>

MFC after:	2 weeks
Relnotes:	yes
2016-08-16 02:30:19 +00:00
Mark Johnston
b7125fa9cd MFV r296989:
6734 dtrace_canstore_statvar() fails for some valid static variables

Reviewed by: Dan McDonald <danmcd@omniti.com>
Approved by: Richard Lowe <richlowe@richlowe.net>
Author: Bryan Cantrill <bryan@joyent.com>

illumos/illumos-gate@d65f2bb4e5

MFC after:	2 weeks
2016-08-16 02:16:54 +00:00
Mark Johnston
ba69823e5c MFV r304056:
5396 fix longjmp clobbering errors

illumos/illumos-gate@67a4bb8f9a

Reviewed by: Josef 'Jeff' Sipek <jeffpc@josefsipek.net>
Reviewed by: Igor Kozhukhov <ikozhukhov@gmail.com>
Approved by: Richard Lowe <richlowe@richlowe.net>
Author: Gary Mills <gary_mills@fastmail.fm>
2016-08-16 02:14:36 +00:00
Mark Johnston
ed12504ac7 dtraceUtil/tst.DataModel32.d.ksh passes on amd64. 2016-08-15 19:47:04 +00:00
Mark Johnston
eb9da1ada8 Fix handling of forward enum declarations in the CTF tools.
Reported by:	mmacy
MFC after:	2 weeks
2016-08-13 19:51:00 +00:00
Enji Cooper
08dde4d421 Highball memory requirement (4GB) with common/{raise,safety}
Both test suites require more memory than my amd64 VM using
GENERIC-NODEBUG can provide and reliably panic it with OOM issues in
dtrace(4).

Some of the testcases fail, but this at least bypasses the panic behavior
on platforms that don't have enough resources

MFC after: 2 weeks
Discussed with: markj
Sponsored by: EMC / Isilon Storage Division
2016-08-10 03:10:34 +00:00
Enji Cooper
454df496b8 Cast result from third parameter to int instead of promoting it to size_t
This resolves a -Wformat issue when the value is used as a format width
precision specifier, i.e. %*s

MFC after: 1 month
Reported by: clang
Sponsored by: EMC / Isilon Storage Division
2016-07-31 06:03:27 +00:00
Mark Johnston
907b59d769 libdtrace: Don't hard-code the native data model.
MFC after:	1 month
2016-07-30 03:09:57 +00:00
Andriy Gapon
7ad541f1f8 MFV r303083: 7164 zdb should be able to open the root dataset
Note: conversion of the manual page change from roff to mdoc is mine.

illumos/illumos-gate@b702644a6e
b702644a6e

https://www.illumos.org/issues/7164
  If the pool/dataset command-line argument is specified with a trailing
  slash, for example, "tank/", we should interpret it as the topmost
  dataset (rather than the whole pool)

Reviewed by: Pavel Zakharov <pavel.zakharov@delphix.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author: Tim Chase <tim@chase2k.com>
PR:		204661
MFC after:	1 week
Relnotes:	yes
2016-07-20 11:15:32 +00:00
Andriy Gapon
495ec94fd7 MFV r303082: 6391 Override default SPA config location via environment
illumos/illumos-gate@ae24175b2b
ae24175b2b

https://www.illumos.org/issues/6391
  When using zdb with non-default SPA config file it is not convenient
  to add -U <non-default-config-file-path> all the time. This commit
  introduces support for setting/overriding SPA config location via
  environment variable 'SPA_CONFIG_PATH'.
  If -U flag is specified in the command line it will override any other
  value as usual.

64d7b6cf75

Reviewed by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed by: Richard Yao <ryao@gentoo.org>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Will Andrews <will@freebsd.org>
Reviewed by: George Wilson <george.wilson@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author: Cyril Plisko <cyril.plisko@mountall.com>
MFC after:	1 week
2016-07-20 10:06:56 +00:00
Eric van Gyzen
464d005d84 zpool(8): update Hot Spares section to mention zfsd(8)
Reviewed and reworded by:	asomers
MFC after:	3 days
Sponsored by:	Dell Inc.
2016-07-13 21:27:10 +00:00
George V. Neville-Neil
15dbc1605b On FreeBSD there is a setsockopt option SO_USER_COOKIE which allows
setting a 32 bit value on each socket. This can be used by applications
and DTrace as a rendezvous point so that an applicaton's data can
more easily be captured at run time. Expose the user cookie via
DTrace by updating the translator in tcp.d and add a quick test
program, a TCP server, that sets the cookie on each connection
accepted.

Reviewed by:	hiren
MFC after:	1 week
Sponsored by:	Limelight Networks
Differential Revision:	https://reviews.freebsd.org/D7152
2016-07-08 23:44:09 +00:00
Alan Somers
58406fff52 Raise the WARNS level in cddl/lib
cddl/lib/libavl/Makefile
cddl/lib/libctf/Makefile
cddl/lib/libnvpair/Makefile
cddl/lib/libumem/Makefile
cddl/lib/libuutil/Makefile
	Increase WARNS to the highest working level for each of these
	libraries

Approved by:	re (gjb, hrs)
MFC after:	4 weeks
Sponsored by:	Spectra Logic Corp
2016-06-23 15:02:57 +00:00
Allan Jude
642711332d Fix missing space in mandoc syntax
Reported by:	rpokala
2016-06-01 06:18:34 +00:00
Allan Jude
00efa805e9 Update zfs(8) and zpool-features(7) man pages with new hashing algorithms
Sponsored by:	ScaleEngine Inc.
2016-06-01 03:36:31 +00:00
Alan Somers
92c6a88b76 Coverity fixes for r300906
lib/libdevdctl/consumer.cc
	In Consumer::DisconnectFromDevd, don't close the socket if it's
	already closed.

cddl/usr.sbin/zfsd/case_file.cc
lib/libdevdctl/consumer.h
	Delete dead code leftover from before devd(8) gained SOCK_SEQPACKET
	support

Reported by:	Coverity
CID:		1356155, 1356169
Sponsored by:	Spectra Logic Corp
2016-05-31 23:26:45 +00:00
Pedro F. Giffuni
0f2297b1bf zfsd: Remove a redundant semicolon, 2016-05-31 21:23:57 +00:00
Pedro F. Giffuni
52832d772e zfsd: minor spelling fix. 2016-05-30 19:32:56 +00:00
Bryan Drewery
fdd9048a63 Avoid more literal-suffix errors with C++11 2016-05-29 00:40:29 +00:00
Alan Somers
7a0c41d5d7 zfsd(8), the ZFS fault management daemon
Add zfsd, which deals with hard drive faults in ZFS pools. It manages
hotspares and replements in drive slots that publish physical paths.

cddl/usr.sbin/zfsd
	Add zfsd(8) and its unit tests

cddl/usr.sbin/Makefile
	Add zfsd to the build

lib/libdevdctl
	A C++ library that helps devd clients process events

lib/Makefile
share/mk/bsd.libnames.mk
share/mk/src.libnames.mk
	Add libdevdctl to the build. It's a private library, unusable by
	out-of-tree software.

etc/defaults/rc.conf
	By default, set zfsd_enable to NO

etc/mtree/BSD.include.dist
	Add a directory for libdevdctl's include files

etc/mtree/BSD.tests.dist
	Add a directory for zfsd's unit tests

etc/mtree/BSD.var.dist
	Add /var/db/zfsd/cases, where zfsd stores case files while it's shut
	down.

etc/rc.d/Makefile
etc/rc.d/zfsd
	Add zfsd's rc script

sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c
	Fix the resource.fs.zfs.statechange message. It had a number of
	problems:

	It was only being emitted on a transition to the HEALTHY state.
	That made it impossible for zfsd to take actions based on drives
	getting sicker.

	It compared the new state to vdev_prevstate, which is the state that
	the vdev had the last time it was opened.  That doesn't make sense,
	because a vdev can change state multiple times without being
	reopened.

	vdev_set_state contains logic that will change the device's new
	state based on various conditions.  However, the statechange event
	was being posted _before_ that logic took effect.  Now it's being
	posted after.

Submitted by:	gibbs, asomers, mav, allanjude
Reviewed by:	mav, delphij
Relnotes:	yes
Sponsored by:	Spectra Logic Corp, iX Systems
Differential Revision:	https://reviews.freebsd.org/D6564
2016-05-28 17:43:40 +00:00
Ruslan Bukin
fed1ca4b71 Add initial DTrace support for RISC-V.
Sponsored by:	DARPA, AFRL
Sponsored by:	HEIF5
2016-05-24 16:41:37 +00:00
George V. Neville-Neil
18849b5da0 Remove the old version of the DTraceToolkit from the source tree.
The DTraceToolkit is part of the Open DTrace effort and is supported
on FreeBSD as a port (sysutils/DTraceToolkit) which has been updated
to properly track toolkit development upstream.

Sponsored by:	DARPA, AFRL
2016-05-19 19:51:39 +00:00
Mark Johnston
aac4e902e1 Fix DTrace test ATF wrapper generation. 2016-05-10 20:25:49 +00:00
Baptiste Daroussin
25fac4a635 Rename dprintf into dbg_printf to avoid collision with dprintf(3)
When dprintf(3) in 2009 was added a _WITH_DPRINTF guard has also been added.
This rename is made in preparation for the removal of this guard
2016-05-10 07:52:17 +00:00
Baptiste Daroussin
a05f81bfa3 rename getline into zgetline to avoid collision with getline(3)
When getline(3) in 2009 was added a _WITH_GETLINE guard has also been added.
This rename is made in preparation for the removal of this guard

Obtained from:	NetBSD
2016-05-10 07:50:57 +00:00
Enji Cooper
430f7286a5 Merge ^/user/ngie/release-pkg-fix-tests to unbreak how test files are installed
after r298107

Summary of changes:

- Replace all instances of FILES/TESTS with ${PACKAGE}FILES. This ensures that
  namespacing is kept with FILES appropriately, and that this shouldn't need
  to be repeated if the namespace changes -- only the definition of PACKAGE
  needs to be changed
- Allow PACKAGE to be overridden by callers instead of forcing it to always be
  `tests`. In the event we get to the point where things can be split up
  enough in the base system, it would make more sense to group the tests
  with the blocks they're a part of, e.g. byacc with byacc-tests, etc
- Remove PACKAGE definitions where possible, i.e. where FILES wasn't used
  previously.
- Remove unnecessary TESTSPACKAGE definitions; this has been elided into
  bsd.tests.mk
- Remove unnecessary BINDIRs used previously with ${PACKAGE}FILES;
  ${PACKAGE}FILESDIR is now automatically defined in bsd.test.mk.
- Fix installation of files under data/ subdirectories in lib/libc/tests/hash
  and lib/libc/tests/net/getaddrinfo
- Remove unnecessary .include <bsd.own.mk>s (some opportunistic cleanup)

Document the proposed changes in share/examples/tests/tests/... via examples
so it's clear that ${PACKAGES}FILES is the suggested way forward in terms of
replacing FILES. share/mk/bsd.README didn't seem like the appropriate method
of communicating that info.

MFC after: never probably
X-MFC with: r298107
PR: 209114
Relnotes: yes
Tested with: buildworld, installworld, checkworld; buildworld, packageworld
Sponsored by: EMC / Isilon Storage Division
2016-05-04 23:20:53 +00:00
Glen Barber
49dae58b28 Fix including Kyuafile in packaged base system.
Fix a related typo while here.

Note, this change results in the Kyuafile inclusion in the runtime
package, which needs to be fixed, however addresses the PR as far
as I can tell in my tests.

PR:		209114
Submitted by:	ngie
Sponsored by:	The FreeBSD Foundation
2016-04-29 05:28:40 +00:00
Andriy Gapon
dbbcddb426 MFV r298471: 6052 decouple lzc_create() from the implementation details
illumos/illumos-gate@26455f9efc
26455f9efc

https://www.illumos.org/issues/6052
  At the moment type parameter of lzc_create() is of dmu_objset_type_t type.
  That exposes an implementation detail and requires sys/fs/zfs.h to be included
  in libzfs_core.h creating unnecessary coupling between libzfs_core interface
  and ZFS internals.
  I think that dmu_objset_type_t should be replaced with a libzfs_core
  enumeration of supported dataset types.
  For ABI reasons the new enumeration could be bit-compatible with
  dmu_objset_type_t.
  For example:
      typedef enum {
          LZC_DST_ZFS = 2,
          LZC_DST_ZVOL
      } lzc_dataset_type_t;

Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Approved by: Richard Lowe <richlowe@richlowe.net>
Author: Andriy Gapon <andriy.gapon@clusterhq.com>

MFC after:	2 weeks
Sponsored by:	ClusterHQ
2016-04-22 13:00:27 +00:00
Baptiste Daroussin
7b7b2a3b8a Print error messages to stderr 2016-04-16 12:32:26 +00:00
Glen Barber
0edd2576c0 MFH
Sponsored by:	The FreeBSD Foundation
2016-04-16 02:32:12 +00:00
Hiren Panchasara
2dbc593176 Fix the 'type' for a few variables from tcpcb.
Reviewed by:		markj
Sponsored by:		Limelight Networks
Differential Revision:	https://reviews.freebsd.org/D5973
2016-04-15 20:27:36 +00:00
Glen Barber
a123f26e92 MFH
Sponsored by:	The FreeBSD Foundation
2016-04-12 17:00:13 +00:00
Mark Johnston
f7d5087af2 libdtrace: Add a missing unlock to an error handler.
Submitted by:	Jihyun Yu <yjh0502@gmail.com>
MFC after:	3 days
2016-04-11 17:57:54 +00:00
Glen Barber
876d357fa7 MFH
Sponsored by:	The FreeBSD Foundation
2016-04-11 15:24:59 +00:00
Mark Johnston
b529028676 Implement support for boot-time DTrace.
This allows one to enable DTrace probes relatively early during boot,
during SI_SUB_DTRACE_ANON, before dtrace(1) can invoked. The desired
enabling is created using dtrace -A, which writes a /boot/dtrace.dof
file and uses nextboot(8) to ensure that DTrace kernel modules are loaded
and that the DOF file describing the enabling is loaded by loader(8)
during the subsequent boot. The trace output can then be fetched with
dtrace -a.

With this commit, boot-time DTrace is only functional on i386 and amd64: on
other architectures, the high-resolution timer frequency is initialized
during SI_SUB_CLOCKS and is thus not available when the anonymous
tracing state is initialized. On x86, the TSC is used and is thus available
earlier.

MFC after:	1 month
Relnotes:	yes
2016-04-10 01:25:48 +00:00
Alexander Motin
eaee150e3f MFV r297760: 6418 zpool should have a label clearing command
Reviewed by: Yuri Pankov <yuri.pankov@nexenta.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Author: Will Andrews <will@firepipe.net>

Closes #83
Closes #32

openzfs/openzfs@9663688425

FreeBSD already had `zpool labelclear` functionality, so this is mostly
just a diff reduction.

MFC after:	1 month
2016-04-09 20:30:50 +00:00
Glen Barber
d60840138f MFH
Sponsored by:	The FreeBSD Foundation
2016-04-04 23:55:32 +00:00
Andriy Gapon
329e9bafe4 fix zfs set canmount=off on an unmounted filesystem
Previously this operation tried to unmount and remount children.
Also see https://www.illumos.org/issues/6428.

MFC after:	2 weeks
X-Needs-Upstreaming:	illumos
2016-04-03 07:42:13 +00:00
Andriy Gapon
34082fa0f8 zfs receive: -u can be ignored sometimes
When force-receiving a filesystem that was already mounted the re-created
filesystem is mounted despite -u flag.

Also see https://www.illumos.org/issues/6412.

PR:		204705
Tested by:	Vladimir Krstulja <vlad-fbsd@acheronmedia.com>
MFC after:	2 weeks
X-Needs-Upstreaming:	illumos
2016-04-03 07:40:33 +00:00
Alexander Motin
8e713cba91 MFV r297505:
6739 userland version of cv_timedwait_hires() always assumes absolute time

Reviewed by: Paul Dagnelie <pcd@delphix.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Dan McDonald <danmcd@omniti.com>
Reviewed by: Robert Mustacchi <rm@joyent.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author: George Wilson <george.wilson@delphix.com>

illumos/illumos-gate@41c6413cb5
2016-04-02 08:34:15 +00:00
Glen Barber
538354481e MFH
Sponsored by:	The FreeBSD Foundation
2016-03-14 18:54:29 +00:00
Pedro F. Giffuni
273df42911 libdtrace: use calloc(3) instead of malloc(3) when it makes sense.
calloc(3) is faster and occasionally safer than malloc(3) + bzero(3).

In one case, pointed out by Mark[1], this also cleans up a calculation.

Reviewed by:	markj [1]
MFC after:	1 week
2016-03-14 00:34:12 +00:00
Glen Barber
7d536dc855 MFH
Sponsored by:	The FreeBSD Foundation
2016-03-10 21:16:01 +00:00
Bryan Drewery
15c433351f DIRDEPS_BUILD: Connect MK_TESTS.
Sponsored by:	EMC / Isilon Storage Division
2016-03-09 22:46:01 +00:00
Alexander Motin
01812876c1 Missed addition to r296563 to fix newer tools to work with older kernel. 2016-03-09 16:05:13 +00:00
Alexander Motin
fe007faa49 MFV r296540: 4448 zfs diff misprints unicode characters
Reviewed by: Igor Kozhukhov <ikozhukhov@gmail.com>
Reviewed by: Toomas Soome <tsoome@me.com>
Approved by: Matthew Ahrens <mahrens@delphix.com>
Author: Joshua M. Clulow <jmc@joyent.com>

illumos/illumos-gate@b211eb9181
2016-03-08 18:53:00 +00:00
Alexander Motin
0a688b37cd MFV r296538: 6544 incorrect comment in libzfs.h about offline status
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Approved by: Dan McDonald <danmcd@omniti.com>
Author: Gerhard Roethlin <git@the-color-black.net>

illumos/illumos-gate@cb605c4d8a
2016-03-08 18:48:20 +00:00
Alexander Motin
63392cf427 MFV r296536: 6551 cmd/zpool: cleanup gcc warnings
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Andy Stormont <astormont@racktopsystems.com>
Approved by: Robert Mustacchi <rm@joyent.com>

illumos/illumos-gate@b327cd3f3b
2016-03-08 18:39:39 +00:00