Commit Graph

894 Commits

Author SHA1 Message Date
Xin LI
7c2f66bb3b MFV r272493:
Show individual disk capacity when doing zpool list -v.

Illumos issue:
    5147 zpool list -v should show individual disk capacity

MFC after:	1 week
2014-10-04 07:56:50 +00:00
Mark Johnston
41da933c87 Hook up support for userland CTF support in DTrace. This required some
modifications to libproc to support fetching the CTF info for a given file.

With this change, dtrace(1) is able to resolve type info for function and
USDT probe arguments, and function return values. In particular, the args[n]
syntax should now work for referencing arguments of userland probes,
provided that the requisite CTF info is available.

The uctf tests pass if the test programs are compiled with CTF info. The
current infrastructure around the DTrace test suite doesn't support this
yet.

Differential Revision:	https://reviews.freebsd.org/D891
MFC after:		1 month
Relnotes:		yes
Sponsored by:		EMC / Isilon Storage Division
2014-10-03 23:20:37 +00:00
Xin LI
c36047bdb7 Add dependencies to various libraries to libzfs and libzpool.
Submitted by:	sef
2014-10-03 20:36:09 +00:00
Mark Johnston
486de25d46 Have dtrace(1) handle SIGPIPE by cleaning up and exiting. Additionally,
install signal handlers when running in list mode (-l), and acknowledge
interrupts by cleaning up and exiting. This ensures that a command like

$ dtrace -l -P 'pid$target' -p <target PID> | less

won't cause the ptrace(2)d target process to be killed if less(1) exits
before all dtrace output is consumed.

Reported by:		Anton Yuzhaninov <citrin+bsd@citrin.ru>
Differential Revision:	https://reviews.freebsd.org/D880
Reviewed by:		rpaulo
MFC after:		1 month
Sponsored by:		EMC / Isilon Storage Division
2014-10-02 22:33:35 +00:00
Xin LI
456a4e16a6 Revert r272189, the committed version was an old version and breaks build.
Pointy hat to:	delphij
2014-09-27 09:39:19 +00:00
Xin LI
0b03875cc2 Add libuutil to dependency list.
Noticed by:	sef
MFC after:	3 days
2014-09-26 21:46:32 +00:00
Mark Johnston
946497ec23 Fix the description of the -h option.
Reported by:	swills
MFC after:	3 days
2014-09-26 07:01:27 +00:00
Rui Paulo
881417d74f Build plockstat if WITH_PLOCKSTAT is defined. 2014-09-23 18:22:40 +00:00
Mark Johnston
527e468fc9 Remove an incorrect close(2) call that was added in r271413.
Reported by:	Coverity
CID:		1238923
X-MFC-With:	r271413
2014-09-22 21:09:30 +00:00
Mark Johnston
f0de463732 Remove some variables that are no longer used as of r271413.
Reported by:	Coverity
CID:		1238924
X-MFC-With:	r271413
2014-09-22 21:02:38 +00:00
Steven Hartland
137a165dd3 Output boot code warning when zpool upgrade -a is used to add features.
In the case where new features where enabled by a zpool upgrade -a the
boot code warning wasn't output.

Submitted by:	Jan Kokemueller
MFC after:	3 days
2014-09-21 19:31:19 +00:00
Will Andrews
c63a110d6d zfs_setprop_error(): Handle errno value E2BIG.
This errno value is emitted by dsl_props_set_check() in
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_prop.c, and
is used to mean that the property value is too long.  For the record,
the maximum length is ZAP_MAXVALUELEN, which is 8*1024 bytes.

Instead of claiming an unknown error (and abort()ing), provide
something more specific to the scenario involved.  As far as I
can tell, E2BIG is not emitted for any other scenario.

MFC after:	1 week
Sponsored by:	Spectra Logic
Affects:	All ZFS versions starting 27 Feb 2009 (illumos ccba0801)
		This change modified the value returned by
		dsl_props_set_check(), so that it can distinguish between
		a name that's too long and a value that's too long, but
		libzfs was not updated accordingly.
MFSpectraBSD:	r1051499 on 2014/03/28 11:07:59
2014-09-18 14:09:42 +00:00
Will Andrews
abf6a03921 Fix an assert to tolerate spare parents with more than 2 children.
This can occur if a spare is being spared, which would yield three
children: the original pool drive, the previous spare, and the spare
that is replacing it.

MFC after:	1 week
Sponsored by:	Spectra Logic
Affects:	All ZFS versions starting 7 Jun 2006 (illumos 94de1d4c)
MFSpectraBSD:	r668345 on 2013/06/04 17:10:43
2014-09-18 14:02:25 +00:00
Mark Johnston
dab6e14240 Implement a workaround to allow this test program to be compiled with clang.
It seems that if a pragma is used to define a weak alias for a local
function, the pragma must appear after the function is defined.

PR:		193056
MFC after:	1 week
Sponsored by:	EMC / Isilon Storage Division
2014-09-17 04:02:56 +00:00
Mark Johnston
583b13f588 Fix a number of typos and programming errors in the userland CTF tests. It
seems that they would only pass by chance on illumos; on FreeBSD, they still
fail since userland CTF is not yet supported.

MFC after:	1 week
Sponsored by:	EMC / Isilon Storage Division
2014-09-17 03:40:49 +00:00
Mark Johnston
6574b8ed19 Fix some incorrect endianness checks.
MFC after:	1 week
Sponsored by:	EMC / Isilon Storage Division
2014-09-17 00:54:00 +00:00
Mark Johnston
a5ddd965e0 Fix elfdump(1) usage in the userland CTF tests.
PR:		193110
MFC after:	1 week
Sponsored by:	EMC / Isilon Storage Division
2014-09-16 22:54:15 +00:00
Xin LI
8fb2117122 MFV r271516:
Enable debug printf's when ZFS_DEBUG or debug= is set.

Illumos issue:

    5134 if ZFS_DEBUG or debug= is set, libzpool should enable debug prints

MFC after:	2 weeks
2014-09-13 17:30:46 +00:00
Xin LI
e11a25e8d0 MFV r271511:
Use fnvlist_* to make code more readable.

Illumos issue:
    5135 zpool_find_import_cached() can use fnvlist_*

MFC after:	2 weeks
2014-09-13 16:41:39 +00:00
Mark Johnston
8caf8a8d0e Use the linker to perform relocations in the SUNW_dof section rather than
doing them in drti during startup. This fixes a number of problems with
using USDT probes in stripped executables and shared libraries, and with
USDT probes in static functions.

Reviewed by:	rpaulo
MFC after:	1 month
Sponsored by:	EMC / Isilon Storage Division
Phabric: 	D751
2014-09-11 01:04:56 +00:00
Xin LI
36d45b79de MFV r271225:
Iterate through all the children instead of returning error when we hit
the first error.  This makes the error message give more information
rather than just the first device that causes problem.

Illumos issue:
    5118 When verifying or creating a storage pool, error messages only
	 show one device

MFC after:	2 weeks
2014-09-07 13:22:14 +00:00
Xin LI
8b1bab5161 Fix typo.
Submitted by:	Dmitry Morozovsky <marck rinet ru>
MFC after:	3 days
2014-09-07 11:57:08 +00:00
Xin LI
fa4484104c MFV r270197:
Illumos issue:
    5066 remove support for non-ANSI compilation
    5068 Remove SCCSID() macro from <macros.h>

MFC after:	2 weeks
2014-08-22 22:13:36 +00:00
Xin LI
4ddb46f694 Make DTrace stuff compile with C99 standard. 2014-08-22 20:04:51 +00:00
Xin LI
eb86794485 Fix powerpc build:
Chase r270227 and compile lockstat with C99 standard.

Suggested by:	bde
2014-08-22 01:23:38 +00:00
Xin LI
600080d8a5 Include two headers to provide prototype for modfind(2) and kldload(2).
MFC after:	2 weeks
2014-08-21 22:53:14 +00:00
Simon J. Gerraty
ee7b0571c2 Merge head from 7/28 2014-08-19 06:50:54 +00:00
Rui Paulo
40cc45e7c8 Remove the BROKEN_LIBELF section.
This problem was fixed by Kai Wang in 2011.

MFC after:	3 days
2014-08-10 06:56:43 +00:00
Sean Bruno
20c74f8b4a Quiesce a printf warning from clang, %ul -> %lu
Phabric:	https://phabric.freebsd.org/D472
Reviewed by:	mahrens delphij
2014-08-08 18:00:23 +00:00
Mark Johnston
47890cb590 Preserve the errno value of an ioctl before calling free(3). Previously,
errno was very occasionally being clobbered, resulting in a bogus error from
dt_consume() and thus an error from dtrace(1).

MFC after:	2 weeks
2014-08-04 18:52:26 +00:00
Xin LI
de5edb1245 MFV r269426:
Double test device size for ztest(1).

Illumos issue:
    5039 ztest should default to larger device sizes
    Author: Matthew Ahrens <mahrens@delphix.com>

MFC after:	2 weeks
2014-08-02 07:47:52 +00:00
George V. Neville-Neil
e1c97af45e Update shellsnoop to work on FreeBSD.
Contributed by: skreuzer
2014-07-31 23:19:01 +00:00
Xin LI
a3cbca537e MFV r269223:
Change dn->dn_dbufs from linked list to AVL tree.

Illumos issues:
  4873 zvol unmap calls can take a very long time for larger datasets

MFC after:	2 weeks
2014-07-29 08:42:22 +00:00
George V. Neville-Neil
9971b08e4c Update the header printing to match the rest of the output.
Pointed out by: mdf
2014-07-26 20:42:54 +00:00
George V. Neville-Neil
0100ace8a9 Since we cannot yet display the name print the file descriptor so that
it can be looked up manually using procstat(1).
2014-07-26 20:11:36 +00:00
George V. Neville-Neil
d9a499ce10 Remove debugging options from the rwsnoop script. 2014-07-26 19:25:52 +00:00
George V. Neville-Neil
69d22158da Port the rwsnoop DTrace Toolkit script to FreeBSD.
Remove dependency on the Korn Shell.
Remove Zones in favor of Jails.
Remove support (for now) for filename printing.
2014-07-26 19:21:53 +00:00
Xin LI
7e37b1e609 MFV r269010:
Import Illumos changes to address the following Illumos issues:
  4976 zfs should only avoid writing to a failing non-redundant
       top-level vdev
  4978 ztest fails in get_metaslab_refcount()
  4979 extend free space histogram to device and pool
  4980 metaslabs should have a fragmentation metric
  4981 remove fragmented ops vector from block allocator
  4982 space_map object should proactively upgrade when feature
       is enabled
  4984 device selection should use fragmentation metric

MFC after:	2 weeks
2014-07-26 10:20:48 +00:00
Xin LI
e36b1c7478 Diff reduction against Illumos.
MFC after:	2 weeks
2014-07-25 22:58:55 +00:00
Xin LI
7882b61f60 MFV r268848:
Instead of asserting all zio's be properly aligned, only assert
on the logical ones.

Cap uberblocks at 8k, otherwise with ashift=17, there would be
only one uberblock.

This fixes a problem that zdb would trip assert on pools with
ashift >= 0xe (8k).

While there, also change the code so it only attempt to condense
space map unless the uncondensed size consumes greater than
zfs_metaslab_condense_block_threshold blocks.

Illumos issue:
  4958 zdb trips assert on pools with ashift >= 0xe

MFC after:	2 weeks
2014-07-18 20:41:40 +00:00
Xin LI
7079d5877c MFV r268714:
Improve extreme rewind import.

When doing an "extreme rewind" import ("zpool import -XF"), we attempt
to verify all data in the pool, essentially scrubbing the entire pool.
The problem is that spa_load_verify_cb() issues an unbounded number of
concurrent scrub i/os.  This can lead to all of memory being used for
these zio's, wedging the system. Like normal scrub, we need to put a
cap on the number of outstanding i/os, and have the traverse thread
block when we reach this cap.

For this purpose the cap can be very large (10,000) to optimize the
elevator algorithm.  Three kernel tunables have been added:

	vfs.zfs.spa_load_verify_maxinflight
	vfs.zfs.spa_load_verify_metadata
	vfs.zfs.spa_load_verify_data

The latter two tunables controls whether metadata and/or user data
when doing extreme rewind.

Make 'zpool import -T' imply scrub.

Make zpool import -T <txg> accept hexadecimal values for the txg when
prefixed with 0x.

Skip txg's for which there is no uberblock when doing extreme rewind.

Skip reading all user data twice by skipping prefetches when doing
extreme rewinds as we do not access via the ARC.

Illumos issues:
  4970 need controls on i/o issued by zpool import -XF
  4971 zpool import -T should accept hex values
  4972 zpool import -T implies extreme rewind, and thus a scrub
  4973 spa_load_retry retries the same txg
  4974 spa_load_verify() reads all data twice

MFC after:	2 weeks
2014-07-15 22:44:04 +00:00
Xin LI
6066807a42 Bump mdoc date after r268621.
X-MFC-With:	r268621
2014-07-14 17:54:36 +00:00
Steven Hartland
d143a6f700 Don't report non-native block-size pools under zpool status -x
zpool status -x is used to identify pools that are exhibiting
errors or are otherwise unavailable, therefore non-native
block-size pools shouldn't be reported.

Also update man page to clarify other additional conditions
which won't cause a pool to be displayed under zpool status -x.

Sponsored by:	Multiplay
2014-07-14 14:33:03 +00:00
Xin LI
1b174fa1eb MFV r268455:
Use reserved space for ZFS administrative commands.

We reserve 1/2^spa_slop_shift = 1/32 or 3.125% of pool space (or 32MB at
least) for system use.  Most ZPL operations, e.g. write(2), creat(2), will
fail with ENOSPC if we fall below this.

Certain operations, e.g. file removal and most administrative actions,
still permitted until half of the slop space is used.  This would allow
users to use these operations to free up space in the pool when pool is
close to full but half of slop space is still free.

A very restricted set of operations that frees up space or change quota
are always permitted, regardless of the amount of free space.

MFC after:	 2 weeks
2014-07-09 23:14:59 +00:00
Xin LI
b1396c9f98 MFV r268454:
Refresh zpool list for each interval in order to produce fresh
output.

Illumos issue: 4966 zpool list iterator does not update output

MFC after:	 2 weeks
2014-07-09 21:07:20 +00:00
Xin LI
ad9b19c1e8 MFV r268453:
Diff reduction against Illumos.

MFC after:	 2 weeks
2014-07-09 20:57:42 +00:00
Marcel Moolenaar
e7d939bda2 Remove ia64.
This includes:
o   All directories named *ia64*
o   All files named *ia64*
o   All ia64-specific code guarded by __ia64__
o   All ia64-specific makefile logic
o   Mention of ia64 in comments and documentation

This excludes:
o   Everything under contrib/
o   Everything under crypto/
o   sys/xen/interface
o   sys/sys/elf_common.h

Discussed at: BSDcan
2014-07-07 00:27:09 +00:00
Xin LI
9cc8a15b2e MFV r268121:
4924 LZ4 Compression for metadata

illumos/illumos-gate@b8289d24d8

MFC after:	2 weeks
2014-07-01 22:31:09 +00:00
Xin LI
aa882b9048 MFV r268119:
4914 zfs on-disk bookmark structure should be named *_phys_t

illumos/illumos-gate@7802d7bf98

MFC after:	2 weeks
2014-07-01 21:51:30 +00:00
Xin LI
55f6421982 - Fix handling of "new" style of ioctl in compatiblity mode [1];
- Reorganize code and reduce diff from upstream;
 - Improve forward compatibility shims for previous kernel;

Reported by:	sbruno [1]
X-MFC-With:	r268075
2014-07-01 20:57:39 +00:00
Xin LI
be78a8db97 MFV r267570:
4756 metaslab_group_preload() could deadlock

illumos/illumos-gate@30beaff42d

MFC after:	2 weeks
2014-07-01 08:36:56 +00:00
Xin LI
6bab9dd0e4 MFV r267568:
4891 want zdb option to dump all metadata

illumos/illumos-gate@df15e419cb

MFC after:	2 weeks
2014-07-01 08:20:34 +00:00
Xin LI
71eaf0fda7 MFV r267566:
4390 i/o errors when deleting filesystem/zvol can lead to space map corruption

MFC after:	2 weeks
2014-07-01 07:29:42 +00:00
Xin LI
29441ba3fa MFV r267565:
4757 ZFS embedded-data block pointers ("zero block compression")
4913 zfs release should not be subject to space checks

MFC after:	2 weeks
2014-07-01 06:43:15 +00:00
Rui Paulo
a43f0be9fe MFV illumos
4471 DTrace count() with histogram
4472 DTrace full width distribution histograms
4473 DTrace frequency trails

MFC after:	2 weeks
2014-06-26 23:24:59 +00:00
Rui Paulo
8e648814b0 MFV illumos
4474 DTrace Userland CTF Support
4475 DTrace userland Keyword
4476 DTrace tests should be better citizens
4479 pid provider types
4480 dof emulation is missing checks

MFC after:	2 weeks
2014-06-26 23:21:11 +00:00
Rui Paulo
09779aae33 Add stubs for CTF functions which are not yet implemented.
MFC after:	2 weeks
2014-06-26 22:38:06 +00:00
Rui Paulo
b1f9167f94 MFV illumos
4477 DTrace should speak JSON

MFC after:	2 weeks
2014-06-26 21:45:49 +00:00
Rui Paulo
0c2b601953 MFV illumos r266986:
2915 DTrace in a zone should see "cpu", "curpsinfo", et al
2916 DTrace in a zone should be able to access fds[]
2917 DTrace in a zone should have limited provider access

MFC after:	2 weeks
2014-06-26 19:38:16 +00:00
Rui Paulo
dd9b2abed8 Revert r267898. 2014-06-26 17:34:42 +00:00
Rui Paulo
d8e37c5f72 Bring the following change from the illumos-joyent repository:
commit 78e24ab6803bbe11ba37642624e1498ede5b239d
Author: Bryan Cantrill <bryan@joyent.com>
Date:   Thu Oct 31 01:20:54 2013

    OS-1688 DTrace count() with histogram
    OS-2360 DTrace full width distribution histograms
    OS-2361 DTrace frequency trails

MFC after:	2 weeks
2014-06-26 07:06:43 +00:00
Pedro F. Giffuni
fcb8a88918 MFV r258381:
4251 libdtrace leaks open file handles

Illumos commit:		93ed8d0d4b068b95d0bb50d57bb854df462a8485
			(partial)
Reference:
https://www.illumos.org/issues/4251

Discussed with:	Robert Mustacchi
Obtained from:	Illumos
MFC after:	1 week
2014-06-25 17:27:15 +00:00
Joel Dahl
df2d82e003 mdoc: remove superfluous paragraph macros. 2014-06-23 18:40:21 +00:00
Xin LI
51040796d0 MFV r249332 (illumos-gate 14005:55fc53126003)
Illumos ZFS issues:
  3654 zdb should print number of ganged blocks

MFC after:	2 weeks
2014-06-17 08:11:45 +00:00
Pedro F. Giffuni
1bf222cc83 MFV r266988:
Merge from r258379 missed the tests.

4248 dtrace(1M) should never create DOF with empty probes section
4249 Only probes from the first DTrace object file will be included

Illumos Revision:	54a20ab41aadcb81c53e72fc65886e964e9add59

MFC after:	5 days
2014-06-15 16:54:26 +00:00
Xin LI
2bdf7f79bc MFV r266766:
Add a new zfs property, "redundant_metadata" which can have values "all" or
"most".  The default will be "all", which is the current behavior.  When set
to all, ZFS stores an extra copy of all metadata.  If a single on-disk block
is corrupt, at worst a single block of user data (which is recordsize bytes
long) can be lost.

Setting to "most" will cause us to only store 1 copy of level-1 indirect
blocks of user data files.  This can improve performance of random writes,
because less metadata has to be written.  In practice,  at worst about
100 blocks (of recordsize bytes each) of user data can be lost if a single
on-disk block is corrupt.

The exact behavior of which metadata blocks are stored redundantly may change
in future releases.

Illumos issue: 3835 zfs need not store 2 copies of all metadata

MFC after:	2 weeks
2014-05-27 19:46:11 +00:00
Xin LI
f3ce69748f Explicitly link libzfs against libavl as it is done in OpenSolaris
(4543:12bb2876a62e).  Without this, some third party applications
may break because the lack of AVL related symbols.

FreeBSD base system are not affected because the FreeBSD ZFS command
line tools were all linked against libavl and thus hide the underlying
issue.

PR:		java/183081
Tested by:	jkim
MFC after:	3 days
2014-05-22 00:01:31 +00:00
Mark Johnston
a69a8c422a Fix tst.ZeroModuleProbes.d.ksh, which was incorrectly modified in r178534.
Since "BEGIN" is not the name of a module, the test would just hang.

MFC after:	3 days
2014-05-19 20:11:55 +00:00
Simon J. Gerraty
fae50821ae Updated dependencies 2014-05-16 14:09:51 +00:00
Mark Johnston
2f28ceeb34 Bind ip/tcp/udp provider translators and symbols to the same versions as in
illumos, rather than using "1.0" everywhere.

Some of the translators use D functions that are not present in version
1.0 (e.g. inet_ntoa()) which can result in libdtrace crashing when running
scripts that restrict themselves to version 1.0
(e.g. with "-x version=1.0").

MFC after:	1 week
2014-05-14 19:02:00 +00:00
Alexander Motin
7585c68486 Comment out some pointless device open/close around reading device IDs.
FreeBSD ZFS port unlike OpenSolaris does not use device IDs, and does not
implement respective devid_*() fuctions.  It is pointless to open devices
just to close them back immediately.

MFC after:	2 weeks
Sponsored by:	iXsystems, Inc.
2014-05-10 15:21:37 +00:00
Simon J. Gerraty
76b28ad6ab Updated dependencies 2014-05-10 05:16:28 +00:00
Simon J. Gerraty
cc3f4b9965 Merge from head 2014-05-08 23:54:15 +00:00
Alexander Motin
afb8674c79 Import adapted OpenSolaris' thread pool API implementation.
The thread pool is used by libzfs to implement parallel disk scanning.
Without this change our dummy wrapper made `zpool import ZZZ` command to
scan all disks sequentially from the single thread when searching for pools.
This change makes it use two threads per CPU, same as in OpenSolaris.

On system with 200 HDDs this change reduces ZFS pool import time from 35
to 22 seconds.
2014-05-08 16:59:36 +00:00
Mark Johnston
f0736f4cec Re-apply r248644. This fixes an annoying problem which caused dtrace -c to
fail to attach to stripped binaries. With the _r_debug_postinit symbol,
dtrace(1) can now set a breakpoint in the victim process after it has
registered its DOF table(s) with the kernel. r_debug_state cannot be used
for this purpose since it is called before DOF is made available, in which
case dtrace(1) cannot create USDT probes before the program begins
execution.

MFC after:	2 weeks
2014-05-08 03:43:18 +00:00
Warner Losh
c6063d0da8 Use src.opts.mk in preference to bsd.own.mk except where we need stuff
from the latter.
2014-05-06 04:22:01 +00:00
Mark Johnston
604457091d Remove a duplicate definition.
MFC after:	3 days
2014-05-04 03:37:39 +00:00
Simon J. Gerraty
9d2ab4a62d Merge head 2014-04-27 08:13:43 +00:00
Warner Losh
98407b8bc2 Spell NO_PROFILE= as MK_PROFILE=no. 2014-04-25 19:25:26 +00:00
Steven Hartland
fdc8132f2b Silence compiler warning due to missing return in idmap_id_to_numeric_domain_rid 2014-04-24 01:20:43 +00:00
Steven Hartland
d9dab6e243 Eliminated optarg global being used outside of the function which called getopt
MFC after:	2 weeks
2014-04-24 01:12:52 +00:00
Xin LI
f8587167e4 MFV r264829:
3897 zfs filesystem and snapshot limits

MFC after:	2 weeks
2014-04-23 20:29:46 +00:00
Julio Merino
38f0b757fd Add placeholder Kyuafiles for various top-level hierarchies.
This change adds tests/ directories in the source tree to create various
subdirectories in /usr/tests/ and to install placeholder Kyuafiles for
them.

the relevant hierarchies are: cddl, etc, games, gnu and secure.

The reason for this is to simplify the addition of new test programs for
utilities or libraries under any of these directories.  Doing so on a
case by case basis is unnecessary and is quite an obscure process.
2014-04-21 21:39:25 +00:00
Xin LI
613074ec08 MFV r264666:
4374 dn_free_ranges should use range_tree_t

illumos/illumos-gate@bf16b11e8d

MFC after:	2 weeks
2014-04-18 21:15:12 +00:00
Mark Johnston
7df83e08ba Replace a few Solarisisms with their corresponding FreeBSDisms to make a few
printf tests pass.
2014-04-15 02:32:00 +00:00
Mark Johnston
28af9281f7 Use the correct format specifiers for wide characters and strings of wide
characters.

MFC after:	1 week
2014-04-15 02:28:08 +00:00
Xin LI
8621ec029b Take into account when zpool history block grows exceeding 128KB in zpool(8)
and zdb(8) by growing the buffer on demand with a cap of 1GB (specified in
spa_history_create_obj()).

PR:		bin/186574
Submitted by:	Andrew Childs <lorne cons org nz> (with changes)
MFC after:	2 weeks
2014-04-14 18:38:14 +00:00
Warner Losh
3bdf775801 NO_MAN= has been deprecated in favor of MAN= for some time, go ahead
and finish the job. ncurses is now the only Makefile in the tree that
uses it since it wasn't a simple mechanical change, and will be
addressed in a future commit.
2014-04-13 05:21:56 +00:00
Alexander Motin
537650f54d Add property and sysctl to control how ZVOLs are exposed to OS.
New ZFS property volmode and sysctl vfs.zfs.vol.mode allow switching ZVOL
between three modes:
 geom -- existing fully functional behavior (default);
 dev -- exposing volumes only as raw disk device file in devfs;
 none -- not exposing volumes outside ZFS.

The "dev" mode is less functional (can't be partitioned, mounted, etc),
but it is faster, and in some scenarios with untrusted consumers safer.
It can be useful for NAS, VM block storages, etc.
The "none" mode may be convenient for backup servers, etc. that don't
need direct data access.

Due to the way ZVOL is integrated with main ZFS code, those property
and sysctl are checked only during pool import and volume creation.

MFC after:	1 month
Sponsored by:	iXsystems, Inc.
2014-04-05 13:01:44 +00:00
Pedro F. Giffuni
23e4da439c MFV r258379;
4248 dtrace(1M) should never create DOF with empty probes section
4249 Only probes from the first DTrace object file will be included

Illumos Revision:	4a20ab41aadcb81c53e72fc65886e964e9add59

Reference:
https://www.illumos.org/issues/4248
https://www.illumos.org/issues/4249

Obtained from:	Illumos
MFC after:	1 month
2014-04-02 15:32:44 +00:00
Xin LI
49a6711083 MFV r263887:
3993 zpool(1M) and zfs(1M) should support -p for "list" and "get"
4700 "zpool get" doesn't support -H or -o options

MFC after:	2 weeks
2014-03-28 23:12:00 +00:00
Xin LI
2df503a63a MFV 263436-263438:
3947 zpool(1M) references nonexistent zfs-features(5)
  4540 zpool(1M) man page doesn't describe "readonly" property
  3948 zfs sync=default is not accepted
  4611 zfs(1M) still mentions 'send -r' in synopsis
  4415 zpool(1M) man page missing "import -m" description
  4570 Document dedupditto pool property
  4572 Dedup-related documentation additions for zpool and zdb.
  1371 Add -D option description to zpool(1M) manpage
  4571 Add documentation for -T and interval to "zpool list"

MFC after:	2 weeks
2014-03-21 01:32:25 +00:00
Xin LI
cd3ea9e055 Remove unused option -r from zpool.
Submitted by:	Richard Yao <ryao gentoo org>
MFC after:	2 weeks
2014-03-19 23:04:52 +00:00
Alan Somers
2d7c8761fc cddl/contrib/opensolaris/lib/libuutil/common/uu_avl.c
Fix a memory leak in uu_avl_pool_create: pthread_mutex_init without
	a corresponding pthread_mutex_destroy.  It shows up, among other
	places, when doing "zfs list".

MFC after:	3 weeks
Sponsored by:	Spectra Logic Corporation
2014-03-07 23:01:35 +00:00
John-Mark Gurney
1189b87f96 mark that libctf depends upon libz so that if you dlopen libctf, you
don't get:
Undefined symbol "zError"
2014-03-05 23:37:25 +00:00
Mark Johnston
44c25a3d82 When our linker merges .SUNW_dof sections from multiple files, it simply
concatenates the DOF tables into one section. Previously, the USDT init
code in drti.o would only look at the first table in the DOF section; with
this change, it iterates over all the tables, passing each DOF table to
the kernel.

PR:		186821
Submitted by:	Fedor Indutny <fedor@indutny.com>
MFC after:	1 month
2014-03-01 23:09:07 +00:00
Mark Johnston
ae9f1a185c 4478 dtrace_dof_maxsize is far too small
illumos/illumos-gate@d339a29bb4

PR:		187027
MFC after:	1 week
2014-02-28 02:04:41 +00:00
Xin LI
9af4c716d0 MFV r262570:
4626 libzfs memleak in zpool_in_use()

illumos/illumos-gate@fb13f48f1d

MFC after:	2 weeks
2014-02-27 21:50:46 +00:00
Mark Johnston
0339a1c2b4 Move some files that are identical on i386 and amd64 to an x86 subdirectory
rather than keeping duplicate copies.

Discussed with:	avg
MFC after:	1 week
2014-02-27 01:04:35 +00:00
Mark Johnston
33db01542c 1452 DTrace buffer autoscaling should be less violent
illumos/illumos-gate@6fb4854bed

This fixes the tst.resize1.d and tst.resize2.d DTrace tests, which have
been failing since r261122 since they were causing dtrace(1) to attempt to
allocate and use large amounts of memory, and get killed by the OOM killer
as a result.

MFC after:	1 month
2014-02-22 05:18:55 +00:00
Mark Felder
eba76eb552 Fix formatting.
"Manpages should start a new sentence on a new line. This makes it easier
for translators to track changes." -jhb

Approved by:	jhb
MFC after:	3 days
Sponsored by:	SupraNet Communications, Inc
2014-02-17 13:23:49 +00:00
Andriy Gapon
ae2ea19ddc zfs.8: fix garbled options in a sample zfs send -R command line
MFC after:	5 days
2014-02-14 15:21:21 +00:00
Andriy Gapon
b5ef255191 zpool.8: fix typo in option description of labelclear command
MFC after:	5 days
2014-02-14 15:20:49 +00:00
Mark Felder
2d084599da Add caveat to zpool manpage indicating that we do not automatically activate
hot spares. This should be MFC'd to all STABLE branches.

Upon the availability of zfsd, the zpool manpage on relevant branches should
be updated to remove this caveat and document hot spare's reliance on zfsd.

Approved by:	avg
MFC after:	1 week
Sponsored by:	SupraNet Communications
2014-02-11 15:38:29 +00:00
Kai Wang
d8b88105c2 Only declare `bysz' variable under little endian archs. 2014-01-29 09:58:05 +00:00
Kai Wang
a36ae249b8 MFH@261151. 2014-01-25 14:02:02 +00:00
Kai Wang
a52b8956ff Simplify DWARF version check.
Submitted by:	emaste
2014-01-25 11:27:09 +00:00
Andriy Gapon
9a5be1a0d8 dtrace: remove unexplained 16MB limitation from dt_alloc/dt_zalloc
The limitation was introduced in r178556 without any note or comment.
It seems pretty artificial and now it leads to problems like the following:
  $ dtrace -x bufsize=17m -n ...
  dtrace: processing aborted: Memory allocation failure
OpenSolaris and illumos never had this limitation.

Sponsored by:	HybridCluster
2014-01-24 15:04:02 +00:00
Kai Wang
529a53abe2 Let ctfconvert accept DWARF version 3 and 4. 2014-01-22 13:43:54 +00:00
Kai Wang
945a2095f6 MFH@260917. 2014-01-20 19:38:44 +00:00
Kai Wang
d32f014939 Clang 3.4 will sometimes emit DIE for struct/union member before
emitting the DIE for the type of that member. ctfconvert can not
handle this properly and will calculate a wrong member bit offset.
Same struct/union type from different .o file will be treated as
different types when their member bit offsets are different, and
gets added/merged multiple times. This will in turn cause many other
structs/pointers/typedefs that refer to the duplicated struct/union
gets added/merged multiple times and eventually causes numerous
duplicated CTF types in the kernel.debug file.

The simple workaround here is to make use of DW_AT_byte_size attribute
of the member DIE to calculate the bits occupied by the member's type,
without actually resolving the type.
2014-01-20 01:35:14 +00:00
Kai Wang
e3fa6e64ee * Make die_mem_offset() be able to handle DW_AT_data_member_location
attributes generated by Clang 3.4.
* Document how different compilers generate DW_AT_data_member_location
  attributes differently.
* Document the quirks about DW_FORM_data[48].
2014-01-19 13:48:02 +00:00
Andriy Gapon
df217ba9ba zdb -R: do not treat numeric parameters to a flag as more flags
Reviewed by:	Matthew Ahrens <mahrens@delphix.com>
MFC after:	1 week
2014-01-17 10:18:45 +00:00
Kai Wang
aedbdd2c5c We should not set the unnamed DIE's name to "__anon__" since that will
bring back a known issue with DTrace regarding type name
comparison. Instead, we can set the name to an empty string.

Pointed out by:	     avg
2014-01-17 08:44:12 +00:00
Kai Wang
1391789ee2 If function die_name() finds a DIE without a name, set its name to
"__anon__". This hack is used to workaround a issue that compilers
like GCC could generate DW_TAG_base_type DIE without a name.

Note that we didn't need this before because the old libdwarf
internally set all the unnamed DIE's name to "__anon__".
2014-01-16 22:28:33 +00:00
Kai Wang
58fc1c99e4 Convert ctfconvert to use the new libdwarf API. 2014-01-16 21:56:05 +00:00
Andriy Gapon
c38e50a638 zinject must use ioctl(2) compatibility wrapper
MFC after:	8 days
Sponsored by:	HybridCluster
2014-01-16 12:21:21 +00:00
Xin LI
ab0b9f6b30 MFV r260154 + 260182:
4369 implement zfs bookmarks
4368 zfs send filesystems from readonly pools

Illumos/illumos-gate@78f1710053

MFC after:	2 weeks
2014-01-02 07:34:36 +00:00
Xin LI
7353c6e991 MFV r260152:
4208 Typo in zfs_main.c: "posxiuser"

illumos/illumos-gate@f38cb554a5

Note: this is a stripped down version of Illumos change.

MFC after:	2 weeks
2014-01-01 01:23:40 +00:00
Xin LI
f4c8ba8370 MFV r259170:
4370 avoid transmitting holes during zfs send

4371 DMU code clean up

illumos/illumos-gate@43466aae47

NOTE: Make sure the boot code is updated if a zpool upgrade is
done on boot zpool.

MFC after:	2 weeks
2014-01-01 00:45:28 +00:00
Xin LI
f86b34932a MFV r258972:
4373 add block contents print to zstreamdump

illumos/illumos-gate@994fb6b8a9

MFC after:	2 weeks
2013-12-31 21:37:24 +00:00
Xin LI
db2aff5f8b MFV r242733:
3306 zdb should be able to issue reads in parallel
3321 'zpool reopen' command should be documented in the man page
and help message

illumos/illumos-gate@31d7e8fa33

FreeBSD porting notes: the kernel part of this changeset depends
on Solaris buf(9S) interfaces and are not really applicable for
our use.  vdev_disk.c is patched as-is to reduce diverge from
upstream, but vdev_file.c is left intact.

MFC after:	2 weeks
2013-12-31 19:39:15 +00:00
Mark Johnston
2b374230b1 When clearing relocations to __dtrace* symbols, handle both SHT_REL and
SHT_RELA sections properly instead of assuming that the relocation section
is of type SHT_REL.

Submitted by:	Prashanth Kumar <pra_udupi@yahoo.co.in> (original version)
MFC after:	1 month
2013-12-29 19:27:32 +00:00
Xin LI
4e5c0539bb MFV r258384:
2583 Add -p (parsable) option to zfs list

illumos/illumos-gate@43d68d68c1

MFC after:	2 weeks
2013-12-25 00:39:04 +00:00
Xin LI
9b94cdc122 Fix incorrect markup introduced in r259813.
Pointy hat to:	delphij
X-MFC-after:	r259813
2013-12-24 07:27:55 +00:00
Xin LI
1aaa945f67 MFV r258374:
4171 clean up spa_feature_*() interfaces

4172 implement extensible_dataset feature for use by other zpool
features

illumos/illumos-gate@2acef22db7

MFC after:	2 weeks
2013-12-24 07:14:25 +00:00
Xin LI
ec097c1634 MFV r258373:
4168 ztest assertion failure in dbuf_undirty

4169 verbatim import causes zdb to segfa
4170 zhack leaves pool in ACTIVE state

illumos/illumos-gate@7fdd916c47

MFC after:	2 weeks
2013-12-24 06:56:17 +00:00
Alexander Motin
f192c4873d Don't even try to read vdev labels from devices smaller then SPA_MINDEVSIZE
(64MB).  Even if we would find one somehow, ZFS kernel code rejects such
devices.  It is funny to look on attempts to read 4 256K vdev labels from
1.44MB floppy, though it is not very practical and quite slow.
2013-12-10 12:36:44 +00:00
Xin LI
48af354c07 Don't panic when we get ZPOOL_STATUS_NON_NATIVE_ASHIFT
while listing importable pools.

MFC after:	3 days
2013-12-09 18:52:21 +00:00
Joel Dahl
2727e97436 mdoc: remove EOL whitespace. 2013-12-06 21:22:33 +00:00
Mark Johnston
bc968a581f Enable some previously-disabled DTrace tests for umod, ufunc and usym. They
expect the installed ksh binary to be named "ksh", which is not the case
when it's installed on FreeBSD via the shells/ksh93 port. Allow for it to be
"ksh93" as well so that the tests can actually pass.
2013-12-04 01:40:39 +00:00
Mark Johnston
e9be89ce40 The uaddr, ufunc, umod and usym functions all seem to work as expected on
FreeBSD, so stop hiding them behind a "#if defined(sun)".

Reported by:	Prashanth Kumar <pra_udupi@yahoo.co.in>
2013-12-04 01:35:04 +00:00
Mark Johnston
33d8425001 Use mkstemp(3) to create the temporary file used in the FreeBSD-specific
portions of dtrace_program_link().
2013-12-03 03:40:47 +00:00
Andriy Gapon
456a87bb3b MFV r258371,r258372: 4101 metaslab_debug should allow for fine-grained control
4101 metaslab_debug should allow for fine-grained control
4102 space_maps should store more information about themselves
4103 space map object blocksize should be increased
4104 ::spa_space no longer works
4105 removing a mirrored log device results in a leaked object
4106 asynchronously load metaslab

illumos/illumos-gate@0713e232b7

Note that some tunables have been removed and some new tunables have
been added.  Of particular note, FreeBSD-only knob
vfs.zfs.space_map_last_hope is removed as it was a nop for some time now
(after one of the previous merges from upstream).

MFC after:	11 days
Sponsored by:	HybridCluster [merge]
2013-11-28 19:37:22 +00:00
Andriy Gapon
2a4704ab01 MFV r255255: 4045 zfs write throttle & i/o scheduler performance work
illumos/illumos-gate@69962b5647

Please note the following changes:
- zio_ioctl has lost its priority parameter and now TRIM is executed
  with 'now' priority
- some knobs are gone and some new knobs are added; not all of them are
  exposed as tunables / sysctls yet

MFC after:	10 days
Sponsored by:	HybridCluster [merge]
2013-11-26 09:57:14 +00:00
Andriy Gapon
34140e78ab 734 taskq_dispatch_prealloc() desired
943 zio_interrupt ends up calling taskq_dispatch with TQ_SLEEP
illumos/illumos-gate@5aeb94743e

Essentially FreeBSD taskqueues already operate in a mode that
was added to Illumos with taskq_dispatch_ent change.
We even exposed the superior FreeBSD interface as taskq_dispatch_safe.
Now we just rename taskq_dispatch_safe to taskq_dispatch_ent and
struct struct ostask to taskq_ent_t, so that code differences will be
minimal.

After this change sys/cddl/compat/opensolaris/sys/taskq.h header is no
longer needed.

Note that this commit is not an MFV because the upstream change was not
individually committed to the vendor area.

MFC after:	8 days
2013-11-26 09:26:18 +00:00
Justin Hibbits
59d74a351f Use 'int' to store the return value of getopt(), rather than char.
On some architectures (powerpc), char is unsigned by default, which means
comparisons against -1 always fail, so the programs get stuck in an
infinite loop.

MFC after:	1 week
2013-11-20 01:42:29 +00:00
Mark Johnston
ee765c2d34 Don't try to use the 32-bit drti.o unless the data model is explicitly set
to ILP32. Otherwise dtrace -G will attempt to use it on amd64 if it can't
determine which data model to use, which happens when -64 is omitted and
no object files are provided, e.g. with

# dtrace -G -n BEGIN

This would result in a linker error, but now works properly.

Also remove an unnecessary #ifdef.

MFC after:	2 weeks
2013-11-09 04:38:16 +00:00
Sean Bruno
09d325677d Quiesce warning assigning to void * from const ctf_header_t * by explicity casting
to void * before assignment.

Submitted as Illumos issue 4287
2013-11-04 21:32:07 +00:00
Sean Bruno
538c3c05e7 spelling in comments fixup
Submitted by:		Joerg Sonnenberger <joerg@britannica.bec.de>
2013-11-04 19:32:35 +00:00
Sean Bruno
bf10ab6b9c Quiesce warning regarding %llf which has no effect.
Submitted as illumos issue #4284

Reviewed by:	delphij
2013-11-04 16:15:43 +00:00
Sean Bruno
6392bb4f3f This library uses macros to define fprintf behvavior for several object types
The compiler will see the non-string literal arguments to the fprintf calls and
omit warnings for them. Quiese these warnings in contrib code:

cddl/contrib/opensolaris/lib/libnvpair/libnvpair.c:743:12: warning: format
  string is not a string literal (potentially insecure) [-Wformat-security]
  ARENDER(pctl, nvlist_array, nvl, name, val, nelem);
2013-11-03 21:05:44 +00:00
Mark Johnston
5d76604145 If the initial attempt to open /dev/ksyms fails, kldload the ksyms module
and retry.
2013-10-27 16:18:48 +00:00
Mark Johnston
a10f58d076 Convert the lockstat(1) man page to mdoc and make sure that it gets
installed. Additionally, remove Solaris-specific sections and references,
and replace example outputs with output from lockstat on FreeBSD, since
lockstat's output contains stack traces.

This change also removes some examples that don't seem to work properly on
FreeBSD. The examples should be re-added when lockstat is fixed.

Reported by:	avg
MFC after:	1 week
2013-10-27 16:01:11 +00:00
Steven Hartland
818d321f67 Added support for the 'zfs list -t snap' and 'zfs snap' aliases which are
available under Oracle Solaris 11.

This includes an update to the ZFS(8) man page to reflect all the
available alias (snap, umount, and recv).

Initial changes obtained from ZFS On Linux + fixes for man page and cmd
help:
10b75496bb
cf81b00a73

Obtained from:	https://github.com/zfsonlinux/zfs
MFC after:	2 weeks
Sponsored by:	Multiplay
2013-10-23 18:22:27 +00:00
Mark Johnston
e572bc11ec Add a function, memstr, which can be used to convert a buffer of
null-separated strings to a single string. This can be used to print the
full arguments of a process using execsnoop (from the DTrace toolkit) or
with the following one-liner:

dtrace -n 'syscall::execve:return {trace(curpsinfo->pr_psargs);}'

Note that this relies on the process arguments being cached via the struct
proc, which means that it will not work for argvs longer than
kern.ps_arg_cache_limit. However, the following rather non-portable
script can be used to extract any argv at exec time:

fbt::kern_execve:entry
{
    printf("%s", memstr(args[1]->begin_argv, ' ',
        args[1]->begin_envv - args[1]->begin_argv));
}

The debug.dtrace.memstr_max sysctl limits the maximum argument size to
memstr(). Thanks to Brendan Gregg for helpful comments on freebsd-dtrace.

Tested by:	Fabian Keil (earlier version)
MFC after:	2 weeks
2013-10-16 01:39:26 +00:00
Justin Hibbits
30b318b92f Add fasttrap for PowerPC. This is the last piece of the dtrace/ppc puzzle.
It's incomplete, it doesn't contain full instruction emulation, but it should be
sufficient for most cases.

MFC after:	1 month
2013-10-15 15:00:29 +00:00
Simon J. Gerraty
3caf0790a8 Merge head@256284 2013-10-13 02:35:19 +00:00
Simon J. Gerraty
34b33809b7 Updated dependencies 2013-10-13 00:24:00 +00:00
Mark Johnston
3f3c4e0b4a Convert the dtrace(1) man page to mdoc and fix up some aspects of it that
don't make sense on FreeBSD. In particular,

- remove the ATTRIBUTES section,
- remove references to the Solaris Dynamic Tracing Guide, except in the
  SEE ALSO section,
- update the description of the -A option for FreeBSD's implementation,
- remove references to Solaris-specific programs and configuration files,
  and replace them with FreeBSD equivalents where possible.

The content has not changed aside from this.

Approved by:	re (joel)
MFC after:	1 week
2013-10-10 03:50:23 +00:00
Robert Millan
8a675a6396 Fix implicit declaration of jail_getid()
Approved by:	re
2013-10-07 14:22:19 +00:00
Mark Johnston
8298c17c6c Add a separate translator for headers passed to the TCP probes in the
input path. These probes get some of the fields in host order, whereas the
output probes get them in network order, so a single translator isn't
enough. This workaround ensures that the problem is essentially invisble
to users: none of the probe arguments or their fields have changed.

Approved by:	re (hrs)
2013-10-02 17:14:12 +00:00
Xin LI
253aa02fc3 MFV r254750:
Add support of Illumos dumps on zvol over RAID-Z.

Note that this only adds the features.  FreeBSD would
still need more work to support dumping on zvols.

Illumos ZFS issues:
  2932 support crash dumps to raidz, etc. pools

MFC after:	1 month
Approved by:	re (ZFS blanket)
2013-09-21 00:17:26 +00:00
Mark Johnston
9feec372db Use the address of the inpcb rather than the tcpcb to identify TCP
connections. This keeps the tcp provider consistent with the other network
providers.

Approved by:	re (delphij)
2013-09-15 21:38:46 +00:00
Simon J. Gerraty
d466a5b069 Merge head 2013-09-11 18:16:18 +00:00
Xin LI
e8de677c74 MFV r247844 (illumos-gate 13975:ef6409bc370f)
Illumos ZFS issues:
  3582 zfs_delay() should support a variable resolution
  3584 DTrace sdt probes for ZFS txg states

Provide a compatibility shim for Solaris's cv_timedwait_hires
to help aid future porting.

Approved by:	re (ZFS blanket)
2013-09-10 01:46:47 +00:00
Simon J. Gerraty
d1d0158641 Merge from head 2013-09-05 20:18:59 +00:00
Will Andrews
5a5347c3f1 Build all ZFS testing & debugging tools with -g.
These programs and everything using libzpool rely on the embedded asserts to
verify the correctness of operations.  Given that, the core dumps would be
useless without debug symbols.
2013-08-27 04:01:31 +00:00
Pedro F. Giffuni
d69c286983 Merge various CTF fixes from illumos
2942 CTF tools need to handle files which legitimately lack data
2978 ctfconvert still needs to ignore legitimately dataless files on SPARC

Illumos Revisions:	13745:6b3106b4250f
			13754:7231b684c18b

Reference:

https://www.illumos.org/issues/2942
https://www.illumos.org/issues/2978

MFC after:	3 weeks
2013-08-26 22:29:42 +00:00
Mark Johnston
57f6086735 Implement the ip, tcp, and udp DTrace providers. The probe definitions use
dynamic translation so that their arguments match the definitions for
these providers in Solaris and illumos. Thus, existing scripts for these
providers should work unmodified on FreeBSD.

Tested by:	gnn, hiren
MFC after:	1 month
2013-08-25 21:54:41 +00:00
Xin LI
83b6a67e66 MFV r254751:
Don't treat the parameter as a number (pool GUID) when there is
error converting it from string, instead, treat it as the pool
name.

Illumos ZFS issues:
  1765 assert triggered in libzfs_import.c trying to import pool
       name beginning with a number
2013-08-24 00:54:47 +00:00
Xin LI
b333d51d8f MFV r254748:
Fix memory leak in libzfs's iter_dependents_cb().

Illumos ZFS issues:
  4061 libzfs: memory leak in iter_dependents_cb()
2013-08-24 00:29:34 +00:00
Xin LI
cc16dea626 MFV r254746:
To quote original Illumos ticket:

libctf thinks that any ELF file containing more than 65536 sections is
corrupt, because it doesn't understand the SHN_XINDEX magic.

Illumos DTrace issues:
  4005 libctf can't deal with extended sections
2013-08-23 23:58:56 +00:00
Xin LI
3f0164abf3 MFV r254422:
Illumos DTrace issues:
  3089 want ::typedef
  3094 libctf should support removing a dynamic type
  3095 libctf does not validate arrays correctly
  3096 libctf does not validate function types correctly
2013-08-23 23:21:24 +00:00
Justin T. Gibbs
439d30d121 Enhance the ZFS vdev layer to maintain both a logical and a physical
minimum allocation size for devices.  Use this information to
automatically increase ZFS's minimum allocation size for new top-level
vdevs to a value that more closely matches the optimum device
allocation size.

Use GEOM's stripesize attribute, if set, as the physical sector
size of the GEOM.

Calculate the minimum blocksize of each metaslab class.  Use the
calculated value instead of SPA_MINBLOCKSIZE (512b) when determining
the likelyhood of compression yeilding a reduction in physical space
usage.

Report devices with sub-optimal block size configuration in "zpool
status".  Also properly fail attempts to attach devices with a
logical block size greater than 8kB, since this will cause corruption
to ZFS's label area.

Sponsored by:	Spectra Logic Corporaion
MFC after:	2 weeks

Background
==========
Many modern devices use physical allocation units that are much
larger than the minimum logical allocation size accessible by
external commands.  Two prevalent examples of this are 512e disk
drives (512b logical sector, 4K physical sector) and flash devices
(512b logical sector, 4K or larger allocation block size, and 128k
or larger erase block size).  Operations that modify less than the
physical sector size result in a costly read-modify-write or garbage
collection sequence on these devices.

Simply exporting the true physical sector of the device to ZFS would
yield optimal performance, but has two serious drawbacks:

1) Existing pools created with devices that have different logical
   and physical block sizes, but were configured to use the logical
   block size (e.g. because the OS version used for pool construction
   reported the logical block size instead of the physical block
   size) will suddenly find that the vdev allocation size has
   increased.  This can be easily tolerated for active members of
   the array, but ZFS would prevent replacement of a vdev with
   another identical device because it now appears that the smaller
   allocation size required by the pool is not supported by the new
   device.

2) The device's physical block size may be too large to be supported
   by ZFS.  The optimal allocation size for the vdev may be quite
   large.  For example, a RAID controller may export a vdev that
   requires read-modify-write cycles unless accessed using 64k
   aligned/sized requests.  ZFS currently has an 8k minimum block
   size limit.

Reporting both the logical and physical allocation sizes for vdevs
solves these problems.  A device may be used so long as the logical
block size is compatible with the configuration.  By comparing the
logical and physical block sizes, new configurations can be optimized
and administrators can be notified of any existing pools that are
sub-optimal.

sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/spa.h:
	Add the SPA_ASHIFT constant.  ZFS currently has a hard upper
	limit of 13 (8k) for ashift and this constant is used to
	both document and enforce this limit.

sys/cddl/contrib/opensolaris/uts/common/sys/fs/zfs.h:
	Add the VDEV_AUX_ASHIFT_TOO_BIG error code.

	Add fields for exporting the configured, logical, and
	physical ashift to the vdev_stat_t structure.

	Add VDEV_STAT_VALID() macro which can be used to verify the
	presence of required vdev_stat_t fields in nvlist data.

sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c:
	Provide a SYSCTL_PROC handler for "max_auto_ashift".  Since
	the limit is only referenced long after boot when a create
	operation occurs, there's no compelling need for it to be
	a boot time configurable tunable.  This also allows the
	validation code for the max_auto_ashift value to be contained
	within the sysctl handler.

	Populate the new fields in the vdev_stat_t structure.

	Fail vdev opens if the vdev reports an ashift larger than
	SPA_MAXASHIFT.

	Propogate vdev_logical_ashift and vdev_physical_ashift between
	child and parent vdevs as is done for vdev_ashift.

	In vdev_open(), restore code that fails opens for devices
	where vdev_ashift grows.  This can only happen now if the
	device's logical ashift grows, which means it really isn't
	safe to use the device.

sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev_impl.h:
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c:
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_file.c:
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c:
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_mirror.c:
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_missing.c:
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_raidz.c:
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_root.c:
	Update the vdev_open() API so that both logical (what was
	just ashift before) and physical ashift are reported.

sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev_impl.h:
	Add two new fields, vdev_physical_ashift and vdev_logical_ashift,
	to vdev_t.

sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c:
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_config.c:
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c:
	Add vdev_ashift_optimize().  Call it anytime a new top-level
	vdev is allocated.

cddl/contrib/opensolaris/cmd/zpool/zpool_main.c:
	Add text for the VDEV_AUX_ASHIFT_TOO_BIG error.

	For each sub-optimally configured leaf vdev, report configured
	and native block sizes.

cddl/contrib/opensolaris/cmd/zpool/zpool_main.c:
cddl/contrib/opensolaris/lib/libzfs/common/libzfs.h:
cddl/contrib/opensolaris/lib/libzfs/common/libzfs_status.c:
	Introduce a new zpool status: ZPOOL_STATUS_NON_NATIVE_ASHIFT.
	This status is reported on healthy pools containing vdevs
	configured to use a block size smaller than their reported
	physical block size.

cddl/contrib/opensolaris/lib/libzfs/common/libzfs_status.c:
	Update find_vdev_problem() and supporting functions to
	provide the full vdev_stat_t structure to problem checking
	routines, and to allow decent into replacing vdevs.

	Add a vdev_non_native_ashift() validator which is used on
	the full vdev tree to check for ZPOOL_STATUS_NON_NATIVE_ASHIFT.

cddl/contrib/opensolaris/lib/libzpool/common/kernel.c:
cddl/contrib/opensolaris/lib/libzpool/common/sys/zfs_context.h:
	Enhance sysctl userland stubs now that a SYSCTL_PROC handler
	is used in vdev.c.

sys/cddl/contrib/opensolaris/uts/common/fs/zfs/metaslab.c:
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/metaslab_impl.h:
	When the group membership of a metaslab class changes (i.e.
	when a vdev is added or removed from a pool), walk the group
	list to determine the smallest block size currently available
	and record this in the metaslab class.

sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/metaslab.h:
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/metaslab.c:
	Add the metaslab_class_get_minblocksize() accessor.

sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zio_compress.h:
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio_compress.c:
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:
	In zio_compress_data(), take the minimum blocksize as an
	input parameter instead of assuming SPA_MINBLOCKSIZE.

sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:
	In l2arc_compress_buf(), pass SPA_MINBLOCKSIZE as the minimum
	blocksize of the device.  The l2arc code performs has it's own
	code for deciding if compression is worth while, so this
	effectively disables zio_compress_data() from second guessing
	the original decision.

sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:
	In zio_write_bp_init(), use the minimum blocksize of the
	normal metaslab class when compressing data.
2013-08-21 04:10:24 +00:00
Xin LI
2640fb93f5 MFV r254421:
Illumos ZFS issues:
  3996 want a libzfs_core API to rollback to latest snapshot
2013-08-21 00:04:31 +00:00
Rui Paulo
f6ddb56a60 Load the dtraceall module if /dev/dtrace/dtrace doesn't exist.
MFC after:	3 days
2013-08-10 23:17:09 +00:00
Xin LI
43667c1f68 MFV r254079:
Illumos ZFS issues:
  3957 ztest should update the cachefile before killing itself
  3958 multiple scans can lead to partial resilvering
  3959 ddt entries are not always resilvered
  3960 dsl_scan can skip over dedup-ed blocks if
       physical birth != logical birth
  3961 freed gang blocks are not resilvered and can cause pool to suspend
  3962 ztest should print out zfs debug buffer before exiting
2013-08-08 23:38:31 +00:00
Xin LI
9d2f243aa6 MFV r254071:
Fix a regression introduced by fix for Illumos bug #3834.  Quote from
Matthew Ahrens on the Illumos issue:

ztest fails this assertion because ztest_dmu_read_write() does
        dmu_tx_hold_free(tx, bigobj, bigoff, bigsize);
and then
    dmu_object_set_checksum(os, bigobj,
        (enum zio_checksum)ztest_random_dsl_prop(ZFS_PROP_CHECKSUM), tx);

If the region to free is past the end of the file, the DMU assumes that there
will be nothing to do for this object.  However, ztest does set_checksum(),
which must modify the dnode.  The fix is for ztest to also call

    dmu_tx_hold_bonus(tx, bigobj);

so we can account for the dirty data associated with setting the checksum

Illumos ZFS issues:
  3955 ztest failure: assertion refcount_count(&tx->tx_space_written)
         + delta <= tx->tx_space_towrite
2013-08-07 22:21:00 +00:00
Xin LI
4f7b34578b MFV r254070:
Merge vendor bugfix for ZFS test suite that triggers false positives.

Illumos ZFS issues:
  3949 ztest fault injection should avoid resilvering devices
  3950 ztest: deadman fires when we're doing a scan
  3951 ztest hang when running dedup test
  3952 ztest: ztest_reguid test and ztest_fault_inject don't place nice together
2013-08-07 21:16:14 +00:00
Robert Millan
852bec83ed Fix implicit declaration of warnx(). 2013-08-04 16:25:46 +00:00
Xin LI
d637247e1f MFV r253781 + r253871:
Illumos ZFS issues:
  3894 zfs should not allow snapshot of inconsistent dataset

MFC after:	2 weeks
2013-07-30 21:02:09 +00:00
Steven Hartland
b79849e50c MFV r253784:
Fix zfs send -D hang after processing requiring a CTRL+C to interrupt due to
pthread_join prior to fd close.

This was introduced by r251646 (MFV r251644)

Illumos ZFS issue:
  3909 "zfs send -D" does not work

MFC after:	1 day
2013-07-30 20:45:27 +00:00
Pedro F. Giffuni
41840d7587 DTrace: re-apply r249426 now that the underlying issues have been solved.
Merge change from illumos:

3519 DTrace fails to resolve const types from fbt
3520 dtrace internal error -- token type 316 is not a valid D
     compilation token
3521 clean up dtrace unit tests

Illumos Revision:	e98f46c

Reference:
https://www.illumos.org/issues/3519
https://www.illumos.org/issues/3520
https://www.illumos.org/issues/3521

Tested by:	Fabian Keil
Obtained from:	Illumos
MFC after:	1 month
2013-07-28 01:02:17 +00:00
Pedro F. Giffuni
a98ff31738 DTrace: re-merge remainder of r249367 (original from Illumos).
Bring back some important fixes from Illumos:

3022 DTrace: keys should not affect the sort order when sorting by value
3023 it should be possible to dereference dynamic variables
3024 D integer narrowing needs some work

We particularly avoid the LD_NOLAZYLOAD changes that Illumos made
as those don't apply to FreeBSD and were causing problems in
interactive mode.

Illumos Revision:	13758:23432da34147

Reference:

https://www.illumos.org/issues/3022
https://www.illumos.org/issues/3023
https://www.illumos.org/issues/3024

MFC after:	1 month
Tested by:	markj
2013-07-28 00:45:20 +00:00
Mark Johnston
6e660824a8 Use kern_ioctl() rather than ioctl() for testing the FBT provider, since the
latter doesn't exist in FreeBSD. All the tests under fbtprovider pass now.
2013-07-27 21:31:48 +00:00
Pedro F. Giffuni
e75a1c78e0 Style issue in r253661.
Pointed out by:	avg
MFC after:	1 month
2013-07-26 14:37:23 +00:00
Pedro F. Giffuni
c7cdfecc89 Fix a segfault in ctfmerge due to a bug in gcc.
GCC can generate bogus dwarf attributes with DW_AT_byte_size
set to 0xFFFFFFFF.
The issue was originaly detected in NetBSD but it has been
adapted for portability and to avoid compiler warnings.

Reference:
https://www.illumos.org/issues/3776

Obtained from:	NetBSD
MFC after:	1 month
2013-07-26 00:28:19 +00:00
Xin LI
c92bc5e996 Manually merge part of vendor import r238583 from Illumos.
Illumos changeset: 13680:2bd022a765e2
Illumos ZFS issue:

    2671 zpool import should not fail if vdev ashift has increased

MFC after:	3 days
2013-07-18 00:22:42 +00:00
Martin Matuska
566081efdc Fix misleading or remove irrelevant illumos messages and manpage references
in the zfs command.

PR:		bin/178996
Submitted by:	Peter Schaefer <peter.schaefer@wilhelmheinrichs.de>
MFC after:	3 days
2013-07-04 22:26:38 +00:00
Xin LI
e33806a54a MFV r252215:
Restore a previous behavior before r251646, where when destructing
ZFS snapshot, the ioctl would return ENOENT when it hit any of
them in the errlist (the new behavior was only return ENOENT when
all returns error).

Illumos ZFS issues:
  3829 fix for 3740 changed behavior of zfs destroy/hold/release ioctl

MFC after:	1 week
2013-06-25 22:14:32 +00:00
Xin LI
302dc71948 Diff reduction against Illumos, no real change to code itself.
This marks vendor branch revision 252213 as merged, the actual code was
committed in r245479.

MFC after:	1 week
2013-06-25 21:51:52 +00:00
Steven Hartland
b7f3763e6c Fixed ZFS zpool freeze (debug command) not processing due to invalid ioctl call syntax.
MFC after:	1 week
2013-06-21 15:30:46 +00:00
Xin LI
9625321547 MFV r251644:
Poor ZFS send / receive performance due to snapshot
hold / release processing (by smh@)

Illumos ZFS issues:
  3740 Poor ZFS send / receive performance due to snapshot
       hold / release processing

MFC after:      2 weeks
2013-06-12 07:07:06 +00:00
Xin LI
3b245f3ee1 MFV r251624:
txg commit callbacks don't work

Illumos ZFS issues:
  3747 txg commit callbacks don't work

MFC after:      2 weeks
2013-06-11 19:29:31 +00:00
Xin LI
23fe4c8a55 MFV r251623:
zpool create should treat -O mountpoint and -m the same

Illumos ZFS issues:
  3745 zpool create should treat -O mountpoint and -m the same

MFC after:      2 weeks
2013-06-11 19:25:49 +00:00
Xin LI
4acaabea05 MFV r251619:
ZFS needs better comments.

Illumos ZFS issues:
  3741 zfs needs better comments

MFC after:      2 weeks
2013-06-11 19:02:36 +00:00
Mark Johnston
18161786c6 Port the SDT test now that it's possible to create SDT probes that take
seven arguments.

The original test uses Solaris' uadmin system call to trigger the test
probe; this change adds a sysctl to the dtrace_test module and gets the test
program to trigger the test probe via the sysctl handler.

The test is currently failing on amd64 because of some bugs in the way that
probe arguments beyond the first five are obtained - these bugs will be
fixed in a separate change.
2013-06-02 00:33:36 +00:00
Mark Johnston
f9a93b641a Remove a block of code that was not intended to be part of the previous
revision.

X-MFC with:	r250812
2013-05-20 02:10:18 +00:00
Mark Johnston
ba6cafe265 Re-introduce another part of r249367. This commit fixes a register leak in
dt_cg_ptrsize() and generally cleans up some of the error handling around
register allocation.

This change corresponds to part of illumos-gate commit e5803b76927480:
  3025 register leak in D code generation

Reviewed by:	pfg
Obtained from:	illumos
MFC after:	1 month
2013-05-19 17:14:36 +00:00
Mark Johnston
fa0deba90f Convert a couple of helper scripts used to test the ip provider to work on
FreeBSD. In the IPv6 case, try each interface before returning an error;
each IPv6-enabled interface will have a link-local address even if the link
isn't up.

MFC after:	1 week
2013-05-15 22:56:24 +00:00
Mark Johnston
d02e5a206d head -n doesn't take negative arguments on FreeBSD, so instead use sed '$d'
to remove the last line of a file.
2013-05-12 16:29:09 +00:00
Mark Johnston
09e6105ff4 Bring back part of r249367 by adding DTrace's temporal option, which allows
users to guarantee that the output of DTrace scripts will be time-ordered.
This option is enabled by adding the line

  #pragma D option temporal

to the beginning of a script, or by adding '-x temporal' to the arguments of
dtrace(1).

This change fixes a bug in the original port of the temporal option. This
bug was causing some assertions to fail, so they had been disabled; in this
revision the assertions are working properly and are enabled.

The DTrace version number has been bumped from 1.9.0 to 1.9.1 to reflect
the language change that's being introduced.

This change corresponds to part of illumos-gate commit e5803b76927480:
  3021 option for time-ordered output from dtrace(1M)

Reviewed by:	pfg
Obtained from:	illumos
MFC after:	1 month
2013-05-12 16:26:33 +00:00
Brooks Davis
168ba1089a Work around the implementation of LIBRARIES_ONLY. It causes drti.o to
not be installed in /usr/lib32 on systems with compat-32 support.

This fix has two parts.  First, the build is forced by linking drti.o
into a dummy internal library.  Second, the object file is installed
manually in the LIBRARIES_ONLY case.

MFC after:	3 days
2013-05-07 22:05:57 +00:00
Andriy Gapon
5d33cbbf9d revert r248644 because of the regression for usdt probes
USDT probes are advertised to kernel by initialization code with
atrribute((constructor))).  It seems that on Solaris the .init-ish code
of the main object is executed before RD_PREINIT point is hit.  On
FreeBSD that is not the case.  And because on FreeBSD there is no other
well-defined point between RD_PREINIT and main() we have to parse a
DTrace script when main is hit, for time being.

A footnote: currently we actually post RD_POSTINIT event, but that's a
bug because the event is triggered by hitting r_debug_state which
happens before any init code is executed.

Reported by:	markj
2013-04-25 07:04:56 +00:00
Martin Matuska
7f168e5ddb Respect the enoent_ok flag if reporting error for holding an non-existing
snapshot.

Related illumos ZFS issue:
  3699 zfs hold or release of a non-existent snapshot does not output error

Reported by:	Steven Hartland <smh@FreeBSD.org>
MFC after:	3 days
2013-04-25 06:55:57 +00:00
Ed Schouten
c0794a841d Fix -Wmissing-variable-declarations compiler warnings.
References:
https://www.illumos.org/issues/3700
2013-04-19 19:38:39 +00:00
Pedro F. Giffuni
195ce43708 DTrace: NFS translators should be split into client/server pieces
Belatedly update the provider with changes from OpenSolaris:

PSARC 2008/050 DTrace NFS v3 Provider
6696397 NFS v3 provider reports all UDP clients as 0.0.0.0
6711844 assert: vp->v_shrlocks

Obtained from:	Illumos
2013-04-18 01:20:08 +00:00
Pedro F. Giffuni
2e654ff9df DTrace: Revert r249426
This change actually depends on r249367 which had to be reverted

Pointy Hat:	pfg
2013-04-17 02:40:07 +00:00
Pedro F. Giffuni
03836978be DTrace: Revert r249367
The following change from illumos brought caused DTrace to
pause in an interactive environment:

3026 libdtrace should set LD_NOLAZYLOAD=1 to help the pid provider

This was not detected during testing because it doesn't
affect scripts.

We shouldn't be changing the environment, especially since the
LD_NOLAZYLOAD option doesn't apply to our (GNU) ld.
Unfortunately the change from upstream was made in such a way
that it is very difficult to separate this change from the
others so, at least for now, it's better to just revert
everything.

Reference:
https://www.illumos.org/issues/3026

Reported by:	Navdeep Parhar and Mark Johnston
2013-04-17 02:20:17 +00:00
Pedro F. Giffuni
acc929508b DTrace: print() should try to resolve function pointers
Merge changes from illumos:

3675 DTrace print() should try to resolve function pointers
3676 dt_print_enum hardcodes a value of zero

Illumos Revision:	b1fa6326238973aeaf12c34fcda75985b6c06be1

Reference:
https://www.illumos.org/issues/3675
https://www.illumos.org/issues/3676

Obtained from:	Illumos
MFC after:	1 month
2013-04-16 19:39:27 +00:00
Pawel Jakub Dawidek
fe3fcf7b3a Correct error message.
Reported by:	Dirk Engling <erdgeist@erdgeist.org>
2013-04-16 12:31:16 +00:00
Pedro F. Giffuni
98aff789ce DTrace: NFS translators should be split into client/server pieces
Merge change from illumos:

1731 DTrace NFS translators should be split into client/server pieces

Illumos Revision:	13523:6763769941d2

This code seems to be currently unused on FreeBSD.

Reference:
https://www.illumos.org/issues/1731

Obtained from:	Illumos
MFC after:	1 week
2013-04-15 20:16:31 +00:00
Simon J. Gerraty
89e0f621b2 New/updated dependencies 2013-04-14 01:22:39 +00:00
Simon J. Gerraty
69e6d7b75e sync from head 2013-04-12 20:48:55 +00:00
Pedro F. Giffuni
d184218c18 Dtrace: resolve const types from fbt and other fixes.
Merge change from illumos:

3519 DTrace fails to resolve const types from fbt
3520 dtrace internal error -- token type 316 is not a valid D
     compilation token
3521 clean up dtrace unit tests

Illumos Revision:	e98f46c

Reference:
https://www.illumos.org/issues/3519
https://www.illumos.org/issues/3520
https://www.illumos.org/issues/3521

Tested by:	Fabian Keil
Obtained from:	Illumos
MFC after:	1 month
2013-04-12 20:10:27 +00:00
Pedro F. Giffuni
ddd5b8e9b4 DTrace: option for time-ordered output
Merge changes from illumos:

3021 option for time-ordered output from dtrace(1M)
3022 DTrace: keys should not affect the sort order when sorting by value
3023 it should be possible to dereference dynamic variables
3024 D integer narrowing needs some work
3025 register leak in D code generation
3026 libdtrace should set LD_NOLAZYLOAD=1 to help the pid provider

This brings yet another feature implemented in upstream DTrace.
A complete description is available here:
http://dtrace.org/blogs/ahl/2012/07/28/my-new-dtrace-favorite/

This change bumps the DT_VERS_* number to 1.9.1 in
accordance to what is done in illumos.

This change was somewhat complicated because upstream is mixed many
changes in an individual commit and some of the tests don't really
apply to us.

There are also appear to be differences in timestamping with Solaris
so we had to workaround some assertions making sure no regression
happened.

Special thanks to Fabian Keil for changes and testing.

Illumos Revisions:	13758:23432da34147

Reference:
https://www.illumos.org/issues/3021
https://www.illumos.org/issues/3022
https://www.illumos.org/issues/3023
https://www.illumos.org/issues/3024
https://www.illumos.org/issues/3025
https://www.illumos.org/issues/1694

Tested by:	Fabian Keil
Obtained from:	Illumos
MFC after:	1 months
2013-04-11 16:24:36 +00:00
Martin Matuska
12eef24d20 Fix libzfs to report error instead of returning zero if trying to hold or
release a non-existing snapshot of a existing dataset. In recursive case
error is reported if no snapshots with the requested name have been found.

Problem and proposed solution reported to illumos:
  3699 zfs hold or release of a non-existent snapshot does not output error

MFC after:	8 days
2013-04-11 07:49:16 +00:00
Martin Matuska
83b4af1142 ZFS expects a copyout of zfs_cmd_t on an ioctl error. Our sys_ioctl()
doesn't copyout in this case.

To solve this issue a new struct zfs_iocparm_t is introduced consisting of:
- zfs_ioctl_version (future backwards compatibility purposes)
- user space pointer to zfs_cmd_t (copyin and copyout)
- size of zfs_cmd_t (verification purposes)

The copyin and copyout of zfs_cmd_t is now done the illumos (vendor) way
what makes porting of new changes easier and ensures correct behavior if
returning an error.

MFC after:	10 days
2013-04-09 22:27:44 +00:00
Martin Matuska
bb60e990e9 MFV r249185:
Allow zdb to output a histogram of compressed block sizes.

Illumos ZFS issues:
  3641 want a histogram of compressed block sizes

MFC after:	3 weeks
2013-04-06 17:18:48 +00:00
Martin Matuska
f1b5c26470 MFV r248217:
Merge change from vendor to reduce diff only.
ZFS dtrace probes are not supported on FreeBSD yet.

Illumos ZFS issues:
  3598 want to dtrace when errors are generated in zfs

MFC after:	3 weeks
2013-04-06 10:39:38 +00:00
Dimitry Andric
51ff6adc94 Follow up to r247960 and rr247960 by also amending ctfmerge. For the
only other case where STT_FILE symbols are used, in symit_next() in
cddl/contrib/opensolaris/tools/ctf/cvt/input.c, save the basename of the
symbol, instead of the full pathname.

Reported by:	avg
Tested by:	avg, jimharris
MFC after:	1 week
2013-04-01 21:16:32 +00:00
Pedro F. Giffuni
9f4c7ba460 Dtrace: enablings on defunct providers prevent providers from unregistering
Merge change from illumos:

1368 enablings on defunct providers prevent providers from unregistering

We try to address some underlying differences between the Solaris
and FreeBSD implementations: dtrace_attach() / dtrace_detach() are
currently unimplemented in FreeBSD but the new code from illumos
makes use of taskq so some adaptations were made to dtrace_open()
and dtrace_close() to handle them appropriately.

Illumos Revision:	r13430:8e6add739e38

Reference:
https://www.illumos.org/issues/1368

Reviewed by:	gnn
Tested by:	Fabian Keil
Obtained from:	Illumos
MFC after:	3 weeks
2013-04-01 19:13:46 +00:00
George V. Neville-Neil
03a9b7c085 Commit a patch that fixes a problem in the #pragma statement when searching
for and loading dependent modules.  This addresses a bug seen with
io.d where it was being doubly included.

PR:		171678
Submitted by:	 Mark Johnston
MFC after:	2 weeks
2013-03-28 20:31:03 +00:00
George V. Neville-Neil
02b3906712 Revert previous change in favor of a fix to the actual dtrace
libraries that addresses a #pragma issue.
2013-03-28 20:22:43 +00:00
George V. Neville-Neil
3e3d9dcd39 Remove dependency code that caused a double inclusion.
Pointed out by: rpaulo and others
2013-03-28 20:12:46 +00:00
Pedro F. Giffuni
f5678b698a Dtrace: dtrace.c erroneously checks for memory alignment on amd64.
Merge change from illumos:

3511 dtrace.c erroneously checks for memory alignment on amd64

Illumos Revision:	c93cc65

Reference:
https://www.illumos.org/issues/3511

Obtained from:	Illumos
MFC after:	3 weeks
2013-03-26 20:17:08 +00:00
Pedro F. Giffuni
5472787377 Dtrace: Add SUN MDB-like type-aware print() action.
Merge change from illumos:

1694 Add type-aware print() action

This is a very nice feature implemented in upstream Dtrace.
A complete description is available here:
http://dtrace.org/blogs/eschrock/2011/10/26/your-mdb-fell-into-my-dtrace/

This change bumps the DT_VERS_* number to 1.9.0 in
accordance to what is done in illumos.

While here also include some minor cleanups to ease further merging
and appease clang with a fix by Fabian Keil.

Illumos Revisions:	13501:c3a7090dbc16
			13483:f413e6c5d297

Reference:
https://www.illumos.org/issues/1560
https://www.illumos.org/issues/1694

Tested by:	Fabian Keil
Obtained from:	Illumos
MFC after:	1 month
2013-03-25 20:38:09 +00:00
Pedro F. Giffuni
730cecb05a Dtrace: add toupper()/tolower() and enhancements to lltostr().
Merge changes from illumos:

1451 DTrace needs toupper()/tolower() subroutines
1457 lltostr() D subroutine should take an optional base

This change bumps the DT_VERS_* number to 1.8.1 in
accordance to what is done in illumos.

The test suite we currently include is outdated and
doesnt support some updates in tst.subr.d which had to
be left out for now.

Illumos Revisions:	r13458 5e394d8db762
			r13459 c3454574dd1a

Reference:
https://www.illumos.org/issues/1451
https://www.illumos.org/issues/1457

Tested by:	Fabian Keil
Obtained from:	Illumos
MFC after:	1 month
2013-03-25 15:40:57 +00:00
Pedro F. Giffuni
f2e66d30b8 Dtrace: add optional size argument to tracemem().
Merge change from illumos:

1455 DTrace tracemem() should take an optional size argument

Our local enhancements to dt_print_bytes were equivalent to
those in illumos but we made it match the illumos version
to ease further code merges.

For now leave out tst.smallsize.d and tst.smallsize.d.out
since those don't seem to work cleanly on FreeBSD.

This change bumps the DT_VERS_* number to 1.7.1 in accordance
to what is done in illumos.

Illumos Revision:	13457:571b0355c2e3

Reference:
https://www.illumos.org/issues/1455

Tested by:	Fabian Keil
Obtained from:	Illumos
MFC after:	1 month
2013-03-24 19:12:08 +00:00
Andriy Gapon
083a3ffdec dtrace: ensure that we can always catch a process (e.g. when -c is used)
It is not guaranteed that a program has a symbol table entry for main
and thus that it would be possible to set a breakpoint on it.

Reviewed by:	rpaulo
Discussed with:	rpaulo
MFC after:	13 days
2013-03-23 08:57:54 +00:00
Martin Matuska
05f49d92ef Merge libzfs_core branch:
includes MFV 238590, 238592, 247580

MFV 238590, 238592:
  In the first zfs ioctl restructuring phase, the libzfs_core library was
  introduced. It is a new thin library that wraps around kernel ioctl's.
  The idea is to provide a forward-compatible way of dealing with new
  features. Arguments are passed in nvlists and not random zfs_cmd fields,
  new-style ioctls are logged to pool history using a new method of
  history logging.

  http://blog.delphix.com/matt/2012/01/17/the-future-of-libzfs/

MFV 247580 [1]:
  To address issues of several deadlocks and race conditions the locking
  code around dsl_dataset was rewritten and the interface to synctasks
  was changed.

User-Visible Changes:
  "zfs snapshot" can create more arbitrary snapshots at once (atomically)
  "zfs destroy" destroys multiple snapshots at once
  "zfs recv" has improved performance

Backward Compatibility:
  I have extended the compatibility layer to support full backward
  compatibility by remapping or rewriting the responsible ioctl arguments.
  Old utilities are fully supported by the new kernel module.

Forward Compatibility:
  New utilities work with old kernels with the following restrictions:
    - creating, destroying, holding and releasing of multiple snapshots
      at once is not supported, this includes recursive (-r) commands

Illumos ZFS issues:
  2882 implement libzfs_core
  2900 "zfs snapshot" should be able to create multiple,
       arbitrary snapshots at once
  3464 zfs synctask code needs restructuring

References:
  https://www.illumos.org/issues/2882
  https://www.illumos.org/issues/2900
  https://www.illumos.org/issues/3464 [1]

MFC after:	1 month
Sponsored by:	Hybrid Logic Inc. [1]
2013-03-21 08:38:03 +00:00
Martin Matuska
07091d8f14 MFV r247580:
Merge synctask code restructuring from vendor.

Modify forward and backward compatibility to support new change.

Illumos ZFS issues:
  3464 zfs synctask code needs restructuring

Sponsored by:	Hybrid Logic Ltd.
2013-03-19 12:51:18 +00:00
Martin Matuska
876a84e867 MFC @248461 2013-03-18 09:39:51 +00:00
Martin Matuska
6f4accc2de Move common zfs ioctl compatibility functions (userland) into libzfs_compat.c
Introduce additional constants for zfs ioctl versions
2013-03-18 09:32:29 +00:00
Martin Matuska
d97b41582d Fix working with zfs_ioctl_version in libzfs_compat.h and include mirror
lzc_ioctl_version in libzfs_core
2013-03-17 22:24:08 +00:00
Martin Matuska
67ebc12d49 Add forwards compatibility for libzfs_core
Unsupported: creation of multiple snapshots including "zfs snapshot -r"
2013-03-17 18:33:06 +00:00
Martin Matuska
e2b4467975 libzfs_core:
- provide complete backwards compatibility (old utility, new kernel)
  - add zfs_cmd_t compatibility mapping in both directions
  - determine ioctl address in zfs_ioctl_compat.c
2013-03-17 10:57:04 +00:00
Martin Matuska
0afa556b1b MFV r248266:
Import minor ZFS changes from vendor

Illumos ZFS issues:
  3604 zdb should print bpobjs more verbosely (fix zdb hang)
  3606 zpool status -x shouldn't warn about old on-disk format

MFC after:	3 days
2013-03-14 10:02:59 +00:00
Martin Matuska
1a5160106d Update zfs.8 manpage date (missing in r247585)
MFC:	together with r247585
2013-03-14 08:18:40 +00:00
Simon J. Gerraty
7cf3a1c6b2 Updated dependencies 2013-03-11 17:21:52 +00:00
Martin Matuska
a03fbc7ecf MFC @248093 2013-03-09 11:57:51 +00:00
Dimitry Andric
e86b103cc6 Fix error in r247960: actually assign the basename to match.iim_file.
Pointed out by:	avg
Pointy hat to:	dim
MFC after:	1 week
X-MFC-With:	r247960
2013-03-07 22:43:50 +00:00
Dimitry Andric
8b0d8e9f2f Make ctfconvert work correctly on clang-compiled object files. Clang
puts the full original source filename in the STT_FILE entry of the ELF
symbol table, while gcc saves only the basename.

Since the DWARF DW_AT_name attribute contains the full source filename,
both for clang and gcc, ctfconvert takes just the basename of it, for
matching with the STT_FILE entry.  So when attempting to match with such
an entry, use its basename, if necessary.

Reported by:	avg
MFC after:	1 week
2013-03-07 22:16:35 +00:00
Martin Matuska
db49d41d8a Move libzfs compat functions to libzfs_compat.h
This header is used by both libzfs_core and libzfs libraries
2013-03-06 10:40:50 +00:00
Martin Matuska
400c4069a5 MFV r247845:
Import ZFS bpobj bugfix from vendor.

Illumos ZFS issues:
  3603 panic from bpobj_enqueue_subobj()
  3604 zdb should print bpobjs more verbosely

References:
  https://www.illumos.org/issues/3603
  https://www.illumos.org/issues/3604

MFC after:	1 week
2013-03-05 18:54:41 +00:00
Martin Matuska
dce1a726f2 WiP merge of libzfs_core (MFV r238590, r238592)
not yet working, ioctl handling needs to be changed
2013-03-05 08:09:53 +00:00
Martin Matuska
4abd59512a MFV r247316:
Merge new read-only zfs properties from vendor (illumos)

Illumos ZFS issues:
  3588 provide zfs properties for logical (uncompressed) space used and
       referenced

References:
  https://www.illumos.org/issues/3588

MFC after:	2 weeks
2013-03-01 21:58:51 +00:00
Martin Matuska
bb508e7732 Fix the zfs_ioctl compat layer to support zfs_cmd size change introduced
in r247265 (ZFS deadman thread). Both new utilities now support the old
kernel and new kernel properly detects old utilities.

For future backwards compatibility, the vfs.zfs.version.ioctl read-only
sysctl has been introduced. With this sysctl zfs utilities will be able
to detect the ioctl interface version of the currently loaded zfs module.

As a side effect, the zfs utilities between r247265 and this revision don't
support the old kernel module. If you are using HEAD newer or equal than
r247265, install the new kernel module (or whole kernel) first.

MFC after:	10 days
2013-03-01 09:42:58 +00:00
Martin Matuska
24245e76ea MFV 247176, 247178, 247315:
Import metaslab_sync() speedup from vendor (illumos).

Illumos ZFS issues:
  3552 condensing one space map burns 3 seconds of CPU in spa_sync() thread
  3564 spa_sync() spends 5-10% of its time in metaslab_sync() (when not
       condensing)
  3578 transferring the freed map to the defer map should be constant time
  3579 ztest trips assertion in metaslab_weight()

References:
  https://www.illumos.org/issues/3552
  https://www.illumos.org/issues/3564
  https://www.illumos.org/issues/3578
  https://www.illumos.org/issues/3579

MFC after:	2 weeks
2013-02-27 14:45:23 +00:00
Martin Matuska
e70664bafc MFV v242732:
Merge the ZFS I/O deadman thread from vendor (illumos).
This feature panics the system on hanging ZFS I/O, helps debugging
and resumes failed service.

The panic behavior can be controlled with the loader-only tunables:
vfs.zfs.deadman_enabled (enable or disable panic on stalled ZFS I/O)
vfs.zfs.deadman_synctime (expiration time for stalled ZFS I/O)

By default, ZFS I/O deadman is enabled by default on amd64 and i386
excluding virtual guest machines.

Illumos ZFS issues:
  3246 ZFS I/O deadman thread

References:
  https://www.illumos.org/issues/3246

MFC after:	2 weeks
2013-02-25 12:33:31 +00:00
Justin T. Gibbs
cf4e0cc15e Orphaned processes that are being traced are killed by the
kernel.  Properly restore, continue, and detach from processes
being DTraced when DTrace exits with an error so the program
being inspected is not terminated.

cddl/contrib/opensolaris/cmd/dtrace/dtrace.c:
	In fatal(), the generic error handler, close the DTrace
	handle as is done in the "probe/script" error handler
	dfatal().  fatal() can be invoked after DTrace attaches
	to processes (e.g. a script specified by command line
	argument can't be found) and closing the handle will
	release them.

Submitted by:	Spectra Logic Corporation
Reviewed by:	rpaulo, gnn
2013-02-20 17:46:38 +00:00
Simon J. Gerraty
f5f7c05209 Updated dependencies 2013-02-16 01:23:54 +00:00
Xin LI
f667ff300d Use the right year.
Noticed by:	Denis Ahrens <denis h3q com>
2013-02-11 18:22:26 +00:00
Martin Matuska
ff20578569 MFV r246392:
Import vendor ZFS bugfix fixing a possible deadlock in arc_read().

Illumos ZFS issues:
  3498 panic in arc_read(): !refcount_is_zero(&pbuf->b_hdr->b_refcnt)

References:
  https://www.illumos.org/issues/3498

MFC after:	2 weeks
2013-02-11 12:42:11 +00:00
Martin Matuska
fd9778c236 MFV r246388:
Import vendor bugfixes

Illumos ZFS issues:
  3422 zpool create/syseventd race yield non-importable pool
  3425 first write to a new zvol can fail with EFBIG

References:
  https://www.illumos.org/issues/3422
  https://www.illumos.org/issues/3425

MFC after:	2 weeks
2013-02-10 19:32:55 +00:00
Martin Matuska
1af420e000 Backport vendor changes in zfs(8) manual page (MFV r246389)
Illumos ZFS issues:
  3380 zfs man page: documentation for zfs allow is confusing

References:
  https://www.illumos.org/issues/3380

MFC after:	2 weeks
2013-02-10 17:10:07 +00:00
Gavin Atkinson
9a71ddccd0 Correct spelling of "daemon". No .Dd bump.
Noticed by:	Nathan Rich <Nathan.Rich dynastysystems com>
MFC after:	3 days
2013-02-10 14:28:07 +00:00
Joel Dahl
1848dd2aec mdoc: Remove EOL whitespace. 2013-02-09 07:01:05 +00:00
Xin LI
ef17620fc8 MFV r245512:
* Illumos zfs issue #3035 [1] LZ4 compression support in ZFS.

LZ4 is a new high-speed BSD-licensed compression algorithm created
by Yann Collet that delivers very high compression and decompression
performance compared to lzjb (>50% faster on compression, >80% faster
on decompression and around 3x faster on compression of incompressible
data), while giving better compression ratio [1].

This version of LZ4 corresponds to upstream's [2] revision 85.

Please note that for obvious reasons this is not backward read
compatible.  This means once a pool have LZ4 compressed data, these
data can no longer be read by older ZFS implementations.

Local changes:

 - On-stack hash table disabled and using kernel slab allocator
   instead, at this time.  This requires larger kernel thread stack
   for zio workers.  This may change in the future should we adjusted
   the zio workers' thread stack size.
 - likely and unlikely will be undefined if they are already defined,
   this is required for i386 XEN build.
 - Removed De Bruijn sequence based __builtin_ctz family of builtins
   in favor of the latter.  Both GCC and clang supports these builtins.
 - Changed the way the LZ4 code detects endianness.
 - Manual pages modifications to mention the feature based on Illumos
   counterpart.
 - Boot loader changes to make it support LZ4 decompression.

[1] https://www.illumos.org/issues/3035
[2] http://code.google.com/p/lz4/source/list

Obtained from:	Illumos (13921:9d721847e469)
Tested on:	FreeBSD/amd64
MFC after:	1 month
2013-02-09 06:39:28 +00:00
David E. O'Brien
d9a447559b Sync with HEAD. 2013-02-08 16:10:16 +00:00
Brooks Davis
afd9b4631f Add a new LIBRARIES_ONLY make variable to disable the build and install
of files other than the actual libraries.

Use LIBRARIES_ONLY to supress the inclusion of files in the lib32
distribution that are duplicates of files in base.

Sponsored by:	DARPA, AFRL
Reviewed by:	emaste
2013-01-17 17:27:10 +00:00
Steven Hartland
0e8f70b4bc Reports pools which have a removed l2cache disk under -x as this is what
happens when a cache device is dropped for any reason.

Reviewed by:	pjd
Approved by:	pjd (mentor)
MFC after:	2 weeks
2013-01-15 23:34:24 +00:00
Xin LI
290a1ba9a4 The current ZFS code expects ddt_zap_count to always succeed by asserting
the underlying zap_count() to return no errors.  However, it is possible
that the pool reaches to such a state where zap_count would return error,
leading to panics when a pool is imported.

This commit changes the ddt_zap_count to return error returned from
zap_count and handle the error appropriately.  With this change, it's now
possible to let zpool rollback damaged transaction groups and import the
pool.

Obtained from:	ZFS on Linux github (e8fd45a0f9)
MFC after:	1 month
2013-01-10 19:26:56 +00:00
Pawel Jakub Dawidek
1bba6bdf13 Allow to create pool even if mount point directory is not empty if -f is given.
Obtained from:	WHEEL Systems
2012-12-30 13:47:40 +00:00
Martin Matuska
4d50603f6b MFV r244245:
Merge two zdb bugfixes from vendor (illumos)

illumos-gate 13894:f4af77f6bbd2
3397 zdb <pool> <objnum> output is too verbose
3398 zdb can't dump feature flags zap objects

References:
https://www.illumos.org/issues/3397
https://www.illumos.org/issues/3398

MFC after:	1 week
2012-12-15 10:02:11 +00:00
Steven Hartland
425fb6dbaf Fixes zfs receive errors caused by snapshot replication being processed in a
random order instead of creation order.

Eliminates needless filesystem renames caused by removed parent snapshots
which subsequently causes many more errors.

PR:		kern/172259
Submitted by:	Steven Hartland
Reviewed by:	pjd (mentor)
Approved by:	pjd (mentor)
MFC after:	2 weeks
2012-12-13 22:03:07 +00:00
Martin Matuska
6e5bba4ae1 Update manpage dates in zfs.8 and zpool.8
MFC after:	2 weeks
2012-11-26 13:16:55 +00:00
Martin Matuska
7faa32552f MFV r243395:
Introduce a new dataset aclmode setting "restricted" to protect ACL's
being destroyed or corrupted by a drive-by chmod.

illumos-gate 13889:a67716f16746
3254 add support in zfs for aclmode=restricted

References:
https://www.illumos.org/issues/3254

MFC after:	2 weeks
2012-11-26 12:24:39 +00:00
Martin Matuska
dd801aa546 MFV r243013 and r243267:
Import the zio nop-write improvement from Illumos. To reduce I/O,
nop-write omits overwriting data if the checksum (cryptographically
secure) of new data matches the checksum of existing data.
It also saves space if snapshots are in use.

It currently works only on datasets with enabled compression, disabled
deduplication and sha256 checksums.

IllumOS 13887:196932ec9e6a and 13888:7204b3392a58
3236 zio nop-write

References:
https://www.illumos.org/issues/3236

MFC after:	2 weeks
2012-11-25 16:32:07 +00:00
Martin Matuska
64ad6bf196 zfs sha256 checksum is missing in zfs.8 manpage
MFC after:	3 days
2012-11-25 11:26:36 +00:00
Martin Matuska
2f06dfc9a3 MFV r243012:
Illumos 13886:e3261d03efbf

3349 zpool upgrade -V bumps the on disk version number, but leaves
     the in core version

References:
https://www.illumos.org/issues/3349

MFC after:	1 week
2012-11-25 10:53:42 +00:00
Sergey Kandaurov
9098689d87 Sort SEE ALSO xrefs by the manual section number and add a missing comma. 2012-11-15 20:50:18 +00:00
Martin Matuska
82e5de4d20 Move zpool-features manual page from section 5 to section 7
and fix references

Reported by:	pluknet
MFC after:	1 week
2012-11-14 01:16:29 +00:00
Sergey Kandaurov
0af021e861 Fix an apparent typo in the manual section number in .Dt: it should be 5. 2012-11-13 21:08:49 +00:00
Xin LI
a9b09a3f3c MFV r242729 (mm):
Illumos r13840:97fd5cdf328a:

3145 single-copy arc
3212 ztest: race condition between vdev_online() and spa_vdev_remove()

Illumos r13849:3468a95b27cd:

3258 ztest's use of file descriptors is unstable
2012-11-10 01:52:52 +00:00
Simon J. Gerraty
7cd2dcf076 Updated/new Makefile.depend 2012-11-08 21:24:17 +00:00
Justin Hibbits
c757049235 Implement DTrace for PowerPC. This includes both 32-bit and 64-bit.
There is one known issue:  Some probes will display an error message along the
lines of:  "Invalid address (0)"

I tested this with both a simple dtrace probe and dtruss on a few different
binaries on 32-bit.  I only compiled 64-bit, did not run it, but I don't expect
problems without the modules loaded.  Volunteers are welcome.

MFC after:	1 month
2012-11-07 23:45:09 +00:00
Simon J. Gerraty
23090366f7 Sync from head 2012-11-04 02:52:03 +00:00
Konstantin Belousov
5050aa86cf Remove the support for using non-mpsafe filesystem modules.
In particular, do not lock Giant conditionally when calling into the
filesystem module, remove the VFS_LOCK_GIANT() and related
macros. Stop handling buffers belonging to non-mpsafe filesystems.

The VFS_VERSION is bumped to indicate the interface change which does
not result in the interface signatures changes.

Conducted and reviewed by:	attilio
Tested by:	pho
2012-10-22 17:50:54 +00:00
Martin Matuska
1048151528 Add missing initialization for do_prefix.
Corrects porting error in r238391

Vendor issue and changeset reference:
2883 changing "canmount" property to "on" should not always remount dataset
https://www.illumos.org/issues/2883
Changeset 13743:95aba6e49b9f

Reported by:	Guido Falsi <mad@madpilot.net>, avg
Obtained from:	illumos (issue #2883)
MFC after:	1 week
2012-10-17 22:14:09 +00:00
Kevin Lo
ba670ce023 Make sure that each va_start has one and only one matching va_end,
especially in error cases.
2012-09-28 07:51:30 +00:00
Martin Matuska
8469b12c2e Merge recent vendor changes in ZFS.
Illumos issued covered:
2811 missing implementation: zfs send -r
3139 zdb dies when it tries to determine path of unlinked file
3189 kernel panic in ZFS test suite during hotspare_onoffline_004_neg
3208 moving zpool cross-endian results in incorrect user/group accounting

References:
  https://www.illumos.org/issues/ + [issue_id]

Obtained from:	illumos (vendor/illumos, vendor/illumos-sys)
MFC after:	2 weeks
2012-09-26 09:37:58 +00:00
Pawel Jakub Dawidek
c622f88dd2 It is possible to recursively destroy snapshots even if the snapshot
doesn't exist on a dataset we are starting from. For example if we
have the following configuration:

	tank
	tank/foo
	tank/foo@snap
	tank/bar
	tank/bar@snap

We can execute:

	# zfs destroy -t tank@snap

eventhough tank@snap doesn't exit.

Unfortunately it is not possible to do the same with recursive rename:

	# zfs rename -r tank@snap tank@pans
	cannot open 'tank@snap': dataset does not exist

...until now. This change allows to recursively rename snapshots even if
snapshot doesn't exist on the starting dataset.

Sponsored by:	rsync.net
MFC after:	2 weeks
2012-09-23 20:12:10 +00:00
Pawel Jakub Dawidek
bcb77be2b7 Add TRIM support.
The code builds a map of regions that were freed. On every write the
code consults the map and eventually removes ranges that were freed
before, but are now overwritten.

Freed blocks are not TRIMed immediately. There is a tunable that defines
how many txg we should wait with TRIMming freed blocks (64 by default).

There is a low priority thread that TRIMs ranges when the time comes.
During TRIM we keep in-flight ranges on a list to detect colliding
writes - we have to delay writes that collide with in-flight TRIMs in
case something will be reordered and write will reached the disk before
the TRIM. We don't have to do the same for in-flight writes, as
colliding writes just remove ranges to TRIM.

Sponsored by:	multiplay.co.uk

This work includes some important fixes and some improvements obtained
from the zfsonlinux project, including TRIMming entire vdevs on pool
create/add/attach and on pool import for spare and cache vdevs.

Obtained from:	zfsonlinux
Submitted by:	Etienne Dechamps <etienne.dechamps@ovh.net>
2012-09-23 19:40:58 +00:00
Baptiste Daroussin
b3da58854b Update usage to show the possiblity to use jail name
Reported by:	bdrewery
MFC after:	1 month
2012-09-19 16:28:38 +00:00
Baptiste Daroussin
ec6ddea1a8 Allow zfs jail and zfs unjail to use both jailnames and jailid
Reviewed by:	pjd
MFC after:	1 month
2012-09-19 16:20:49 +00:00
Martin Matuska
4c5238d576 Merge recent zfs vendor changes, sync code and adjust userland DEBUG.
Illumos issued covered:
1884 Empty "used" field for zfs *space commands
3006 VERIFY[S,U,P] and ASSERT[S,U,P] frequently check if first argument
     is zero
3028 zfs {group,user}space -n prints (null) instead of numeric GID/UID
3048 zfs {user,group}space [-s|-S] is broken
3049 zfs {user,group}space -t doesn't really filter the results
3060 zfs {user,group}space -H output isn't tab-delimited
3061 zfs {user,group}space -o doesn't use specified fields order
3064 usr/src/cmd/zpool/zpool_main.c misspells "successful"
3093 zfs {user,group}space's -i is noop
3098 zfs userspace/groupspace fail without saying why when run as non-root

References:
  https://www.illumos.org/issues/ + [issue_id]

Obtained from:	illumos (vendor/illumos, vendor/illumos-sys)
MFC after:	2 weeks
2012-09-12 18:05:43 +00:00
Joel Dahl
0dbe75e145 Remove trailing whitespace. 2012-09-06 19:26:59 +00:00
Glen Barber
104d19a8c3 Typo fix and minor word swap.
PR:		171356
Submitted by:	bdrewery
MFC After:	3 days
2012-09-05 23:07:02 +00:00
Martin Matuska
4a24a25b2f Merge recent vendor changes and sync code:
1862 incremental zfs receive fails for sparse file > 8PB
3112 ztest does not honor ZFS_DEBUG
3122 zfs destroy filesystem should prefetch blocks
3129 'zpool reopen' restarts resilvers
3130 ztest failure: Assertion failed:
       0 == dmu_objset_destroy(name, B_FALSE) (0x0 == 0x10)

References:
  https://www.illumos.org/issues/1862
  https://www.illumos.org/issues/3112
  https://www.illumos.org/issues/3122
  https://www.illumos.org/issues/3129
  https://www.illumos.org/issues/3130

Obtained from:	illumos (vendor/illumos, vendor/illumos-sys)
MFC after:	2 weeks
2012-09-05 12:02:09 +00:00
Glen Barber
b825120ab7 Add myself to copyright sections, per CDDL license.
Requested by:	mm
2012-09-03 00:05:21 +00:00
Rui Paulo
e04dfc407c Finish porting execsnoop to FreeBSD. This includes replacing the zonename
with a jail ID and removing the project ID from the list of options.
2012-09-01 08:14:21 +00:00
Joel Dahl
b7d683e608 Remove trailing whitespace. 2012-09-01 06:23:13 +00:00
Joel Dahl
86006ccf73 Remove trailing whitespace. 2012-09-01 06:12:14 +00:00
Joel Dahl
6ef6e985ec Mdoc fixes. 2012-09-01 06:07:27 +00:00
Joel Dahl
8508578c7f Minor mdoc fixes. 2012-08-31 21:44:12 +00:00
Joel Dahl
d56b30618d Mdoc fixes. 2012-08-31 17:18:41 +00:00
Martin Matuska
6e767def16 Merge recent vendor changes:
3100 zvol rename fails with EBUSY when dirty
3104 eliminate empty bpobjs
3120 zinject hangs in zfsdev_ioctl() due to uninitialized zc

References:
  https://www.illumos.org/issues/3100
  https://www.illumos.org/issues/3104
  https://www.illumos.org/issues/3120

Obtained from:	illumos (vendor/illumos, vendor/illumos-sys)
MFC after:	2 weeks
2012-08-28 12:25:37 +00:00
Martin Matuska
078f0b3e8d Add missing parts to zpool-features.5 manual page
Obtained from:	vendor/illumos/dist
MFC after:	2 weeks
2012-08-27 19:52:55 +00:00
Martin Matuska
671303c6d5 Merge recent vendor changes:
3086 unnecessarily setting DS_FLAG_INCONSISTENT on async destroyed datasets
3090 vdev_reopen() during reguid causes vdev to be treated as corrupt
3102 vdev_uberblock_load() and vdev_validate() may read the wrong label

Referenes:
  https://www.illumos.org/issues/3086
  https://www.illumos.org/issues/3090
  https://www.illumos.org/issues/3102

PR:		kern/170912, kern/170914
Obtained from:	illumos (changeset #13776, #13777)
MFC after:	2 weeks
2012-08-23 19:32:57 +00:00
Marcel Moolenaar
7750ad47a9 Sync FreeBSD's bmake branch with Juniper's internal bmake branch.
Requested by: Simon Gerraty <sjg@juniper.net>
2012-08-22 19:25:57 +00:00
Martin Matuska
1e350ef423 Update zfs(8) manpage with illumos version of "zfs diff"
Illumos issue:
  2399 zfs manual page does not document use of "zfs diff"

References:
  https://www.illumos.org/issues/2399

PR:		docs/170764
Obtained from:	ssh://anonhg@hg.illumos.org/illumos-gate
MFC after:	1 week
2012-08-19 10:34:40 +00:00
Glen Barber
e5561269e1 - Fix source dataset snapshot name in Example 15.
- Bump date.

MFC after:	3 days
X-MFC-With:	r239216
2012-08-12 20:30:15 +00:00
Glen Barber
c980404ee4 Remove a leading space that breaks rendering.
MFC after:	3 days
2012-08-12 20:23:59 +00:00
George V. Neville-Neil
26d121f5df When we return with an error we cannot unlock the mutex, because
it's been freed.  Protect against that, hopefully unlikely, case.

Reviewed by:	rpaulo
MFC after: 2 weeks
2012-08-01 19:27:12 +00:00
Martin Matuska
68f76b5452 Fix wrong indent according to style(9)
MFC after:	2 weeks
2012-07-31 17:32:28 +00:00
Martin Matuska
f45b531d72 Fix reporting of root pool upgrade notice.
MFC after:	2 weeks
2012-07-31 17:28:28 +00:00
Martin Matuska
e9832bb1da Partial MFV (illumos-gate 13753:2aba784c276b)
2762 zpool command should have better support for feature flags

References:
https://www.illumos.org/issues/2762

MFC after:	2 weeks
2012-07-30 23:14:24 +00:00
George V. Neville-Neil
96b3267700 Revert previous commit. The bug was actually caused by an issue
in pre 1.8.5 versions of sudo which were sending too many
SIGINTs to processes when the user hit Ctrl-C.

Pointed out by:	avg@, rpaulo@, sbruno@
2012-07-25 17:49:01 +00:00
George V. Neville-Neil
d83f1ab98e Fix a bug in interrupt handling so that we're only considered
impatient if we sent more than 2 INT signals.  This fixes a bug where
we wouldn't see aggregations print on the command line if we Ctrl-C'd
a dtrace script or command line invocation.

MFC after:	2 weeks
2012-07-24 18:01:28 +00:00
Pedro F. Giffuni
694a0093a7 Dtrace: improve handling of library paths.
Merge changes from illumos

906 dtrace depends_on pragma should search all library paths, not just the
current one

949 dtrace should only include the first instance of a library found on
its library path

Illumos Revisions:	13353:936a1e45726c
			13354:2b2c36a81512

Reference:
https://www.illumos.org/issues/906
https://www.illumos.org/issues/949

Tested by:	Fabian Keil
Obtained from:	Illumos
MFC after:	3 weeks
2012-07-17 19:57:34 +00:00
Pedro F. Giffuni
ba73774268 Use libc's strndup() instead of Dtrace's reimplementation.
Corresponds partially to OpenSolaris change:
PSARC 2010/299 GNU/Linux/BSD compatibility functions
6901783 strndup would be nice

MFC after:	2 weeks
2012-07-15 21:20:31 +00:00
Pedro F. Giffuni
b99795d637 Merge illumos commit 13455:7205f7794835
1458 D compiler fails to generate error on sizeof() an undefined struct

Reference:
https://www.illumos.org/issues/1458

Tested by:	Fabian Keil
MFC after:	3 weeks
2012-07-14 20:24:00 +00:00
Martin Matuska
bc06fa9281 Merge illumos commit 13749:df4cd82e2b60
1796 "ZFS HOLD" should not be used when doing "ZFS SEND" froma read-only pool
2871 support for __ZFS_POOL_RESTRICT used by ZFS test suite
2903 zfs destroy -d does not work
2957 zfs destroy -R/r sometimes fails when removing defer-destroyed snapshot

References:
https://www.illumos.org/issues/1796
https://www.illumos.org/issues/2871
https://www.illumos.org/issues/2903
https://www.illumos.org/issues/2957

MFC after:	1 week
2012-07-13 12:37:34 +00:00
Martin Matuska
04cb23e3b1 Change behavior introduced in r237119 to vendor solution
References:
https://www.illumos.org/issues/2883

PR:		167905
Obtained from:	illumos (issue #2883)
MFC after:	2 weeks
2012-07-12 06:29:54 +00:00
George V. Neville-Neil
3fac94ba94 Initial commit of an I/O provider for DTrace on FreeBSD.
These probes are most useful when looking into the structures
they provide, which are listed in io.d.  For example:

dtrace -n 'io:genunix::start { printf("%d\n", args[0]->bio_bcount); }'

Note that the I/O systems in FreeBSD and Solaris/Illumos are sufficiently
different that there is not a 1:1 mapping from scripts that work
with one to the other.
MFC after:	1 month
2012-07-11 16:27:02 +00:00
Ed Maste
24d2e21d9b Restore r211786 by rpaulo:
Port dtruss to FreeBSD.

  Sponsored by:   The FreeBSD Foundation

It appears the change was reverted by r235380.
2012-07-05 00:52:23 +00:00
Pawel Jakub Dawidek
a905af7e2d Fix an obvious typo.
MFC after:	3 days
2012-07-04 17:36:26 +00:00
Dimitry Andric
ae88a6d9ac Fix clang warning, introduced in the recent dtrace import.
MFC after:	   3 days
2012-07-03 12:08:55 +00:00
Pedro F. Giffuni
fed980b201 Safer fix for building with Clang.
error: format specifies type 'long long' but the argument
has type 'int64_t' (aka 'long')

Reported by:	Ed Maste
2012-06-28 16:44:29 +00:00
Pedro F. Giffuni
db0cb5be21 Fix build with Clang.
error: format specifies type 'long long' but the argument
has type 'int64_t' (aka 'long')

Reported by:	Ed Maste
2012-06-28 15:38:14 +00:00
Pedro F. Giffuni
675cf9154b Bring llquantize support into Dtrace.
Bryan Cantrill implemented the equivalent of semi-log graph
paper for Dtrace so llquantize will use one logarithmic and
one linear scale.

Special thanks to Mark Peek for providing fix to an
assertion and to Fabian Keill for testing the port.

Illumos Revision:	13355:15b74a2a9a9d

Reference:
https://www.illumos/issues/905

Obtained from:	Illumos
Tested by:	Fabian Keill, mp
MFC after:	4 days
2012-06-27 04:39:30 +00:00
Martin Matuska
153e430149 Do not remount ZFS dataset if changing canmount property to "on" and
dataset is already mounted.

PR:		167905
Submitted by:	Bryan Drewery <bryan@shatow.net>
MFC after:	1 week
2012-06-15 07:38:21 +00:00
Martin Matuska
0ca5e953e7 Document the -v flag for zpool list.
PR:		168970
Suggested by:	Marcelo Araujo <araujo@FreeBSD.org>
MFC after:	3 days
2012-06-12 14:40:19 +00:00
Martin Matuska
2d9cf57e18 Introduce "feature flags" for ZFS pools (bump SPA version to 5000).
Add first feature "com.delphix:async_destroy" (asynchronous destroy
of ZFS datasets).
Implement features support in ZFS boot code.

Illumos revisions merged:
13700:2889e2596bd6
13701:1949b688d5fb
2619 asynchronous destruction of ZFS file systems
2747 SPA versioning with zfs feature flags

References:
https://www.illumos.org/issues/2619
https://www.illumos.org/issues/2747

Obtained from:	illumos (issue #2619, #2747)
MFC after:	1 month
2012-06-11 11:35:22 +00:00
Glen Barber
0f135da354 Clean up trailing whitespace.
MFC after:	3 days
X-MFC-With:	r236776
2012-06-09 03:34:34 +00:00
Glen Barber
a82472f2de Fix a typo: s/deafult/default
MFC after:	3 days
2012-06-09 03:33:06 +00:00
Martin Matuska
923bc36411 Import Illumos revision 13715:351036203e4b
2803 zfs get guid pretty-prints the output

References:
https://www.illumos.org/issues/2803

Obtained from:	illumos (issue #2803)
MFC after:	3 days
2012-06-07 08:32:53 +00:00
George V. Neville-Neil
c5069a4756 Add DTrace's io.d, which handles tranlsations for file, buffer and
device info structures as well as the fds[] array.  This is a raw
version of the file, unmodified, to be used as a baseline.
2012-06-05 18:58:05 +00:00
Martin Matuska
2182d44714 Import illumos changeset 13570:3411fd5f1589
1948 zpool list should show more detailed pool information

Display per-vdev information with "zpool list -v".
The added expandsize property has currently no value on FreeBSD.
This changeset allows adding expansion support to individual vdevs
in the future.

References:
https://www.illumos.org/issues/1948

Obtained from:	illumos (issue #1948)
MFC after:	2 weeks
2012-05-27 16:00:00 +00:00
Martin Matuska
d9727dc29c Import illumos changeset 13605:b5c2b5db80d6 (partial)
763 FMD msg URLs should refer to something visible

Replace sun.com URL's with illumos.org

References:
https://www.illumos.org/issues/763

Obtained from:	illumos (issue #763)
MFC after:	1 week
2012-05-27 12:31:57 +00:00
Martin Matuska
e21ea64ae1 Import illumos changeset 13564:cf89c0c60496
1946 incorrect formatting when listing output of multiple pools with
zpool iostat -v

References:
https://www.illumos.org/issues/1946

Obtained from:	illumos (issue #1946)
MFC after:	1 week
2012-05-27 12:22:15 +00:00
Martin Matuska
6977304500 Import illumos changeset 13571:a5771a96228c
1950 ztest backwards compatibility testing option

References:
https://www.illumos.org/issues/1950

Obtained from:	illumos (issue #1950)
MFC after:	2 weeks
2012-05-27 11:37:24 +00:00
Pawel Jakub Dawidek
ff8dbdbf30 Correct error message.
MFC after:	3 days
2012-05-25 17:19:30 +00:00
Andriy Gapon
5e1ba35cea zpool_find_import_impl: another /dev/dsk -> /dev fix
This seems to fix zdb -e behavior.

PR:		bin/155104
Submitted by:	swell.k@gmail.com
MFC after:	2 weeks
2012-05-15 17:11:00 +00:00
Andriy Gapon
0002b76d93 zpool_do_import: use /dev instead of /dev/dsk as a default
This affects behavior of zpool import without -d option.

Reported by:	Bruce Cran <bruce@cran.org.uk>
Submitted by:	Fabian Keil <freebsd-listen@fabiankeil.de>
MFC after:	2 weeks
2012-05-15 17:07:29 +00:00
George V. Neville-Neil
151ba977e9 Move sh to the correct path for FreeBSD. 2012-05-14 22:02:01 +00:00
George V. Neville-Neil
334aca4e78 Move sh to the correct location for FreeBS (/bin/sh) 2012-05-14 22:00:24 +00:00
George V. Neville-Neil
562490e2da Fix execsnoop by changing exece and exec to be FreeBSD's execve.
Reference sh in the correct location (/bin/sh)
2012-05-14 21:59:47 +00:00
George V. Neville-Neil
44f24c137a Fix opensnoop for FreeBSD by removing probes with 64 at the end as
these are unnecessary.

Reference sh in the correct location (/bin/sh)
2012-05-14 21:58:22 +00:00
George V. Neville-Neil
f2f9999d8b Import dtracetoolkit into cddl/contrib 2012-05-12 21:25:48 +00:00
Martin Matuska
a837775a9e Import illumos changeset 13686:4bc0783f6064
2703 add mechanism to report ZFS send progress

If the zfs send command is used with the -v flag, the amount of bytes
transmitted is reported in per second updates.

References:
https://www.illumos.org/issues/2703

Obtained from:	illumos (issue #2703)
MFC after:	2 weeks
2012-05-10 10:39:45 +00:00
Martin Matuska
a4392ad6af Remove unreachable break in zfs_main.c
Partial import of illumos changeset 13622:e5889df1eaac
2077 lots of unreachable breaks in illumos gate

References:
https://www.illumos.org/issues/2077

PR:		kern/167370
Submitted by:	Marcelo Araujo <araujo@FreeBSD.org>
Obtained from:	illumos (issue #2077)
MFC after:	1 week
2012-05-10 09:10:31 +00:00
Martin Matuska
7538677f35 Add support for force unmounting ZFS filesystems during "zfs rename"
with the -f flag.

Reimplementation of the illumos changeset 13677:a0cbef703c12
2635 'zfs rename -f' to perform force unmount

References:
https://www.illumos.org/issues/2635

PR:		kern/164447
Suggested by:	Marcelo Araujo <araujo@FreeBSD.org>
Obtained from:	illumos (issue #2635)
MFC after:	1 week
2012-05-10 08:57:58 +00:00
Martin Matuska
9846ab4ed2 Import illumos changeset 13618:c6ae14a341e8 [1]:
2088 zdb could use a reasonable manual page

Rewrite new zdb manpage to mdoc(7)

References:
https://www.illumos.org/issues/2088

PR:		kern/167467 [1]
Submitted by:	Marcelo Araujo <araujo@FreeBSD.org> [1]
Reviewed by:	mm
Obtained from:	Illumos (issue #2088)
MFC after:	1 week
2012-05-10 04:11:39 +00:00
Glen Barber
2fccbf04cc General mdoc(7) and typo fixes.
PR:		167696
Submitted by:	Nobuyuki Koganemaru (kogane!jp.freebsd.org)
MFC after:	3 days
2012-05-10 02:07:00 +00:00
Robert Millan
e851fbe3de GNU/kFreeBSD portability fix. This glue <stdlib.h> overrides header protection
in the system-wide version of <stdlib.h> by wrapping the #include_next
<stdlib.h> within the scope of its own header protection. On FreeBSD this has
no effect, since both header protections are equivalent. However the GNU version
of <stdlib.h> implements a special header protection mechanism which allows it
to be included multiple times (in different modes).

Simply by moving the #include_next off the header protection, we allow
system-wide <stdlib.h> to implement its own protection policy, whichever that
may be.
2012-04-30 23:12:16 +00:00
Ryan Stone
c6024848dd Implement the D "cpu" variable, which returns curcpu. I have chosen not
to follow the example of OpenSolaris and its descendants, which implemented
cpu as an inline that took a value out of curthread.  At certain points in
the FreeBSD scheduler curthread->td_oncpu will no longer be valid (in
particukar, just before the thread gets descheduled) so instead I have
implemented this as its own built-in variable.

Sponsored by:	Sandvine Inc.
MFC after:	1 week
2012-04-26 01:07:03 +00:00
Pawel Jakub Dawidek
7aa828debf Add -u option to 'zfs create' that prevents file system from being
automatically mounted. This is similar to the 'zfs receive -u'.

MFC after:	1 week
2012-04-24 13:44:46 +00:00
Xin LI
ba89ae6d5d - Correct a typo which prevents 'lzjb' be displayed;
- Use quotes when tab is used.
2012-04-22 07:50:24 +00:00
Martin Matuska
c277658f24 Fix typo miror -> mirror
Reported by:	Glen Barber <gjb@FreeBSD.org>
MFC after:	3 days
2012-04-16 08:19:43 +00:00
George V. Neville-Neil
72cc304aba Change SIGUSR1 to SIGTHR to properly wake up a process that is being
traced.  The use of SIGUSR1 caused traced processes (those attached to
with dtrace -p) to exit when dtrace exited.

MFC in: 1 week
2012-04-13 20:22:08 +00:00
Oleksandr Tymoshenko
457c7edfbd Enable build of DTrace-related userland parts for MIPS 2012-03-24 05:29:07 +00:00
Oleksandr Tymoshenko
51df478cba Add stub file for pid probe. It's required although pid probe is not supported
on MIPS yet
2012-03-24 05:27:39 +00:00
Oleksandr Tymoshenko
a6425ab52d Maintain target's byte order for multi-byte fields in CTF structures.
CTF format is not cross-platform by design, e.g. it is not guaranteed
that data generated by ctfconvert/ctfmerge on one architecture will
be successfuly read on another. CTF structures are saved/restored
using naive approach. Roughly it looks like:
	write(fd, &ctf_struct, sizeof(ctf_struct))
	read(fd, &ctf_struct, sizeof(ctf_struct))

By sheer luck memory layout of all type-related CTF structures is the same
on amd64/i386/mips32/mips64. It's different on ARM though. sparc, ia64,
powerpc, and powerpc64 were not tested. So in order to get file compatible
with dtrace on ARM it should be compiled on ARM. Alternative solution would
be to have "signatures" for every platform and ctfmerge should convert host's
reperesentation of CTF structure to target's one using "signature" as template.

This patch checks byte order of ELF files used for generating CTF record
and makes sure that byte order of data written to resulting files is the same
as target's byte order.
2012-03-24 01:47:33 +00:00
Martin Matuska
e7af90ab00 Analogous to r232059, add a parameter for the ZFS file system:
allow.mount.zfs:
	allow mounting the zfs filesystem inside a jail

This way the permssions for mounting all current VFCF_JAIL filesystems
inside a jail are controlled wia allow.mount.* jail parameters.

Update sysctl descriptions.
Update jail(8) and zfs(8) manpages.

TODO:	document the connection of allow.mount.* and VFCF_JAIL for kernel
	developers

MFC after:	10 days
2012-02-26 16:30:39 +00:00
Martin Matuska
dd7f80a359 Import illumos changeset 13608 [1]:
add support for "-t <datatype>" argument to zfs get

References:
https://www.illumos.org/issues/1936

Update zfs(8) manpage in respect of [1].
Fix typo in zfs(8) manpage.

Obtained from:	illumos (issue #1936)
MFC after:	1 week
2012-02-23 19:13:19 +00:00
Martin Matuska
ae5458e1fb Merge illumos revision 13594:
uninitialized variables in zfs(8) may make snapshots undestroyable [1]

References:
https://www.illumos.org/issues/2067 [1]

Obtained from:	illumos (issue #2067)
MFC after:	3 days
2012-02-07 18:35:18 +00:00
Pedro F. Giffuni
a3681f9010 Set SVN text/plain property for some shell scripts that
happen to have a .exe extension.

While here fix the shebang of a shell script that
was looking for /bin/bash.

Reviewed by:	gnn
Approved by:	jhb (mentor)
MFC after:	2 weeks
2012-01-28 00:17:17 +00:00