Commit Graph

662 Commits

Author SHA1 Message Date
delphij
aee9275df5 Don't panic when we get ZPOOL_STATUS_NON_NATIVE_ASHIFT
while listing importable pools.

MFC after:	3 days
2013-12-09 18:52:21 +00:00
joel
49a4972ff7 mdoc: remove EOL whitespace. 2013-12-06 21:22:33 +00:00
markj
231dd1c8cb Enable some previously-disabled DTrace tests for umod, ufunc and usym. They
expect the installed ksh binary to be named "ksh", which is not the case
when it's installed on FreeBSD via the shells/ksh93 port. Allow for it to be
"ksh93" as well so that the tests can actually pass.
2013-12-04 01:40:39 +00:00
markj
11fb004776 The uaddr, ufunc, umod and usym functions all seem to work as expected on
FreeBSD, so stop hiding them behind a "#if defined(sun)".

Reported by:	Prashanth Kumar <pra_udupi@yahoo.co.in>
2013-12-04 01:35:04 +00:00
markj
80b61d514a Use mkstemp(3) to create the temporary file used in the FreeBSD-specific
portions of dtrace_program_link().
2013-12-03 03:40:47 +00:00
avg
9932b97e88 MFV r258371,r258372: 4101 metaslab_debug should allow for fine-grained control
4101 metaslab_debug should allow for fine-grained control
4102 space_maps should store more information about themselves
4103 space map object blocksize should be increased
4104 ::spa_space no longer works
4105 removing a mirrored log device results in a leaked object
4106 asynchronously load metaslab

illumos/illumos-gate@0713e232b7

Note that some tunables have been removed and some new tunables have
been added.  Of particular note, FreeBSD-only knob
vfs.zfs.space_map_last_hope is removed as it was a nop for some time now
(after one of the previous merges from upstream).

MFC after:	11 days
Sponsored by:	HybridCluster [merge]
2013-11-28 19:37:22 +00:00
avg
51e896cc78 MFV r255255: 4045 zfs write throttle & i/o scheduler performance work
illumos/illumos-gate@69962b5647

Please note the following changes:
- zio_ioctl has lost its priority parameter and now TRIM is executed
  with 'now' priority
- some knobs are gone and some new knobs are added; not all of them are
  exposed as tunables / sysctls yet

MFC after:	10 days
Sponsored by:	HybridCluster [merge]
2013-11-26 09:57:14 +00:00
avg
d1f2a86401 734 taskq_dispatch_prealloc() desired
943 zio_interrupt ends up calling taskq_dispatch with TQ_SLEEP
illumos/illumos-gate@5aeb94743e

Essentially FreeBSD taskqueues already operate in a mode that
was added to Illumos with taskq_dispatch_ent change.
We even exposed the superior FreeBSD interface as taskq_dispatch_safe.
Now we just rename taskq_dispatch_safe to taskq_dispatch_ent and
struct struct ostask to taskq_ent_t, so that code differences will be
minimal.

After this change sys/cddl/compat/opensolaris/sys/taskq.h header is no
longer needed.

Note that this commit is not an MFV because the upstream change was not
individually committed to the vendor area.

MFC after:	8 days
2013-11-26 09:26:18 +00:00
jhibbits
eda0b2bf25 Use 'int' to store the return value of getopt(), rather than char.
On some architectures (powerpc), char is unsigned by default, which means
comparisons against -1 always fail, so the programs get stuck in an
infinite loop.

MFC after:	1 week
2013-11-20 01:42:29 +00:00
markj
6f96d8451e Don't try to use the 32-bit drti.o unless the data model is explicitly set
to ILP32. Otherwise dtrace -G will attempt to use it on amd64 if it can't
determine which data model to use, which happens when -64 is omitted and
no object files are provided, e.g. with

# dtrace -G -n BEGIN

This would result in a linker error, but now works properly.

Also remove an unnecessary #ifdef.

MFC after:	2 weeks
2013-11-09 04:38:16 +00:00
sbruno
3d7ba62a0b Quiesce warning assigning to void * from const ctf_header_t * by explicity casting
to void * before assignment.

Submitted as Illumos issue 4287
2013-11-04 21:32:07 +00:00
sbruno
029584a9a1 spelling in comments fixup
Submitted by:		Joerg Sonnenberger <joerg@britannica.bec.de>
2013-11-04 19:32:35 +00:00
sbruno
189f0aae34 Quiesce warning regarding %llf which has no effect.
Submitted as illumos issue #4284

Reviewed by:	delphij
2013-11-04 16:15:43 +00:00
sbruno
8fc848171e This library uses macros to define fprintf behvavior for several object types
The compiler will see the non-string literal arguments to the fprintf calls and
omit warnings for them. Quiese these warnings in contrib code:

cddl/contrib/opensolaris/lib/libnvpair/libnvpair.c:743:12: warning: format
  string is not a string literal (potentially insecure) [-Wformat-security]
  ARENDER(pctl, nvlist_array, nvl, name, val, nelem);
2013-11-03 21:05:44 +00:00
markj
a5afef3ae8 If the initial attempt to open /dev/ksyms fails, kldload the ksyms module
and retry.
2013-10-27 16:18:48 +00:00
markj
9d398866fd Convert the lockstat(1) man page to mdoc and make sure that it gets
installed. Additionally, remove Solaris-specific sections and references,
and replace example outputs with output from lockstat on FreeBSD, since
lockstat's output contains stack traces.

This change also removes some examples that don't seem to work properly on
FreeBSD. The examples should be re-added when lockstat is fixed.

Reported by:	avg
MFC after:	1 week
2013-10-27 16:01:11 +00:00
smh
a5552b83b2 Added support for the 'zfs list -t snap' and 'zfs snap' aliases which are
available under Oracle Solaris 11.

This includes an update to the ZFS(8) man page to reflect all the
available alias (snap, umount, and recv).

Initial changes obtained from ZFS On Linux + fixes for man page and cmd
help:
10b75496bb
cf81b00a73

Obtained from:	https://github.com/zfsonlinux/zfs
MFC after:	2 weeks
Sponsored by:	Multiplay
2013-10-23 18:22:27 +00:00
markj
3ecc6f1298 Add a function, memstr, which can be used to convert a buffer of
null-separated strings to a single string. This can be used to print the
full arguments of a process using execsnoop (from the DTrace toolkit) or
with the following one-liner:

dtrace -n 'syscall::execve:return {trace(curpsinfo->pr_psargs);}'

Note that this relies on the process arguments being cached via the struct
proc, which means that it will not work for argvs longer than
kern.ps_arg_cache_limit. However, the following rather non-portable
script can be used to extract any argv at exec time:

fbt::kern_execve:entry
{
    printf("%s", memstr(args[1]->begin_argv, ' ',
        args[1]->begin_envv - args[1]->begin_argv));
}

The debug.dtrace.memstr_max sysctl limits the maximum argument size to
memstr(). Thanks to Brendan Gregg for helpful comments on freebsd-dtrace.

Tested by:	Fabian Keil (earlier version)
MFC after:	2 weeks
2013-10-16 01:39:26 +00:00
jhibbits
0b9629ab6a Add fasttrap for PowerPC. This is the last piece of the dtrace/ppc puzzle.
It's incomplete, it doesn't contain full instruction emulation, but it should be
sufficient for most cases.

MFC after:	1 month
2013-10-15 15:00:29 +00:00
markj
0b5457bf66 Convert the dtrace(1) man page to mdoc and fix up some aspects of it that
don't make sense on FreeBSD. In particular,

- remove the ATTRIBUTES section,
- remove references to the Solaris Dynamic Tracing Guide, except in the
  SEE ALSO section,
- update the description of the -A option for FreeBSD's implementation,
- remove references to Solaris-specific programs and configuration files,
  and replace them with FreeBSD equivalents where possible.

The content has not changed aside from this.

Approved by:	re (joel)
MFC after:	1 week
2013-10-10 03:50:23 +00:00
rmh
774f6b00bf Fix implicit declaration of jail_getid()
Approved by:	re
2013-10-07 14:22:19 +00:00
markj
8ff2d52009 Add a separate translator for headers passed to the TCP probes in the
input path. These probes get some of the fields in host order, whereas the
output probes get them in network order, so a single translator isn't
enough. This workaround ensures that the problem is essentially invisble
to users: none of the probe arguments or their fields have changed.

Approved by:	re (hrs)
2013-10-02 17:14:12 +00:00
delphij
d04a7f0144 MFV r254750:
Add support of Illumos dumps on zvol over RAID-Z.

Note that this only adds the features.  FreeBSD would
still need more work to support dumping on zvols.

Illumos ZFS issues:
  2932 support crash dumps to raidz, etc. pools

MFC after:	1 month
Approved by:	re (ZFS blanket)
2013-09-21 00:17:26 +00:00
markj
37ef3ce41e Use the address of the inpcb rather than the tcpcb to identify TCP
connections. This keeps the tcp provider consistent with the other network
providers.

Approved by:	re (delphij)
2013-09-15 21:38:46 +00:00
delphij
a23043347c MFV r247844 (illumos-gate 13975:ef6409bc370f)
Illumos ZFS issues:
  3582 zfs_delay() should support a variable resolution
  3584 DTrace sdt probes for ZFS txg states

Provide a compatibility shim for Solaris's cv_timedwait_hires
to help aid future porting.

Approved by:	re (ZFS blanket)
2013-09-10 01:46:47 +00:00
will
1b508b8cc8 Build all ZFS testing & debugging tools with -g.
These programs and everything using libzpool rely on the embedded asserts to
verify the correctness of operations.  Given that, the core dumps would be
useless without debug symbols.
2013-08-27 04:01:31 +00:00
pfg
615f223c07 Merge various CTF fixes from illumos
2942 CTF tools need to handle files which legitimately lack data
2978 ctfconvert still needs to ignore legitimately dataless files on SPARC

Illumos Revisions:	13745:6b3106b4250f
			13754:7231b684c18b

Reference:

https://www.illumos.org/issues/2942
https://www.illumos.org/issues/2978

MFC after:	3 weeks
2013-08-26 22:29:42 +00:00
markj
29e4661920 Implement the ip, tcp, and udp DTrace providers. The probe definitions use
dynamic translation so that their arguments match the definitions for
these providers in Solaris and illumos. Thus, existing scripts for these
providers should work unmodified on FreeBSD.

Tested by:	gnn, hiren
MFC after:	1 month
2013-08-25 21:54:41 +00:00
delphij
a4cf8ab508 MFV r254751:
Don't treat the parameter as a number (pool GUID) when there is
error converting it from string, instead, treat it as the pool
name.

Illumos ZFS issues:
  1765 assert triggered in libzfs_import.c trying to import pool
       name beginning with a number
2013-08-24 00:54:47 +00:00
delphij
40685a92a9 MFV r254748:
Fix memory leak in libzfs's iter_dependents_cb().

Illumos ZFS issues:
  4061 libzfs: memory leak in iter_dependents_cb()
2013-08-24 00:29:34 +00:00
delphij
677bfa2265 MFV r254746:
To quote original Illumos ticket:

libctf thinks that any ELF file containing more than 65536 sections is
corrupt, because it doesn't understand the SHN_XINDEX magic.

Illumos DTrace issues:
  4005 libctf can't deal with extended sections
2013-08-23 23:58:56 +00:00
delphij
5017a032d2 MFV r254422:
Illumos DTrace issues:
  3089 want ::typedef
  3094 libctf should support removing a dynamic type
  3095 libctf does not validate arrays correctly
  3096 libctf does not validate function types correctly
2013-08-23 23:21:24 +00:00
gibbs
bd47afb289 Enhance the ZFS vdev layer to maintain both a logical and a physical
minimum allocation size for devices.  Use this information to
automatically increase ZFS's minimum allocation size for new top-level
vdevs to a value that more closely matches the optimum device
allocation size.

Use GEOM's stripesize attribute, if set, as the physical sector
size of the GEOM.

Calculate the minimum blocksize of each metaslab class.  Use the
calculated value instead of SPA_MINBLOCKSIZE (512b) when determining
the likelyhood of compression yeilding a reduction in physical space
usage.

Report devices with sub-optimal block size configuration in "zpool
status".  Also properly fail attempts to attach devices with a
logical block size greater than 8kB, since this will cause corruption
to ZFS's label area.

Sponsored by:	Spectra Logic Corporaion
MFC after:	2 weeks

Background
==========
Many modern devices use physical allocation units that are much
larger than the minimum logical allocation size accessible by
external commands.  Two prevalent examples of this are 512e disk
drives (512b logical sector, 4K physical sector) and flash devices
(512b logical sector, 4K or larger allocation block size, and 128k
or larger erase block size).  Operations that modify less than the
physical sector size result in a costly read-modify-write or garbage
collection sequence on these devices.

Simply exporting the true physical sector of the device to ZFS would
yield optimal performance, but has two serious drawbacks:

1) Existing pools created with devices that have different logical
   and physical block sizes, but were configured to use the logical
   block size (e.g. because the OS version used for pool construction
   reported the logical block size instead of the physical block
   size) will suddenly find that the vdev allocation size has
   increased.  This can be easily tolerated for active members of
   the array, but ZFS would prevent replacement of a vdev with
   another identical device because it now appears that the smaller
   allocation size required by the pool is not supported by the new
   device.

2) The device's physical block size may be too large to be supported
   by ZFS.  The optimal allocation size for the vdev may be quite
   large.  For example, a RAID controller may export a vdev that
   requires read-modify-write cycles unless accessed using 64k
   aligned/sized requests.  ZFS currently has an 8k minimum block
   size limit.

Reporting both the logical and physical allocation sizes for vdevs
solves these problems.  A device may be used so long as the logical
block size is compatible with the configuration.  By comparing the
logical and physical block sizes, new configurations can be optimized
and administrators can be notified of any existing pools that are
sub-optimal.

sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/spa.h:
	Add the SPA_ASHIFT constant.  ZFS currently has a hard upper
	limit of 13 (8k) for ashift and this constant is used to
	both document and enforce this limit.

sys/cddl/contrib/opensolaris/uts/common/sys/fs/zfs.h:
	Add the VDEV_AUX_ASHIFT_TOO_BIG error code.

	Add fields for exporting the configured, logical, and
	physical ashift to the vdev_stat_t structure.

	Add VDEV_STAT_VALID() macro which can be used to verify the
	presence of required vdev_stat_t fields in nvlist data.

sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c:
	Provide a SYSCTL_PROC handler for "max_auto_ashift".  Since
	the limit is only referenced long after boot when a create
	operation occurs, there's no compelling need for it to be
	a boot time configurable tunable.  This also allows the
	validation code for the max_auto_ashift value to be contained
	within the sysctl handler.

	Populate the new fields in the vdev_stat_t structure.

	Fail vdev opens if the vdev reports an ashift larger than
	SPA_MAXASHIFT.

	Propogate vdev_logical_ashift and vdev_physical_ashift between
	child and parent vdevs as is done for vdev_ashift.

	In vdev_open(), restore code that fails opens for devices
	where vdev_ashift grows.  This can only happen now if the
	device's logical ashift grows, which means it really isn't
	safe to use the device.

sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev_impl.h:
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c:
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_file.c:
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c:
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_mirror.c:
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_missing.c:
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_raidz.c:
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_root.c:
	Update the vdev_open() API so that both logical (what was
	just ashift before) and physical ashift are reported.

sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev_impl.h:
	Add two new fields, vdev_physical_ashift and vdev_logical_ashift,
	to vdev_t.

sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c:
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_config.c:
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c:
	Add vdev_ashift_optimize().  Call it anytime a new top-level
	vdev is allocated.

cddl/contrib/opensolaris/cmd/zpool/zpool_main.c:
	Add text for the VDEV_AUX_ASHIFT_TOO_BIG error.

	For each sub-optimally configured leaf vdev, report configured
	and native block sizes.

cddl/contrib/opensolaris/cmd/zpool/zpool_main.c:
cddl/contrib/opensolaris/lib/libzfs/common/libzfs.h:
cddl/contrib/opensolaris/lib/libzfs/common/libzfs_status.c:
	Introduce a new zpool status: ZPOOL_STATUS_NON_NATIVE_ASHIFT.
	This status is reported on healthy pools containing vdevs
	configured to use a block size smaller than their reported
	physical block size.

cddl/contrib/opensolaris/lib/libzfs/common/libzfs_status.c:
	Update find_vdev_problem() and supporting functions to
	provide the full vdev_stat_t structure to problem checking
	routines, and to allow decent into replacing vdevs.

	Add a vdev_non_native_ashift() validator which is used on
	the full vdev tree to check for ZPOOL_STATUS_NON_NATIVE_ASHIFT.

cddl/contrib/opensolaris/lib/libzpool/common/kernel.c:
cddl/contrib/opensolaris/lib/libzpool/common/sys/zfs_context.h:
	Enhance sysctl userland stubs now that a SYSCTL_PROC handler
	is used in vdev.c.

sys/cddl/contrib/opensolaris/uts/common/fs/zfs/metaslab.c:
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/metaslab_impl.h:
	When the group membership of a metaslab class changes (i.e.
	when a vdev is added or removed from a pool), walk the group
	list to determine the smallest block size currently available
	and record this in the metaslab class.

sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/metaslab.h:
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/metaslab.c:
	Add the metaslab_class_get_minblocksize() accessor.

sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zio_compress.h:
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio_compress.c:
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:
	In zio_compress_data(), take the minimum blocksize as an
	input parameter instead of assuming SPA_MINBLOCKSIZE.

sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:
	In l2arc_compress_buf(), pass SPA_MINBLOCKSIZE as the minimum
	blocksize of the device.  The l2arc code performs has it's own
	code for deciding if compression is worth while, so this
	effectively disables zio_compress_data() from second guessing
	the original decision.

sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:
	In zio_write_bp_init(), use the minimum blocksize of the
	normal metaslab class when compressing data.
2013-08-21 04:10:24 +00:00
delphij
1b0e7b9e07 MFV r254421:
Illumos ZFS issues:
  3996 want a libzfs_core API to rollback to latest snapshot
2013-08-21 00:04:31 +00:00
rpaulo
10427f9082 Load the dtraceall module if /dev/dtrace/dtrace doesn't exist.
MFC after:	3 days
2013-08-10 23:17:09 +00:00
delphij
ccc3c4970e MFV r254079:
Illumos ZFS issues:
  3957 ztest should update the cachefile before killing itself
  3958 multiple scans can lead to partial resilvering
  3959 ddt entries are not always resilvered
  3960 dsl_scan can skip over dedup-ed blocks if
       physical birth != logical birth
  3961 freed gang blocks are not resilvered and can cause pool to suspend
  3962 ztest should print out zfs debug buffer before exiting
2013-08-08 23:38:31 +00:00
delphij
7291294314 MFV r254071:
Fix a regression introduced by fix for Illumos bug #3834.  Quote from
Matthew Ahrens on the Illumos issue:

ztest fails this assertion because ztest_dmu_read_write() does
        dmu_tx_hold_free(tx, bigobj, bigoff, bigsize);
and then
    dmu_object_set_checksum(os, bigobj,
        (enum zio_checksum)ztest_random_dsl_prop(ZFS_PROP_CHECKSUM), tx);

If the region to free is past the end of the file, the DMU assumes that there
will be nothing to do for this object.  However, ztest does set_checksum(),
which must modify the dnode.  The fix is for ztest to also call

    dmu_tx_hold_bonus(tx, bigobj);

so we can account for the dirty data associated with setting the checksum

Illumos ZFS issues:
  3955 ztest failure: assertion refcount_count(&tx->tx_space_written)
         + delta <= tx->tx_space_towrite
2013-08-07 22:21:00 +00:00
delphij
0171695909 MFV r254070:
Merge vendor bugfix for ZFS test suite that triggers false positives.

Illumos ZFS issues:
  3949 ztest fault injection should avoid resilvering devices
  3950 ztest: deadman fires when we're doing a scan
  3951 ztest hang when running dedup test
  3952 ztest: ztest_reguid test and ztest_fault_inject don't place nice together
2013-08-07 21:16:14 +00:00
rmh
5933e25518 Fix implicit declaration of warnx(). 2013-08-04 16:25:46 +00:00
delphij
c5affee6a3 MFV r253781 + r253871:
Illumos ZFS issues:
  3894 zfs should not allow snapshot of inconsistent dataset

MFC after:	2 weeks
2013-07-30 21:02:09 +00:00
smh
ef92cf9910 MFV r253784:
Fix zfs send -D hang after processing requiring a CTRL+C to interrupt due to
pthread_join prior to fd close.

This was introduced by r251646 (MFV r251644)

Illumos ZFS issue:
  3909 "zfs send -D" does not work

MFC after:	1 day
2013-07-30 20:45:27 +00:00
pfg
03e98d6f13 DTrace: re-apply r249426 now that the underlying issues have been solved.
Merge change from illumos:

3519 DTrace fails to resolve const types from fbt
3520 dtrace internal error -- token type 316 is not a valid D
     compilation token
3521 clean up dtrace unit tests

Illumos Revision:	e98f46c

Reference:
https://www.illumos.org/issues/3519
https://www.illumos.org/issues/3520
https://www.illumos.org/issues/3521

Tested by:	Fabian Keil
Obtained from:	Illumos
MFC after:	1 month
2013-07-28 01:02:17 +00:00
pfg
46097436dc DTrace: re-merge remainder of r249367 (original from Illumos).
Bring back some important fixes from Illumos:

3022 DTrace: keys should not affect the sort order when sorting by value
3023 it should be possible to dereference dynamic variables
3024 D integer narrowing needs some work

We particularly avoid the LD_NOLAZYLOAD changes that Illumos made
as those don't apply to FreeBSD and were causing problems in
interactive mode.

Illumos Revision:	13758:23432da34147

Reference:

https://www.illumos.org/issues/3022
https://www.illumos.org/issues/3023
https://www.illumos.org/issues/3024

MFC after:	1 month
Tested by:	markj
2013-07-28 00:45:20 +00:00
markj
401af7020e Use kern_ioctl() rather than ioctl() for testing the FBT provider, since the
latter doesn't exist in FreeBSD. All the tests under fbtprovider pass now.
2013-07-27 21:31:48 +00:00
pfg
923a920777 Style issue in r253661.
Pointed out by:	avg
MFC after:	1 month
2013-07-26 14:37:23 +00:00
pfg
34504eb8a9 Fix a segfault in ctfmerge due to a bug in gcc.
GCC can generate bogus dwarf attributes with DW_AT_byte_size
set to 0xFFFFFFFF.
The issue was originaly detected in NetBSD but it has been
adapted for portability and to avoid compiler warnings.

Reference:
https://www.illumos.org/issues/3776

Obtained from:	NetBSD
MFC after:	1 month
2013-07-26 00:28:19 +00:00
delphij
5a5f4c42fb Manually merge part of vendor import r238583 from Illumos.
Illumos changeset: 13680:2bd022a765e2
Illumos ZFS issue:

    2671 zpool import should not fail if vdev ashift has increased

MFC after:	3 days
2013-07-18 00:22:42 +00:00
mm
ddbb7a25f4 Fix misleading or remove irrelevant illumos messages and manpage references
in the zfs command.

PR:		bin/178996
Submitted by:	Peter Schaefer <peter.schaefer@wilhelmheinrichs.de>
MFC after:	3 days
2013-07-04 22:26:38 +00:00
delphij
4dfc3c75a2 MFV r252215:
Restore a previous behavior before r251646, where when destructing
ZFS snapshot, the ioctl would return ENOENT when it hit any of
them in the errlist (the new behavior was only return ENOENT when
all returns error).

Illumos ZFS issues:
  3829 fix for 3740 changed behavior of zfs destroy/hold/release ioctl

MFC after:	1 week
2013-06-25 22:14:32 +00:00
delphij
5240466227 Diff reduction against Illumos, no real change to code itself.
This marks vendor branch revision 252213 as merged, the actual code was
committed in r245479.

MFC after:	1 week
2013-06-25 21:51:52 +00:00
smh
08b5f9540c Fixed ZFS zpool freeze (debug command) not processing due to invalid ioctl call syntax.
MFC after:	1 week
2013-06-21 15:30:46 +00:00
delphij
ab3dbcb998 MFV r251644:
Poor ZFS send / receive performance due to snapshot
hold / release processing (by smh@)

Illumos ZFS issues:
  3740 Poor ZFS send / receive performance due to snapshot
       hold / release processing

MFC after:      2 weeks
2013-06-12 07:07:06 +00:00
delphij
421d823726 MFV r251624:
txg commit callbacks don't work

Illumos ZFS issues:
  3747 txg commit callbacks don't work

MFC after:      2 weeks
2013-06-11 19:29:31 +00:00
delphij
8fe6a06d58 MFV r251623:
zpool create should treat -O mountpoint and -m the same

Illumos ZFS issues:
  3745 zpool create should treat -O mountpoint and -m the same

MFC after:      2 weeks
2013-06-11 19:25:49 +00:00
delphij
9d0815fcd1 MFV r251619:
ZFS needs better comments.

Illumos ZFS issues:
  3741 zfs needs better comments

MFC after:      2 weeks
2013-06-11 19:02:36 +00:00
markj
f492c31639 Port the SDT test now that it's possible to create SDT probes that take
seven arguments.

The original test uses Solaris' uadmin system call to trigger the test
probe; this change adds a sysctl to the dtrace_test module and gets the test
program to trigger the test probe via the sysctl handler.

The test is currently failing on amd64 because of some bugs in the way that
probe arguments beyond the first five are obtained - these bugs will be
fixed in a separate change.
2013-06-02 00:33:36 +00:00
markj
b3c31e1254 Remove a block of code that was not intended to be part of the previous
revision.

X-MFC with:	r250812
2013-05-20 02:10:18 +00:00
markj
9e08262228 Re-introduce another part of r249367. This commit fixes a register leak in
dt_cg_ptrsize() and generally cleans up some of the error handling around
register allocation.

This change corresponds to part of illumos-gate commit e5803b76927480:
  3025 register leak in D code generation

Reviewed by:	pfg
Obtained from:	illumos
MFC after:	1 month
2013-05-19 17:14:36 +00:00
markj
e34342dafe Convert a couple of helper scripts used to test the ip provider to work on
FreeBSD. In the IPv6 case, try each interface before returning an error;
each IPv6-enabled interface will have a link-local address even if the link
isn't up.

MFC after:	1 week
2013-05-15 22:56:24 +00:00
markj
08da0b8fad head -n doesn't take negative arguments on FreeBSD, so instead use sed '$d'
to remove the last line of a file.
2013-05-12 16:29:09 +00:00
markj
8f40d761c1 Bring back part of r249367 by adding DTrace's temporal option, which allows
users to guarantee that the output of DTrace scripts will be time-ordered.
This option is enabled by adding the line

  #pragma D option temporal

to the beginning of a script, or by adding '-x temporal' to the arguments of
dtrace(1).

This change fixes a bug in the original port of the temporal option. This
bug was causing some assertions to fail, so they had been disabled; in this
revision the assertions are working properly and are enabled.

The DTrace version number has been bumped from 1.9.0 to 1.9.1 to reflect
the language change that's being introduced.

This change corresponds to part of illumos-gate commit e5803b76927480:
  3021 option for time-ordered output from dtrace(1M)

Reviewed by:	pfg
Obtained from:	illumos
MFC after:	1 month
2013-05-12 16:26:33 +00:00
brooks
dda6e06873 Work around the implementation of LIBRARIES_ONLY. It causes drti.o to
not be installed in /usr/lib32 on systems with compat-32 support.

This fix has two parts.  First, the build is forced by linking drti.o
into a dummy internal library.  Second, the object file is installed
manually in the LIBRARIES_ONLY case.

MFC after:	3 days
2013-05-07 22:05:57 +00:00
avg
8fa4324396 revert r248644 because of the regression for usdt probes
USDT probes are advertised to kernel by initialization code with
atrribute((constructor))).  It seems that on Solaris the .init-ish code
of the main object is executed before RD_PREINIT point is hit.  On
FreeBSD that is not the case.  And because on FreeBSD there is no other
well-defined point between RD_PREINIT and main() we have to parse a
DTrace script when main is hit, for time being.

A footnote: currently we actually post RD_POSTINIT event, but that's a
bug because the event is triggered by hitting r_debug_state which
happens before any init code is executed.

Reported by:	markj
2013-04-25 07:04:56 +00:00
mm
edefb5410f Respect the enoent_ok flag if reporting error for holding an non-existing
snapshot.

Related illumos ZFS issue:
  3699 zfs hold or release of a non-existent snapshot does not output error

Reported by:	Steven Hartland <smh@FreeBSD.org>
MFC after:	3 days
2013-04-25 06:55:57 +00:00
ed
e342f91158 Fix -Wmissing-variable-declarations compiler warnings.
References:
https://www.illumos.org/issues/3700
2013-04-19 19:38:39 +00:00
pfg
6f17eab492 DTrace: NFS translators should be split into client/server pieces
Belatedly update the provider with changes from OpenSolaris:

PSARC 2008/050 DTrace NFS v3 Provider
6696397 NFS v3 provider reports all UDP clients as 0.0.0.0
6711844 assert: vp->v_shrlocks

Obtained from:	Illumos
2013-04-18 01:20:08 +00:00
pfg
d3334c59c7 DTrace: Revert r249426
This change actually depends on r249367 which had to be reverted

Pointy Hat:	pfg
2013-04-17 02:40:07 +00:00
pfg
f9796e10e2 DTrace: Revert r249367
The following change from illumos brought caused DTrace to
pause in an interactive environment:

3026 libdtrace should set LD_NOLAZYLOAD=1 to help the pid provider

This was not detected during testing because it doesn't
affect scripts.

We shouldn't be changing the environment, especially since the
LD_NOLAZYLOAD option doesn't apply to our (GNU) ld.
Unfortunately the change from upstream was made in such a way
that it is very difficult to separate this change from the
others so, at least for now, it's better to just revert
everything.

Reference:
https://www.illumos.org/issues/3026

Reported by:	Navdeep Parhar and Mark Johnston
2013-04-17 02:20:17 +00:00
pfg
c31105ae2f DTrace: print() should try to resolve function pointers
Merge changes from illumos:

3675 DTrace print() should try to resolve function pointers
3676 dt_print_enum hardcodes a value of zero

Illumos Revision:	b1fa6326238973aeaf12c34fcda75985b6c06be1

Reference:
https://www.illumos.org/issues/3675
https://www.illumos.org/issues/3676

Obtained from:	Illumos
MFC after:	1 month
2013-04-16 19:39:27 +00:00
pjd
a87381e520 Correct error message.
Reported by:	Dirk Engling <erdgeist@erdgeist.org>
2013-04-16 12:31:16 +00:00
pfg
7eee57bac2 DTrace: NFS translators should be split into client/server pieces
Merge change from illumos:

1731 DTrace NFS translators should be split into client/server pieces

Illumos Revision:	13523:6763769941d2

This code seems to be currently unused on FreeBSD.

Reference:
https://www.illumos.org/issues/1731

Obtained from:	Illumos
MFC after:	1 week
2013-04-15 20:16:31 +00:00
pfg
0132d9f8dc Dtrace: resolve const types from fbt and other fixes.
Merge change from illumos:

3519 DTrace fails to resolve const types from fbt
3520 dtrace internal error -- token type 316 is not a valid D
     compilation token
3521 clean up dtrace unit tests

Illumos Revision:	e98f46c

Reference:
https://www.illumos.org/issues/3519
https://www.illumos.org/issues/3520
https://www.illumos.org/issues/3521

Tested by:	Fabian Keil
Obtained from:	Illumos
MFC after:	1 month
2013-04-12 20:10:27 +00:00
pfg
2554e81a37 DTrace: option for time-ordered output
Merge changes from illumos:

3021 option for time-ordered output from dtrace(1M)
3022 DTrace: keys should not affect the sort order when sorting by value
3023 it should be possible to dereference dynamic variables
3024 D integer narrowing needs some work
3025 register leak in D code generation
3026 libdtrace should set LD_NOLAZYLOAD=1 to help the pid provider

This brings yet another feature implemented in upstream DTrace.
A complete description is available here:
http://dtrace.org/blogs/ahl/2012/07/28/my-new-dtrace-favorite/

This change bumps the DT_VERS_* number to 1.9.1 in
accordance to what is done in illumos.

This change was somewhat complicated because upstream is mixed many
changes in an individual commit and some of the tests don't really
apply to us.

There are also appear to be differences in timestamping with Solaris
so we had to workaround some assertions making sure no regression
happened.

Special thanks to Fabian Keil for changes and testing.

Illumos Revisions:	13758:23432da34147

Reference:
https://www.illumos.org/issues/3021
https://www.illumos.org/issues/3022
https://www.illumos.org/issues/3023
https://www.illumos.org/issues/3024
https://www.illumos.org/issues/3025
https://www.illumos.org/issues/1694

Tested by:	Fabian Keil
Obtained from:	Illumos
MFC after:	1 months
2013-04-11 16:24:36 +00:00
mm
1a822a3e4d Fix libzfs to report error instead of returning zero if trying to hold or
release a non-existing snapshot of a existing dataset. In recursive case
error is reported if no snapshots with the requested name have been found.

Problem and proposed solution reported to illumos:
  3699 zfs hold or release of a non-existent snapshot does not output error

MFC after:	8 days
2013-04-11 07:49:16 +00:00
mm
6cbe43292e ZFS expects a copyout of zfs_cmd_t on an ioctl error. Our sys_ioctl()
doesn't copyout in this case.

To solve this issue a new struct zfs_iocparm_t is introduced consisting of:
- zfs_ioctl_version (future backwards compatibility purposes)
- user space pointer to zfs_cmd_t (copyin and copyout)
- size of zfs_cmd_t (verification purposes)

The copyin and copyout of zfs_cmd_t is now done the illumos (vendor) way
what makes porting of new changes easier and ensures correct behavior if
returning an error.

MFC after:	10 days
2013-04-09 22:27:44 +00:00
mm
b78a61d3ba MFV r249185:
Allow zdb to output a histogram of compressed block sizes.

Illumos ZFS issues:
  3641 want a histogram of compressed block sizes

MFC after:	3 weeks
2013-04-06 17:18:48 +00:00
mm
696d961cbd MFV r248217:
Merge change from vendor to reduce diff only.
ZFS dtrace probes are not supported on FreeBSD yet.

Illumos ZFS issues:
  3598 want to dtrace when errors are generated in zfs

MFC after:	3 weeks
2013-04-06 10:39:38 +00:00
dim
ed91507629 Follow up to r247960 and rr247960 by also amending ctfmerge. For the
only other case where STT_FILE symbols are used, in symit_next() in
cddl/contrib/opensolaris/tools/ctf/cvt/input.c, save the basename of the
symbol, instead of the full pathname.

Reported by:	avg
Tested by:	avg, jimharris
MFC after:	1 week
2013-04-01 21:16:32 +00:00
pfg
50c97d79d9 Dtrace: enablings on defunct providers prevent providers from unregistering
Merge change from illumos:

1368 enablings on defunct providers prevent providers from unregistering

We try to address some underlying differences between the Solaris
and FreeBSD implementations: dtrace_attach() / dtrace_detach() are
currently unimplemented in FreeBSD but the new code from illumos
makes use of taskq so some adaptations were made to dtrace_open()
and dtrace_close() to handle them appropriately.

Illumos Revision:	r13430:8e6add739e38

Reference:
https://www.illumos.org/issues/1368

Reviewed by:	gnn
Tested by:	Fabian Keil
Obtained from:	Illumos
MFC after:	3 weeks
2013-04-01 19:13:46 +00:00
gnn
9e7e2d6285 Commit a patch that fixes a problem in the #pragma statement when searching
for and loading dependent modules.  This addresses a bug seen with
io.d where it was being doubly included.

PR:		171678
Submitted by:	 Mark Johnston
MFC after:	2 weeks
2013-03-28 20:31:03 +00:00
gnn
d955ec99d7 Revert previous change in favor of a fix to the actual dtrace
libraries that addresses a #pragma issue.
2013-03-28 20:22:43 +00:00
gnn
3793680579 Remove dependency code that caused a double inclusion.
Pointed out by: rpaulo and others
2013-03-28 20:12:46 +00:00
pfg
996e9d6f74 Dtrace: dtrace.c erroneously checks for memory alignment on amd64.
Merge change from illumos:

3511 dtrace.c erroneously checks for memory alignment on amd64

Illumos Revision:	c93cc65

Reference:
https://www.illumos.org/issues/3511

Obtained from:	Illumos
MFC after:	3 weeks
2013-03-26 20:17:08 +00:00
pfg
5304ecc65d Dtrace: Add SUN MDB-like type-aware print() action.
Merge change from illumos:

1694 Add type-aware print() action

This is a very nice feature implemented in upstream Dtrace.
A complete description is available here:
http://dtrace.org/blogs/eschrock/2011/10/26/your-mdb-fell-into-my-dtrace/

This change bumps the DT_VERS_* number to 1.9.0 in
accordance to what is done in illumos.

While here also include some minor cleanups to ease further merging
and appease clang with a fix by Fabian Keil.

Illumos Revisions:	13501:c3a7090dbc16
			13483:f413e6c5d297

Reference:
https://www.illumos.org/issues/1560
https://www.illumos.org/issues/1694

Tested by:	Fabian Keil
Obtained from:	Illumos
MFC after:	1 month
2013-03-25 20:38:09 +00:00
pfg
fe83fcce74 Dtrace: add toupper()/tolower() and enhancements to lltostr().
Merge changes from illumos:

1451 DTrace needs toupper()/tolower() subroutines
1457 lltostr() D subroutine should take an optional base

This change bumps the DT_VERS_* number to 1.8.1 in
accordance to what is done in illumos.

The test suite we currently include is outdated and
doesnt support some updates in tst.subr.d which had to
be left out for now.

Illumos Revisions:	r13458 5e394d8db762
			r13459 c3454574dd1a

Reference:
https://www.illumos.org/issues/1451
https://www.illumos.org/issues/1457

Tested by:	Fabian Keil
Obtained from:	Illumos
MFC after:	1 month
2013-03-25 15:40:57 +00:00
pfg
61cbaa2ebd Dtrace: add optional size argument to tracemem().
Merge change from illumos:

1455 DTrace tracemem() should take an optional size argument

Our local enhancements to dt_print_bytes were equivalent to
those in illumos but we made it match the illumos version
to ease further code merges.

For now leave out tst.smallsize.d and tst.smallsize.d.out
since those don't seem to work cleanly on FreeBSD.

This change bumps the DT_VERS_* number to 1.7.1 in accordance
to what is done in illumos.

Illumos Revision:	13457:571b0355c2e3

Reference:
https://www.illumos.org/issues/1455

Tested by:	Fabian Keil
Obtained from:	Illumos
MFC after:	1 month
2013-03-24 19:12:08 +00:00
avg
880c47f8a7 dtrace: ensure that we can always catch a process (e.g. when -c is used)
It is not guaranteed that a program has a symbol table entry for main
and thus that it would be possible to set a breakpoint on it.

Reviewed by:	rpaulo
Discussed with:	rpaulo
MFC after:	13 days
2013-03-23 08:57:54 +00:00
mm
f1985e5cd7 Merge libzfs_core branch:
includes MFV 238590, 238592, 247580

MFV 238590, 238592:
  In the first zfs ioctl restructuring phase, the libzfs_core library was
  introduced. It is a new thin library that wraps around kernel ioctl's.
  The idea is to provide a forward-compatible way of dealing with new
  features. Arguments are passed in nvlists and not random zfs_cmd fields,
  new-style ioctls are logged to pool history using a new method of
  history logging.

  http://blog.delphix.com/matt/2012/01/17/the-future-of-libzfs/

MFV 247580 [1]:
  To address issues of several deadlocks and race conditions the locking
  code around dsl_dataset was rewritten and the interface to synctasks
  was changed.

User-Visible Changes:
  "zfs snapshot" can create more arbitrary snapshots at once (atomically)
  "zfs destroy" destroys multiple snapshots at once
  "zfs recv" has improved performance

Backward Compatibility:
  I have extended the compatibility layer to support full backward
  compatibility by remapping or rewriting the responsible ioctl arguments.
  Old utilities are fully supported by the new kernel module.

Forward Compatibility:
  New utilities work with old kernels with the following restrictions:
    - creating, destroying, holding and releasing of multiple snapshots
      at once is not supported, this includes recursive (-r) commands

Illumos ZFS issues:
  2882 implement libzfs_core
  2900 "zfs snapshot" should be able to create multiple,
       arbitrary snapshots at once
  3464 zfs synctask code needs restructuring

References:
  https://www.illumos.org/issues/2882
  https://www.illumos.org/issues/2900
  https://www.illumos.org/issues/3464 [1]

MFC after:	1 month
Sponsored by:	Hybrid Logic Inc. [1]
2013-03-21 08:38:03 +00:00
mm
7c87858955 MFV r247580:
Merge synctask code restructuring from vendor.

Modify forward and backward compatibility to support new change.

Illumos ZFS issues:
  3464 zfs synctask code needs restructuring

Sponsored by:	Hybrid Logic Ltd.
2013-03-19 12:51:18 +00:00
mm
09f7f9e4ff MFC @248461 2013-03-18 09:39:51 +00:00
mm
713c2d790d Move common zfs ioctl compatibility functions (userland) into libzfs_compat.c
Introduce additional constants for zfs ioctl versions
2013-03-18 09:32:29 +00:00
mm
da897e350b Fix working with zfs_ioctl_version in libzfs_compat.h and include mirror
lzc_ioctl_version in libzfs_core
2013-03-17 22:24:08 +00:00
mm
3d2567990a Add forwards compatibility for libzfs_core
Unsupported: creation of multiple snapshots including "zfs snapshot -r"
2013-03-17 18:33:06 +00:00
mm
6c7511b96e libzfs_core:
- provide complete backwards compatibility (old utility, new kernel)
  - add zfs_cmd_t compatibility mapping in both directions
  - determine ioctl address in zfs_ioctl_compat.c
2013-03-17 10:57:04 +00:00
mm
d1a8cea992 MFV r248266:
Import minor ZFS changes from vendor

Illumos ZFS issues:
  3604 zdb should print bpobjs more verbosely (fix zdb hang)
  3606 zpool status -x shouldn't warn about old on-disk format

MFC after:	3 days
2013-03-14 10:02:59 +00:00
mm
f7f87457f8 Update zfs.8 manpage date (missing in r247585)
MFC:	together with r247585
2013-03-14 08:18:40 +00:00
mm
ec13ab3f50 MFC @248093 2013-03-09 11:57:51 +00:00
dim
24bac1724c Fix error in r247960: actually assign the basename to match.iim_file.
Pointed out by:	avg
Pointy hat to:	dim
MFC after:	1 week
X-MFC-With:	r247960
2013-03-07 22:43:50 +00:00
dim
be01e90b71 Make ctfconvert work correctly on clang-compiled object files. Clang
puts the full original source filename in the STT_FILE entry of the ELF
symbol table, while gcc saves only the basename.

Since the DWARF DW_AT_name attribute contains the full source filename,
both for clang and gcc, ctfconvert takes just the basename of it, for
matching with the STT_FILE entry.  So when attempting to match with such
an entry, use its basename, if necessary.

Reported by:	avg
MFC after:	1 week
2013-03-07 22:16:35 +00:00
mm
5791f5a0dc Move libzfs compat functions to libzfs_compat.h
This header is used by both libzfs_core and libzfs libraries
2013-03-06 10:40:50 +00:00
mm
9f9f488fa5 MFV r247845:
Import ZFS bpobj bugfix from vendor.

Illumos ZFS issues:
  3603 panic from bpobj_enqueue_subobj()
  3604 zdb should print bpobjs more verbosely

References:
  https://www.illumos.org/issues/3603
  https://www.illumos.org/issues/3604

MFC after:	1 week
2013-03-05 18:54:41 +00:00
mm
99f883783c WiP merge of libzfs_core (MFV r238590, r238592)
not yet working, ioctl handling needs to be changed
2013-03-05 08:09:53 +00:00
mm
2bed8f5691 MFV r247316:
Merge new read-only zfs properties from vendor (illumos)

Illumos ZFS issues:
  3588 provide zfs properties for logical (uncompressed) space used and
       referenced

References:
  https://www.illumos.org/issues/3588

MFC after:	2 weeks
2013-03-01 21:58:51 +00:00
mm
e653c470f7 Fix the zfs_ioctl compat layer to support zfs_cmd size change introduced
in r247265 (ZFS deadman thread). Both new utilities now support the old
kernel and new kernel properly detects old utilities.

For future backwards compatibility, the vfs.zfs.version.ioctl read-only
sysctl has been introduced. With this sysctl zfs utilities will be able
to detect the ioctl interface version of the currently loaded zfs module.

As a side effect, the zfs utilities between r247265 and this revision don't
support the old kernel module. If you are using HEAD newer or equal than
r247265, install the new kernel module (or whole kernel) first.

MFC after:	10 days
2013-03-01 09:42:58 +00:00
mm
18e9a0422b MFV 247176, 247178, 247315:
Import metaslab_sync() speedup from vendor (illumos).

Illumos ZFS issues:
  3552 condensing one space map burns 3 seconds of CPU in spa_sync() thread
  3564 spa_sync() spends 5-10% of its time in metaslab_sync() (when not
       condensing)
  3578 transferring the freed map to the defer map should be constant time
  3579 ztest trips assertion in metaslab_weight()

References:
  https://www.illumos.org/issues/3552
  https://www.illumos.org/issues/3564
  https://www.illumos.org/issues/3578
  https://www.illumos.org/issues/3579

MFC after:	2 weeks
2013-02-27 14:45:23 +00:00
mm
935fd1194a MFV v242732:
Merge the ZFS I/O deadman thread from vendor (illumos).
This feature panics the system on hanging ZFS I/O, helps debugging
and resumes failed service.

The panic behavior can be controlled with the loader-only tunables:
vfs.zfs.deadman_enabled (enable or disable panic on stalled ZFS I/O)
vfs.zfs.deadman_synctime (expiration time for stalled ZFS I/O)

By default, ZFS I/O deadman is enabled by default on amd64 and i386
excluding virtual guest machines.

Illumos ZFS issues:
  3246 ZFS I/O deadman thread

References:
  https://www.illumos.org/issues/3246

MFC after:	2 weeks
2013-02-25 12:33:31 +00:00
gibbs
25887f42a7 Orphaned processes that are being traced are killed by the
kernel.  Properly restore, continue, and detach from processes
being DTraced when DTrace exits with an error so the program
being inspected is not terminated.

cddl/contrib/opensolaris/cmd/dtrace/dtrace.c:
	In fatal(), the generic error handler, close the DTrace
	handle as is done in the "probe/script" error handler
	dfatal().  fatal() can be invoked after DTrace attaches
	to processes (e.g. a script specified by command line
	argument can't be found) and closing the handle will
	release them.

Submitted by:	Spectra Logic Corporation
Reviewed by:	rpaulo, gnn
2013-02-20 17:46:38 +00:00
delphij
7bafa2ca41 Use the right year.
Noticed by:	Denis Ahrens <denis h3q com>
2013-02-11 18:22:26 +00:00
mm
22ab64ccb2 MFV r246392:
Import vendor ZFS bugfix fixing a possible deadlock in arc_read().

Illumos ZFS issues:
  3498 panic in arc_read(): !refcount_is_zero(&pbuf->b_hdr->b_refcnt)

References:
  https://www.illumos.org/issues/3498

MFC after:	2 weeks
2013-02-11 12:42:11 +00:00
mm
81ba0e4f37 MFV r246388:
Import vendor bugfixes

Illumos ZFS issues:
  3422 zpool create/syseventd race yield non-importable pool
  3425 first write to a new zvol can fail with EFBIG

References:
  https://www.illumos.org/issues/3422
  https://www.illumos.org/issues/3425

MFC after:	2 weeks
2013-02-10 19:32:55 +00:00
mm
522c6ecda0 Backport vendor changes in zfs(8) manual page (MFV r246389)
Illumos ZFS issues:
  3380 zfs man page: documentation for zfs allow is confusing

References:
  https://www.illumos.org/issues/3380

MFC after:	2 weeks
2013-02-10 17:10:07 +00:00
gavin
aade44b4e0 Correct spelling of "daemon". No .Dd bump.
Noticed by:	Nathan Rich <Nathan.Rich dynastysystems com>
MFC after:	3 days
2013-02-10 14:28:07 +00:00
joel
538377b2ea mdoc: Remove EOL whitespace. 2013-02-09 07:01:05 +00:00
delphij
c6d9dc1ad2 MFV r245512:
* Illumos zfs issue #3035 [1] LZ4 compression support in ZFS.

LZ4 is a new high-speed BSD-licensed compression algorithm created
by Yann Collet that delivers very high compression and decompression
performance compared to lzjb (>50% faster on compression, >80% faster
on decompression and around 3x faster on compression of incompressible
data), while giving better compression ratio [1].

This version of LZ4 corresponds to upstream's [2] revision 85.

Please note that for obvious reasons this is not backward read
compatible.  This means once a pool have LZ4 compressed data, these
data can no longer be read by older ZFS implementations.

Local changes:

 - On-stack hash table disabled and using kernel slab allocator
   instead, at this time.  This requires larger kernel thread stack
   for zio workers.  This may change in the future should we adjusted
   the zio workers' thread stack size.
 - likely and unlikely will be undefined if they are already defined,
   this is required for i386 XEN build.
 - Removed De Bruijn sequence based __builtin_ctz family of builtins
   in favor of the latter.  Both GCC and clang supports these builtins.
 - Changed the way the LZ4 code detects endianness.
 - Manual pages modifications to mention the feature based on Illumos
   counterpart.
 - Boot loader changes to make it support LZ4 decompression.

[1] https://www.illumos.org/issues/3035
[2] http://code.google.com/p/lz4/source/list

Obtained from:	Illumos (13921:9d721847e469)
Tested on:	FreeBSD/amd64
MFC after:	1 month
2013-02-09 06:39:28 +00:00
brooks
2434ed6f85 Add a new LIBRARIES_ONLY make variable to disable the build and install
of files other than the actual libraries.

Use LIBRARIES_ONLY to supress the inclusion of files in the lib32
distribution that are duplicates of files in base.

Sponsored by:	DARPA, AFRL
Reviewed by:	emaste
2013-01-17 17:27:10 +00:00
smh
cfd489a4ed Reports pools which have a removed l2cache disk under -x as this is what
happens when a cache device is dropped for any reason.

Reviewed by:	pjd
Approved by:	pjd (mentor)
MFC after:	2 weeks
2013-01-15 23:34:24 +00:00
delphij
f8200d9bc4 The current ZFS code expects ddt_zap_count to always succeed by asserting
the underlying zap_count() to return no errors.  However, it is possible
that the pool reaches to such a state where zap_count would return error,
leading to panics when a pool is imported.

This commit changes the ddt_zap_count to return error returned from
zap_count and handle the error appropriately.  With this change, it's now
possible to let zpool rollback damaged transaction groups and import the
pool.

Obtained from:	ZFS on Linux github (e8fd45a0f975c6b8ae8cd644714fc21f14fac2bf)
MFC after:	1 month
2013-01-10 19:26:56 +00:00
pjd
f250bca4bb Allow to create pool even if mount point directory is not empty if -f is given.
Obtained from:	WHEEL Systems
2012-12-30 13:47:40 +00:00
mm
88ab17dd2b MFV r244245:
Merge two zdb bugfixes from vendor (illumos)

illumos-gate 13894:f4af77f6bbd2
3397 zdb <pool> <objnum> output is too verbose
3398 zdb can't dump feature flags zap objects

References:
https://www.illumos.org/issues/3397
https://www.illumos.org/issues/3398

MFC after:	1 week
2012-12-15 10:02:11 +00:00
smh
4995789cde Fixes zfs receive errors caused by snapshot replication being processed in a
random order instead of creation order.

Eliminates needless filesystem renames caused by removed parent snapshots
which subsequently causes many more errors.

PR:		kern/172259
Submitted by:	Steven Hartland
Reviewed by:	pjd (mentor)
Approved by:	pjd (mentor)
MFC after:	2 weeks
2012-12-13 22:03:07 +00:00
mm
7b9ba9c9b6 Update manpage dates in zfs.8 and zpool.8
MFC after:	2 weeks
2012-11-26 13:16:55 +00:00
mm
f906bfd159 MFV r243395:
Introduce a new dataset aclmode setting "restricted" to protect ACL's
being destroyed or corrupted by a drive-by chmod.

illumos-gate 13889:a67716f16746
3254 add support in zfs for aclmode=restricted

References:
https://www.illumos.org/issues/3254

MFC after:	2 weeks
2012-11-26 12:24:39 +00:00
mm
3a0bfecf05 MFV r243013 and r243267:
Import the zio nop-write improvement from Illumos. To reduce I/O,
nop-write omits overwriting data if the checksum (cryptographically
secure) of new data matches the checksum of existing data.
It also saves space if snapshots are in use.

It currently works only on datasets with enabled compression, disabled
deduplication and sha256 checksums.

IllumOS 13887:196932ec9e6a and 13888:7204b3392a58
3236 zio nop-write

References:
https://www.illumos.org/issues/3236

MFC after:	2 weeks
2012-11-25 16:32:07 +00:00
mm
5df7a1a9b3 zfs sha256 checksum is missing in zfs.8 manpage
MFC after:	3 days
2012-11-25 11:26:36 +00:00
mm
82387c4bfa MFV r243012:
Illumos 13886:e3261d03efbf

3349 zpool upgrade -V bumps the on disk version number, but leaves
     the in core version

References:
https://www.illumos.org/issues/3349

MFC after:	1 week
2012-11-25 10:53:42 +00:00
pluknet
24f2bde267 Sort SEE ALSO xrefs by the manual section number and add a missing comma. 2012-11-15 20:50:18 +00:00
mm
add6fbdfee Move zpool-features manual page from section 5 to section 7
and fix references

Reported by:	pluknet
MFC after:	1 week
2012-11-14 01:16:29 +00:00
pluknet
a7147f0cfb Fix an apparent typo in the manual section number in .Dt: it should be 5. 2012-11-13 21:08:49 +00:00
delphij
8e1fdec2c7 MFV r242729 (mm):
Illumos r13840:97fd5cdf328a:

3145 single-copy arc
3212 ztest: race condition between vdev_online() and spa_vdev_remove()

Illumos r13849:3468a95b27cd:

3258 ztest's use of file descriptors is unstable
2012-11-10 01:52:52 +00:00
jhibbits
99d6a5644c Implement DTrace for PowerPC. This includes both 32-bit and 64-bit.
There is one known issue:  Some probes will display an error message along the
lines of:  "Invalid address (0)"

I tested this with both a simple dtrace probe and dtruss on a few different
binaries on 32-bit.  I only compiled 64-bit, did not run it, but I don't expect
problems without the modules loaded.  Volunteers are welcome.

MFC after:	1 month
2012-11-07 23:45:09 +00:00
kib
560aa751e0 Remove the support for using non-mpsafe filesystem modules.
In particular, do not lock Giant conditionally when calling into the
filesystem module, remove the VFS_LOCK_GIANT() and related
macros. Stop handling buffers belonging to non-mpsafe filesystems.

The VFS_VERSION is bumped to indicate the interface change which does
not result in the interface signatures changes.

Conducted and reviewed by:	attilio
Tested by:	pho
2012-10-22 17:50:54 +00:00
mm
c67bd49a18 Add missing initialization for do_prefix.
Corrects porting error in r238391

Vendor issue and changeset reference:
2883 changing "canmount" property to "on" should not always remount dataset
https://www.illumos.org/issues/2883
Changeset 13743:95aba6e49b9f

Reported by:	Guido Falsi <mad@madpilot.net>, avg
Obtained from:	illumos (issue #2883)
MFC after:	1 week
2012-10-17 22:14:09 +00:00
kevlo
a93e845e54 Make sure that each va_start has one and only one matching va_end,
especially in error cases.
2012-09-28 07:51:30 +00:00
mm
e0125673c3 Merge recent vendor changes in ZFS.
Illumos issued covered:
2811 missing implementation: zfs send -r
3139 zdb dies when it tries to determine path of unlinked file
3189 kernel panic in ZFS test suite during hotspare_onoffline_004_neg
3208 moving zpool cross-endian results in incorrect user/group accounting

References:
  https://www.illumos.org/issues/ + [issue_id]

Obtained from:	illumos (vendor/illumos, vendor/illumos-sys)
MFC after:	2 weeks
2012-09-26 09:37:58 +00:00
pjd
212634d921 It is possible to recursively destroy snapshots even if the snapshot
doesn't exist on a dataset we are starting from. For example if we
have the following configuration:

	tank
	tank/foo
	tank/foo@snap
	tank/bar
	tank/bar@snap

We can execute:

	# zfs destroy -t tank@snap

eventhough tank@snap doesn't exit.

Unfortunately it is not possible to do the same with recursive rename:

	# zfs rename -r tank@snap tank@pans
	cannot open 'tank@snap': dataset does not exist

...until now. This change allows to recursively rename snapshots even if
snapshot doesn't exist on the starting dataset.

Sponsored by:	rsync.net
MFC after:	2 weeks
2012-09-23 20:12:10 +00:00
pjd
618888b019 Add TRIM support.
The code builds a map of regions that were freed. On every write the
code consults the map and eventually removes ranges that were freed
before, but are now overwritten.

Freed blocks are not TRIMed immediately. There is a tunable that defines
how many txg we should wait with TRIMming freed blocks (64 by default).

There is a low priority thread that TRIMs ranges when the time comes.
During TRIM we keep in-flight ranges on a list to detect colliding
writes - we have to delay writes that collide with in-flight TRIMs in
case something will be reordered and write will reached the disk before
the TRIM. We don't have to do the same for in-flight writes, as
colliding writes just remove ranges to TRIM.

Sponsored by:	multiplay.co.uk

This work includes some important fixes and some improvements obtained
from the zfsonlinux project, including TRIMming entire vdevs on pool
create/add/attach and on pool import for spare and cache vdevs.

Obtained from:	zfsonlinux
Submitted by:	Etienne Dechamps <etienne.dechamps@ovh.net>
2012-09-23 19:40:58 +00:00
bapt
ba191119a0 Update usage to show the possiblity to use jail name
Reported by:	bdrewery
MFC after:	1 month
2012-09-19 16:28:38 +00:00
bapt
13681004dd Allow zfs jail and zfs unjail to use both jailnames and jailid
Reviewed by:	pjd
MFC after:	1 month
2012-09-19 16:20:49 +00:00
mm
063b8917f2 Merge recent zfs vendor changes, sync code and adjust userland DEBUG.
Illumos issued covered:
1884 Empty "used" field for zfs *space commands
3006 VERIFY[S,U,P] and ASSERT[S,U,P] frequently check if first argument
     is zero
3028 zfs {group,user}space -n prints (null) instead of numeric GID/UID
3048 zfs {user,group}space [-s|-S] is broken
3049 zfs {user,group}space -t doesn't really filter the results
3060 zfs {user,group}space -H output isn't tab-delimited
3061 zfs {user,group}space -o doesn't use specified fields order
3064 usr/src/cmd/zpool/zpool_main.c misspells "successful"
3093 zfs {user,group}space's -i is noop
3098 zfs userspace/groupspace fail without saying why when run as non-root

References:
  https://www.illumos.org/issues/ + [issue_id]

Obtained from:	illumos (vendor/illumos, vendor/illumos-sys)
MFC after:	2 weeks
2012-09-12 18:05:43 +00:00
joel
88d971cbf7 Remove trailing whitespace. 2012-09-06 19:26:59 +00:00
gjb
66a2965d99 Typo fix and minor word swap.
PR:		171356
Submitted by:	bdrewery
MFC After:	3 days
2012-09-05 23:07:02 +00:00
mm
3a59a28395 Merge recent vendor changes and sync code:
1862 incremental zfs receive fails for sparse file > 8PB
3112 ztest does not honor ZFS_DEBUG
3122 zfs destroy filesystem should prefetch blocks
3129 'zpool reopen' restarts resilvers
3130 ztest failure: Assertion failed:
       0 == dmu_objset_destroy(name, B_FALSE) (0x0 == 0x10)

References:
  https://www.illumos.org/issues/1862
  https://www.illumos.org/issues/3112
  https://www.illumos.org/issues/3122
  https://www.illumos.org/issues/3129
  https://www.illumos.org/issues/3130

Obtained from:	illumos (vendor/illumos, vendor/illumos-sys)
MFC after:	2 weeks
2012-09-05 12:02:09 +00:00
gjb
9a1f51023a Add myself to copyright sections, per CDDL license.
Requested by:	mm
2012-09-03 00:05:21 +00:00
rpaulo
d430c71377 Finish porting execsnoop to FreeBSD. This includes replacing the zonename
with a jail ID and removing the project ID from the list of options.
2012-09-01 08:14:21 +00:00
joel
43f4c235d8 Remove trailing whitespace. 2012-09-01 06:23:13 +00:00
joel
e79df87ad2 Remove trailing whitespace. 2012-09-01 06:12:14 +00:00
joel
1a002b0a86 Mdoc fixes. 2012-09-01 06:07:27 +00:00
joel
44c1b764b9 Minor mdoc fixes. 2012-08-31 21:44:12 +00:00
joel
82ce050985 Mdoc fixes. 2012-08-31 17:18:41 +00:00
mm
d9fa2f486e Merge recent vendor changes:
3100 zvol rename fails with EBUSY when dirty
3104 eliminate empty bpobjs
3120 zinject hangs in zfsdev_ioctl() due to uninitialized zc

References:
  https://www.illumos.org/issues/3100
  https://www.illumos.org/issues/3104
  https://www.illumos.org/issues/3120

Obtained from:	illumos (vendor/illumos, vendor/illumos-sys)
MFC after:	2 weeks
2012-08-28 12:25:37 +00:00
mm
2cbcbfd502 Add missing parts to zpool-features.5 manual page
Obtained from:	vendor/illumos/dist
MFC after:	2 weeks
2012-08-27 19:52:55 +00:00
mm
587ce5b6a3 Merge recent vendor changes:
3086 unnecessarily setting DS_FLAG_INCONSISTENT on async destroyed datasets
3090 vdev_reopen() during reguid causes vdev to be treated as corrupt
3102 vdev_uberblock_load() and vdev_validate() may read the wrong label

Referenes:
  https://www.illumos.org/issues/3086
  https://www.illumos.org/issues/3090
  https://www.illumos.org/issues/3102

PR:		kern/170912, kern/170914
Obtained from:	illumos (changeset #13776, #13777)
MFC after:	2 weeks
2012-08-23 19:32:57 +00:00
mm
5b60c129a2 Update zfs(8) manpage with illumos version of "zfs diff"
Illumos issue:
  2399 zfs manual page does not document use of "zfs diff"

References:
  https://www.illumos.org/issues/2399

PR:		docs/170764
Obtained from:	ssh://anonhg@hg.illumos.org/illumos-gate
MFC after:	1 week
2012-08-19 10:34:40 +00:00
gjb
c31cc6a5ee - Fix source dataset snapshot name in Example 15.
- Bump date.

MFC after:	3 days
X-MFC-With:	r239216
2012-08-12 20:30:15 +00:00
gjb
a97ae3c276 Remove a leading space that breaks rendering.
MFC after:	3 days
2012-08-12 20:23:59 +00:00
gnn
3c9b732c82 When we return with an error we cannot unlock the mutex, because
it's been freed.  Protect against that, hopefully unlikely, case.

Reviewed by:	rpaulo
MFC after: 2 weeks
2012-08-01 19:27:12 +00:00
mm
afdda0ef5d Fix wrong indent according to style(9)
MFC after:	2 weeks
> Description of fields to fill in above:                     76 columns --|
> PR:            If a GNATS PR is affected by the change.
> Submitted by:  If someone else sent in the change.
> Reviewed by:   If someone else reviewed your modification.
> Approved by:   If you needed approval for this commit.
> Obtained from: If the change is from a third party.
> MFC after:     N [day[s]|week[s]|month[s]].  Request a reminder email.
> Security:      Vulnerability reference (one per line) or description.
> Empty fields above will be automatically removed.

M    zpool_main.c
2012-07-31 17:32:28 +00:00
mm
caced023cd Fix reporting of root pool upgrade notice.
MFC after:	2 weeks
2012-07-31 17:28:28 +00:00
mm
cf6d705d5b Partial MFV (illumos-gate 13753:2aba784c276b)
2762 zpool command should have better support for feature flags

References:
https://www.illumos.org/issues/2762

MFC after:	2 weeks
2012-07-30 23:14:24 +00:00
gnn
2c77f8a514 Revert previous commit. The bug was actually caused by an issue
in pre 1.8.5 versions of sudo which were sending too many
SIGINTs to processes when the user hit Ctrl-C.

Pointed out by:	avg@, rpaulo@, sbruno@
2012-07-25 17:49:01 +00:00
gnn
2046a3869a Fix a bug in interrupt handling so that we're only considered
impatient if we sent more than 2 INT signals.  This fixes a bug where
we wouldn't see aggregations print on the command line if we Ctrl-C'd
a dtrace script or command line invocation.

MFC after:	2 weeks
2012-07-24 18:01:28 +00:00
pfg
d75ae2f261 Dtrace: improve handling of library paths.
Merge changes from illumos

906 dtrace depends_on pragma should search all library paths, not just the
current one

949 dtrace should only include the first instance of a library found on
its library path

Illumos Revisions:	13353:936a1e45726c
			13354:2b2c36a81512

Reference:
https://www.illumos.org/issues/906
https://www.illumos.org/issues/949

Tested by:	Fabian Keil
Obtained from:	Illumos
MFC after:	3 weeks
2012-07-17 19:57:34 +00:00
pfg
c006769183 Use libc's strndup() instead of Dtrace's reimplementation.
Corresponds partially to OpenSolaris change:
PSARC 2010/299 GNU/Linux/BSD compatibility functions
6901783 strndup would be nice

MFC after:	2 weeks
2012-07-15 21:20:31 +00:00
pfg
8163e6ee8c Merge illumos commit 13455:7205f7794835
1458 D compiler fails to generate error on sizeof() an undefined struct

Reference:
https://www.illumos.org/issues/1458

Tested by:	Fabian Keil
MFC after:	3 weeks
2012-07-14 20:24:00 +00:00
mm
98fa6a1cdb Merge illumos commit 13749:df4cd82e2b60
1796 "ZFS HOLD" should not be used when doing "ZFS SEND" froma read-only pool
2871 support for __ZFS_POOL_RESTRICT used by ZFS test suite
2903 zfs destroy -d does not work
2957 zfs destroy -R/r sometimes fails when removing defer-destroyed snapshot

References:
https://www.illumos.org/issues/1796
https://www.illumos.org/issues/2871
https://www.illumos.org/issues/2903
https://www.illumos.org/issues/2957

MFC after:	1 week
2012-07-13 12:37:34 +00:00
mm
b81339c49f Change behavior introduced in r237119 to vendor solution
References:
https://www.illumos.org/issues/2883

PR:		167905
Obtained from:	illumos (issue #2883)
MFC after:	2 weeks
2012-07-12 06:29:54 +00:00
gnn
786ac82553 Initial commit of an I/O provider for DTrace on FreeBSD.
These probes are most useful when looking into the structures
they provide, which are listed in io.d.  For example:

dtrace -n 'io:genunix::start { printf("%d\n", args[0]->bio_bcount); }'

Note that the I/O systems in FreeBSD and Solaris/Illumos are sufficiently
different that there is not a 1:1 mapping from scripts that work
with one to the other.
MFC after:	1 month
2012-07-11 16:27:02 +00:00
emaste
a4bcd4999b Restore r211786 by rpaulo:
Port dtruss to FreeBSD.

  Sponsored by:   The FreeBSD Foundation

It appears the change was reverted by r235380.
2012-07-05 00:52:23 +00:00
pjd
5caca3f8a7 Fix an obvious typo.
MFC after:	3 days
2012-07-04 17:36:26 +00:00
dim
8c66c622bb Fix clang warning, introduced in the recent dtrace import.
MFC after:	   3 days
2012-07-03 12:08:55 +00:00
pfg
a4279bd26d Safer fix for building with Clang.
error: format specifies type 'long long' but the argument
has type 'int64_t' (aka 'long')

Reported by:	Ed Maste
2012-06-28 16:44:29 +00:00
pfg
2bb5abedc9 Fix build with Clang.
error: format specifies type 'long long' but the argument
has type 'int64_t' (aka 'long')

Reported by:	Ed Maste
2012-06-28 15:38:14 +00:00
pfg
bc23332ce7 Bring llquantize support into Dtrace.
Bryan Cantrill implemented the equivalent of semi-log graph
paper for Dtrace so llquantize will use one logarithmic and
one linear scale.

Special thanks to Mark Peek for providing fix to an
assertion and to Fabian Keill for testing the port.

Illumos Revision:	13355:15b74a2a9a9d

Reference:
https://www.illumos/issues/905

Obtained from:	Illumos
Tested by:	Fabian Keill, mp
MFC after:	4 days
2012-06-27 04:39:30 +00:00
mm
5e0236a3e4 Do not remount ZFS dataset if changing canmount property to "on" and
dataset is already mounted.

PR:		167905
Submitted by:	Bryan Drewery <bryan@shatow.net>
MFC after:	1 week
2012-06-15 07:38:21 +00:00
mm
719aef50bd Document the -v flag for zpool list.
PR:		168970
Suggested by:	Marcelo Araujo <araujo@FreeBSD.org>
MFC after:	3 days
2012-06-12 14:40:19 +00:00
mm
cc61ab2f13 Introduce "feature flags" for ZFS pools (bump SPA version to 5000).
Add first feature "com.delphix:async_destroy" (asynchronous destroy
of ZFS datasets).
Implement features support in ZFS boot code.

Illumos revisions merged:
13700:2889e2596bd6
13701:1949b688d5fb
2619 asynchronous destruction of ZFS file systems
2747 SPA versioning with zfs feature flags

References:
https://www.illumos.org/issues/2619
https://www.illumos.org/issues/2747

Obtained from:	illumos (issue #2619, #2747)
MFC after:	1 month
2012-06-11 11:35:22 +00:00
gjb
70a9f0dfc1 Clean up trailing whitespace.
MFC after:	3 days
X-MFC-With:	r236776
2012-06-09 03:34:34 +00:00
gjb
3683163965 Fix a typo: s/deafult/default
MFC after:	3 days
2012-06-09 03:33:06 +00:00
mm
7e5eee4882 Import Illumos revision 13715:351036203e4b
2803 zfs get guid pretty-prints the output

References:
https://www.illumos.org/issues/2803

Obtained from:	illumos (issue #2803)
MFC after:	3 days
2012-06-07 08:32:53 +00:00
gnn
2d93dc867d Add DTrace's io.d, which handles tranlsations for file, buffer and
device info structures as well as the fds[] array.  This is a raw
version of the file, unmodified, to be used as a baseline.
2012-06-05 18:58:05 +00:00
mm
38ef930079 Import illumos changeset 13570:3411fd5f1589
1948 zpool list should show more detailed pool information

Display per-vdev information with "zpool list -v".
The added expandsize property has currently no value on FreeBSD.
This changeset allows adding expansion support to individual vdevs
in the future.

References:
https://www.illumos.org/issues/1948

Obtained from:	illumos (issue #1948)
MFC after:	2 weeks
2012-05-27 16:00:00 +00:00
mm
daae7d499d Import illumos changeset 13605:b5c2b5db80d6 (partial)
763 FMD msg URLs should refer to something visible

Replace sun.com URL's with illumos.org

References:
https://www.illumos.org/issues/763

Obtained from:	illumos (issue #763)
MFC after:	1 week
2012-05-27 12:31:57 +00:00
mm
a0cba7035e Import illumos changeset 13564:cf89c0c60496
1946 incorrect formatting when listing output of multiple pools with
zpool iostat -v

References:
https://www.illumos.org/issues/1946

Obtained from:	illumos (issue #1946)
MFC after:	1 week
2012-05-27 12:22:15 +00:00
mm
6a061d782c Import illumos changeset 13571:a5771a96228c
1950 ztest backwards compatibility testing option

References:
https://www.illumos.org/issues/1950

Obtained from:	illumos (issue #1950)
MFC after:	2 weeks
2012-05-27 11:37:24 +00:00
pjd
33d132b84d Correct error message.
MFC after:	3 days
2012-05-25 17:19:30 +00:00
avg
63d85c1c12 zpool_find_import_impl: another /dev/dsk -> /dev fix
This seems to fix zdb -e behavior.

PR:		bin/155104
Submitted by:	swell.k@gmail.com
MFC after:	2 weeks
2012-05-15 17:11:00 +00:00
avg
334fd301de zpool_do_import: use /dev instead of /dev/dsk as a default
This affects behavior of zpool import without -d option.

Reported by:	Bruce Cran <bruce@cran.org.uk>
Submitted by:	Fabian Keil <freebsd-listen@fabiankeil.de>
MFC after:	2 weeks
2012-05-15 17:07:29 +00:00
gnn
4a10f14855 Move sh to the correct path for FreeBSD. 2012-05-14 22:02:01 +00:00
gnn
457aaa6db7 Move sh to the correct location for FreeBS (/bin/sh) 2012-05-14 22:00:24 +00:00
gnn
31e5e86278 Fix execsnoop by changing exece and exec to be FreeBSD's execve.
Reference sh in the correct location (/bin/sh)
2012-05-14 21:59:47 +00:00
gnn
1f15977bf4 Fix opensnoop for FreeBSD by removing probes with 64 at the end as
these are unnecessary.

Reference sh in the correct location (/bin/sh)
2012-05-14 21:58:22 +00:00
gnn
2325662608 Import dtracetoolkit into cddl/contrib 2012-05-12 21:25:48 +00:00
gnn
0778d9c814 Revert previous failed cp. 2012-05-12 21:23:31 +00:00
gnn
34bfc80a65 Import dtracetoolkit into cddl/contrib 2012-05-12 21:15:21 +00:00
mm
046ff89626 Import illumos changeset 13686:4bc0783f6064
2703 add mechanism to report ZFS send progress

If the zfs send command is used with the -v flag, the amount of bytes
transmitted is reported in per second updates.

References:
https://www.illumos.org/issues/2703

Obtained from:	illumos (issue #2703)
MFC after:	2 weeks
2012-05-10 10:39:45 +00:00
mm
0417257746 Remove unreachable break in zfs_main.c
Partial import of illumos changeset 13622:e5889df1eaac
2077 lots of unreachable breaks in illumos gate

References:
https://www.illumos.org/issues/2077

PR:		kern/167370
Submitted by:	Marcelo Araujo <araujo@FreeBSD.org>
Obtained from:	illumos (issue #2077)
MFC after:	1 week
2012-05-10 09:10:31 +00:00
mm
e290b949d0 Add support for force unmounting ZFS filesystems during "zfs rename"
with the -f flag.

Reimplementation of the illumos changeset 13677:a0cbef703c12
2635 'zfs rename -f' to perform force unmount

References:
https://www.illumos.org/issues/2635

PR:		kern/164447
Suggested by:	Marcelo Araujo <araujo@FreeBSD.org>
Obtained from:	illumos (issue #2635)
MFC after:	1 week
2012-05-10 08:57:58 +00:00
mm
10f60d496c Import illumos changeset 13618:c6ae14a341e8 [1]:
2088 zdb could use a reasonable manual page

Rewrite new zdb manpage to mdoc(7)

References:
https://www.illumos.org/issues/2088

PR:		kern/167467 [1]
Submitted by:	Marcelo Araujo <araujo@FreeBSD.org> [1]
Reviewed by:	mm
Obtained from:	Illumos (issue #2088)
MFC after:	1 week
2012-05-10 04:11:39 +00:00
gjb
c12b44c606 General mdoc(7) and typo fixes.
PR:		167696
Submitted by:	Nobuyuki Koganemaru (kogane!jp.freebsd.org)
MFC after:	3 days
2012-05-10 02:07:00 +00:00
rmh
392fe85344 GNU/kFreeBSD portability fix. This glue <stdlib.h> overrides header protection
in the system-wide version of <stdlib.h> by wrapping the #include_next
<stdlib.h> within the scope of its own header protection. On FreeBSD this has
no effect, since both header protections are equivalent. However the GNU version
of <stdlib.h> implements a special header protection mechanism which allows it
to be included multiple times (in different modes).

Simply by moving the #include_next off the header protection, we allow
system-wide <stdlib.h> to implement its own protection policy, whichever that
may be.
2012-04-30 23:12:16 +00:00
rstone
ad032fccda Implement the D "cpu" variable, which returns curcpu. I have chosen not
to follow the example of OpenSolaris and its descendants, which implemented
cpu as an inline that took a value out of curthread.  At certain points in
the FreeBSD scheduler curthread->td_oncpu will no longer be valid (in
particukar, just before the thread gets descheduled) so instead I have
implemented this as its own built-in variable.

Sponsored by:	Sandvine Inc.
MFC after:	1 week
2012-04-26 01:07:03 +00:00
pjd
35c58230e2 Add -u option to 'zfs create' that prevents file system from being
automatically mounted. This is similar to the 'zfs receive -u'.

MFC after:	1 week
2012-04-24 13:44:46 +00:00
delphij
37179aa09a - Correct a typo which prevents 'lzjb' be displayed;
- Use quotes when tab is used.
2012-04-22 07:50:24 +00:00
mm
1e792bf1e7 Fix typo miror -> mirror
Reported by:	Glen Barber <gjb@FreeBSD.org>
MFC after:	3 days
2012-04-16 08:19:43 +00:00
gnn
9cc42fd8d2 Change SIGUSR1 to SIGTHR to properly wake up a process that is being
traced.  The use of SIGUSR1 caused traced processes (those attached to
with dtrace -p) to exit when dtrace exited.

MFC in: 1 week
2012-04-13 20:22:08 +00:00
gonzo
a704c8860f Enable build of DTrace-related userland parts for MIPS 2012-03-24 05:29:07 +00:00
gonzo
93809282b9 Add stub file for pid probe. It's required although pid probe is not supported
on MIPS yet
2012-03-24 05:27:39 +00:00
gonzo
58eb4eb2a1 Maintain target's byte order for multi-byte fields in CTF structures.
CTF format is not cross-platform by design, e.g. it is not guaranteed
that data generated by ctfconvert/ctfmerge on one architecture will
be successfuly read on another. CTF structures are saved/restored
using naive approach. Roughly it looks like:
	write(fd, &ctf_struct, sizeof(ctf_struct))
	read(fd, &ctf_struct, sizeof(ctf_struct))

By sheer luck memory layout of all type-related CTF structures is the same
on amd64/i386/mips32/mips64. It's different on ARM though. sparc, ia64,
powerpc, and powerpc64 were not tested. So in order to get file compatible
with dtrace on ARM it should be compiled on ARM. Alternative solution would
be to have "signatures" for every platform and ctfmerge should convert host's
reperesentation of CTF structure to target's one using "signature" as template.

This patch checks byte order of ELF files used for generating CTF record
and makes sure that byte order of data written to resulting files is the same
as target's byte order.
2012-03-24 01:47:33 +00:00
mm
d974ef7be1 Analogous to r232059, add a parameter for the ZFS file system:
allow.mount.zfs:
	allow mounting the zfs filesystem inside a jail

This way the permssions for mounting all current VFCF_JAIL filesystems
inside a jail are controlled wia allow.mount.* jail parameters.

Update sysctl descriptions.
Update jail(8) and zfs(8) manpages.

TODO:	document the connection of allow.mount.* and VFCF_JAIL for kernel
	developers

MFC after:	10 days
2012-02-26 16:30:39 +00:00
mm
4893c2aa29 Import illumos changeset 13608 [1]:
add support for "-t <datatype>" argument to zfs get

References:
https://www.illumos.org/issues/1936

Update zfs(8) manpage in respect of [1].
Fix typo in zfs(8) manpage.

Obtained from:	illumos (issue #1936)
MFC after:	1 week
2012-02-23 19:13:19 +00:00
mm
e0ebe286fb Merge illumos revision 13594:
uninitialized variables in zfs(8) may make snapshots undestroyable [1]

References:
https://www.illumos.org/issues/2067 [1]

Obtained from:	illumos (issue #2067)
MFC after:	3 days
2012-02-07 18:35:18 +00:00
pfg
be60b495b2 Set SVN text/plain property for some shell scripts that
happen to have a .exe extension.

While here fix the shebang of a shell script that
was looking for /bin/bash.

Reviewed by:	gnn
Approved by:	jhb (mentor)
MFC after:	2 weeks
2012-01-28 00:17:17 +00:00
kmacy
e956af45d2 add KM_NODEBUG needed by ARC buffer core dump exclusion change 2012-01-27 22:24:03 +00:00
mm
4f148cb11a Merge illumos revisions 13572, 13573, 13574:
Rev. 13572:
disk sync write perf regression when slog is used post oi_148 [1]

Rev. 13573:
crash during reguid causes stale config [2]
allow and unallow missing from zpool history since removal of pyzfs [5]

Rev. 13574:
leaking a vdev when removing an l2cache device [3]
memory leak when adding a file-based l2arc device [4]
leak in ZFS from metaslab_group_create and zfs_ereport_checksum [6]

References:
https://www.illumos.org/issues/1909 [1]
https://www.illumos.org/issues/1949 [2]
https://www.illumos.org/issues/1951 [3]
https://www.illumos.org/issues/1952 [4]
https://www.illumos.org/issues/1953 [5]
https://www.illumos.org/issues/1954 [6]

Obtained from:	illumos (issues #1909, #1949, #1951, #1952, #1953, #1954)
MFC after:	2 weeks
2012-01-24 23:09:54 +00:00
mm
f7bacb94be Merge illumos revisions 13540, 13562:
illumos rev 13540 [1]:
Removal of pyzfs broke delegation for volumes

illumos rev 13562 [2]:
zfs allow arguments not parsed correctly after pyzfs removal

References:
https://www.illumos.org/issues/1726 [1]
https://www.illumos.org/issues/1977 [2]

Obtained from:	illumos (issues #1726, #1977)
MFC after:	1 week
2012-01-22 08:06:36 +00:00
pjd
8e3e986346 Dramatically optimize listing snapshots when user requests only snapshot
names and wants to sort them by name, ie. when executes:

	# zfs list -t snapshot -o name -s name

Because only name is needed we don't have to read all snapshot properties.

Below you can find how long does it take to list 34509 snapshots from a single
disk pool before and after this change with cold and warm cache:

before:

	# time zfs list -t snapshot -o name -s name > /dev/null
	cold cache: 525s
	warm cache: 218s

after:

	# time zfs list -t snapshot -o name -s name > /dev/null
	cold cache: 1.7s
	warm cache: 1.1s

MFC after:	1 week
2012-01-21 21:12:53 +00:00
mm
d86f9f51f1 Add one more copyright line accidentially removed in r228103
MFC after:	3 days
2012-01-20 22:56:57 +00:00
mm
22e70588b6 Add accidentially removed copyright lines in r228103
Reported by:	pjd
MFC after:	3 days
2012-01-20 22:55:16 +00:00
pjd
65adbd19de Add missing file permission to open(O_CREAT) call. This doesn't really matter
here, as we open /dev/null for test purposes, but it is useful for consistency
and further grepping for such bugs.
2012-01-04 07:25:10 +00:00
dim
cecb0df44a In cddl/contrib/opensolaris/cmd/dtrace/dtrace.c, fix some obtuse
snprintf tricks.

MFC after:	1 week
2011-12-16 21:48:09 +00:00
dim
4296e211b6 In cddl/contrib/opensolaris/tools/ctf/cvt/ctfmerge.c, a size_t is passed
to fprintf as a field width.  It should be an int instead, so cast it.

MFC after:	1 week
2011-12-16 21:29:46 +00:00
dim
d3f0164628 In cddl/contrib/opensolaris/lib/libdtrace/common/dt_handle.c, some
uint64_t values are snprintf'd using %llx.  On amd64, uint64_t is
typedef'd as unsigned long, so cast the values to u_longlong_t, as is
done similarly in the rest of the file.

MFC after:	1 week
2011-12-16 15:12:42 +00:00
dim
c9021301d1 In cddl/contrib/opensolaris/lib/libdtrace/common/dt_consume.c, some
uint64_t values are snprintf'd using %llx.  On amd64, uint64_t is
typedef'd as unsigned long, so cast the values to u_longlong_t, as is
done similarly in the rest of the file.

MFC after:	1 week
2011-12-16 15:04:47 +00:00
dim
d2766f8c76 In cddl/contrib/opensolaris/lib/libdtrace/common/dt_subr.c, the
dt_popc() function assumes that either _ILP32 or _LP64 is defined,
otherwise it has no suitable implementation.

However, the _ILP32 and _LP64 macros come from isa_defs.h, which is not
included in this file.  Add the include now, to get the macros defined.

MFC after:	1 week
2011-12-15 23:06:46 +00:00
dim
e6a24af3b9 In cddl/contrib/opensolaris/lib/libdtrace/common/dt_link.c, use the
correct printf format for a key_t (aka long).

MFC after:	1 week
2011-12-15 22:46:04 +00:00
dim
b69a7d17be In cddl/contrib/opensolaris/lib/libdtrace/i386/dt_isadep.c, use the
correct printf format for an unsigned long.

MFC after:	1 week
2011-12-15 22:37:33 +00:00
dim
4d784918ed Cast away a clang alignment warning in drti.c's fixsymbol() function.
This code only runs on i386 and amd64, so there should be no problems if
buf + sec->dofs_offset is not aligned (which is unlikely anyway).

MFC after:	1 week
2011-12-15 22:10:27 +00:00
mm
9d05d9bfba Some mdoc(7) style and typo fixes to zfs(8).
Submitted by:	Nobuyuki Koganemaru <n-kogane@syd.odn.ne.jp>
MFC after:	3 days
2011-12-08 19:38:42 +00:00
mm
aad7cc1b2b Remove unnecesary "Ns" macros and add missing command example to zpool(8).
Reported by:	Nobuyuki Koganemaru <kogane@FreeBSD.org>
MFC after:	3 days
2011-12-02 19:56:46 +00:00
mm
73cab71971 Merge new ZFS features from illumos:
1644 add ZFS "clones" property
https://www.illumos.org/issues/1644

1645 add ZFS "written" and "written@..." properties
https://www.illumos.org/issues/1645

1646 "zfs send" should estimate size of stream
https://www.illumos.org/issues/1646

1647 "zfs destroy" should determine space reclaimed by destroying multiple
snapshots
https://www.illumos.org/issues/1647

1693 persistent 'comment' field for a zpool
https://www.illumos.org/issues/1693

1708 adjust size of zpool history data
https://www.illumos.org/issues/1708

1748 desire support for reguid in zfs
https://www.illumos.org/issues/1748

Obtained from:	illumos (changesets 13514, 13524, 13525)
MFC after:	1 month
2011-11-28 21:40:00 +00:00
rwatson
1a5f36a1c6 Change the Makefile in cddl/lib/drti to use bsd.lib.mk instead of
bsd.prog.mk -- we need to compile PIC, which requires a library build.
With this change, USDT (userspace DTrace probes) work from within
shared libraries.

PR:		kern/159046
Submitted by:	Alex Samorukov <samm at os2.kiev.ua>
Comments by:	Scott Lystig Fritchie <slfritchie at snookles.com>
MFC after:	3 days
2011-11-28 10:01:36 +00:00
mm
db9c9e87f1 Use singular form for zfs destroy snapshot in zfs(8).
MFC after:	6 days
2011-11-28 09:33:13 +00:00
mm
2cde0a6b78 Add missing warning to zfs(8) for using "zfs destroy" with -r and -R flags.
Obtained from:	illumos
MFC after:	6 days
2011-11-28 09:14:51 +00:00
mm
cbfa4256ba Add missing -n flag to "zpool import" description.
MFC after:	1 week
2011-11-27 20:24:07 +00:00
mm
42c0be29d9 Fix zfs(8) and zpool(8) context help to repport supported flags.
MFC after:	3 days
2011-11-27 11:46:09 +00:00
mm
3fe92eb448 Update ZFS manual pages to a mdoc(7) reimplementation.
The zfs(8) and zpool(8) manual pages now match the state of the ZFS module
and have been customized for FreeBSD.

The new texts of the "Deduplication" subsection in zfs(8), the zpool "split"
command, the zfs "dedup" property and several other missing parts have been
added from illumos or OpenSolaris snv_134 (CDDL-licensed).

The mdoc(7) reimplementation of whole manual pages, the descriptions of the
zpool "readonly" property, "zfs diff" command and descriptions of several
other missing command flags and/or options were authored by myself.

MFC after:	1 week
2011-11-27 11:44:20 +00:00
mm
e4473aeb77 Revert back to revision 227649 because of license uncertainity.
Manual pages from OpenSolaris svn_134 are still properly CDDL licensed
but I have been informed that the parts from s11ex are uncertain even
if they contain a CDDL header.
2011-11-21 20:33:18 +00:00
mm
c40e1792cb Fix mismerge in copyright of zpool(8).
MFC after:	4 days
2011-11-21 15:33:37 +00:00
mm
ef1f257773 Update zfs(8) and zpool(8) manpages from CDDL-licensed sources [1].
Improved alignment for a maximum width of 80 characters.
Mark unsupported parts as such.

Reported to vendor:	Illumos issue #1801

References:
https://www.illumos.org/issues/1801

Obtained from:	OpenSolaris CDDL manual pages (snv_134, s11express) [1]
MFC after:	4 days
2011-11-21 15:26:20 +00:00
mm
1c31f8b500 For now, remove MANFILTER from ZFS manpages (bad effect on html output).
MFC after:	4 days
2011-11-21 14:50:17 +00:00
mm
18b29f3fb8 Update and desolarization of zdb(8) and zstreamdump(1) manual pages:
- synchronized to match new vendor code [1]
- removed ATTRIBUTES sections
- updated SEE ALSO sections
- properly updated copyright information (required by CDDL)
- remove empty lines via MANFILTER

Obtained from:	Illumos [1]
MFC after:	5 days
2011-11-20 13:11:29 +00:00
mm
3985e4e2af More zfs(8) manpage fixes:
- remove shareiscsi property
- mark casesensitivity property as unsupported
- remove reference to Solaris Administration Guide

MFC after:	1 week
2011-11-18 02:25:54 +00:00
mm
d52a207cf9 Fix reference to fsync(2).
Add more references to SEE ALSO section.

MFC after:	1 week
2011-11-18 02:06:09 +00:00
mm
b7c8d365cc Update and desolarization of zfs(8) and zpool(8) manual pages:
- synchronized to match new vendor code (Illumos rev. 13513) [1]
- removed references to sun commands (replaced with FreeBSD commands)
- removed ATTRIBUTES sections
- updated SEE ALSO sections
- properly updated copyright information (required by CDDL)
- remove empty lines via MANFILTER

zfs(8) only:
- replaced "Zones" section with new "Jails" section
- removed misleading "ZFS Volumes as Swap or Dump Devices" section
- updated shareiscsi and sharesmb option information (not supported on FreeBSD)
- replace zoned property with jailed property

zpool(8) only:
- updated device names in examples

Obtained from:	Illumos (as of rev. 13513:f84d4672fdbd) [1]
MFC after:	1 week
2011-11-18 01:28:52 +00:00
mm
c416d11ddd Import upstream changesets for the output of the "zpool" command:
952 separate intent logs should be obvious in 'zpool iostat' output
1337 `zpool status -D' should tell if there are no DDT entries

References:
https://www.illumos.org/issues/952
https://www.illumos.org/issues/1337

Obtained from:	Illumos (issues 952, 1337; changesets 13384, 13432)
MFC after:	1 week
2011-11-14 08:29:49 +00:00
rmh
1ab901bfdd Fix a few gratuitous library dependencies. Some of the ZFS utilities
are linked with libraries they don't use:

 - zinject doesn't use libavl
 - ztest doesn't use libz
 - zdb uses neither libavl nor libz
 - zfs uses neither libbsdxml nor libm, nor libsbuf
 - zpool uses neither libbsdxml nor libm, nor libsbuf

In addition, libzfs needs libm because it uses pow(), however it isn't
linked with -lm.  This went unnoticed because all its users had -lm before.

Reviewed by:	pjd, mm
Approved by:	kib (mentor)
MFC after:	1 week
2011-10-30 16:29:04 +00:00
pjd
b8b7c9f487 Update copyright to include myself.
MFC after:	2 weeks
2011-10-24 21:22:55 +00:00
pjd
7f814b700d Extend r226676 to allow rename without unmount even for file systems with
non-legacy mountpoints. It is better to be able to rename such file systems and
let them be mounted in old places until next reboot than using live CD, etc. to
rename with remount.

This is implemented by adding -u option to 'zfs rename'. If file system's
mountpoint property is set to 'legacy' or 'none', there is no need to specify -u.

Update zfs(8) manual page to reflect this addition.

MFC after:	2 weeks
2011-10-24 21:14:50 +00:00
pjd
abdbea6659 Allow to rename file systems without remounting if it is possible.
It is possible for file systems with 'mountpoint' preperty set to 'legacy'
or 'none' - we don't have to change mount directory for them.
Currently such file systems are unmounted on rename and not even mounted back.

This introduces layering violation, as we need to update 'f_mntfromname'
field in statfs structure related to mountpoint (for the dataset we are
renaming and all its children).

In my opinion it is worth it, as it allow to update FreeBSD in even cleaner
way - in ZFS-only configuration root file system is ZFS file system with
'mountpoint' property set to 'legacy'. If root dataset is named system/rootfs,
we can snapshot it (system/rootfs@upgrade), clone it (system/oldrootfs),
update FreeBSD and if it doesn't boot we can boot back from system/oldrootfs
and rename it back to system/rootfs while it is mounted as /. Before it was
not possible, because unmounting / was not possible.

MFC after:	2 weeks
2011-10-24 00:38:09 +00:00
pjd
29ea4ed10e zdb: access dp_free_bpobj only if pool version is >= SPA_VERSION_DEADLISTS
Submitted by:	avg
MFC after:	3 days
2011-10-21 13:56:17 +00:00