Commit Graph

513 Commits

Author SHA1 Message Date
delphij
ec5d67aa46 MFC r260183: MFV r260154 + 260182:
4369 implement zfs bookmarks
4368 zfs send filesystems from readonly pools

Illumos/illumos-gate@78f1710053
2014-03-20 00:28:53 +00:00
delphij
11b0e21c75 MFC r259850: MFV r258384:
2583 Add -p (parsable) option to zfs list

illumos/illumos-gate@43d68d68c1
2014-03-20 00:24:35 +00:00
delphij
18a0059ca2 MFC r256999 (smh):
Added support for the 'zfs list -t snap' and 'zfs snap' aliases which are
available under Oracle Solaris 11.

This includes an update to the ZFS(8) man page to reflect all the
available alias (snap, umount, and recv).

Initial changes obtained from ZFS On Linux + fixes for man page and cmd
help:
10b75496bb
cf81b00a73

Obtained from:  https://github.com/zfsonlinux/zfs
2014-03-20 00:10:58 +00:00
delphij
ede49d4106 MFC r260150: MFV r259170:
4370 avoid transmitting holes during zfs send

4371 DMU code clean up

illumos/illumos-gate@43466aae47

NOTE: Make sure the boot code is updated if a zpool upgrade is
done on boot zpool.
2014-03-19 23:55:03 +00:00
delphij
874f949f14 MFC r260138: MFV r242733:
3306 zdb should be able to issue reads in parallel
3321 'zpool reopen' command should be documented in the man page
and help message

illumos/illumos-gate@31d7e8fa33

FreeBSD porting notes: the kernel part of this changeset depends
on Solaris buf(9S) interfaces and are not really applicable for
our use.  vdev_disk.c is patched as-is to reduce diverge from
upstream, but vdev_file.c is left intact.
2014-03-19 23:44:03 +00:00
delphij
5b9b9ced50 MFC r259813 + r259813: MFV r258374:
4171 clean up spa_feature_*() interfaces

4172 implement extensible_dataset feature for use by other zpool
features

illumos/illumos-gate@2acef22db7
2014-03-19 23:36:12 +00:00
delphij
dda91b34ac MFC r262577: MFV r262570:
4626 libzfs memleak in zpool_in_use()
2014-03-14 01:09:42 +00:00
eadler
1f967fac12 MFC r261774 by feld:
Add caveat to zpool manpage indicating that we do not automatically activate
	hot spares. This should be MFC'd to all STABLE branches.

	Upon the availability of zfsd, the zpool manpage on relevant branches should
	be updated to remove this caveat and document hot spare's reliance on zfsd.

Requested by:	feld
2014-02-24 17:01:27 +00:00
avg
444af87821 MFC r260811: zdb -R: do not treat numeric parameters to a flag as more flags 2014-02-17 17:48:38 +00:00
avg
e67db98e94 MFC r260703: zinject must use ioctl(2) compatibility wrapper 2014-02-17 17:46:15 +00:00
avg
51714698c3 MFC r258717: MFV r258371,r258372: 4101 metaslab_debug should allow for
fine-grained control
2014-02-17 17:11:38 +00:00
avg
6f834c6747 MFC r261122: dtrace: remove unexplained 16MB limitation from dt_alloc/dt_zalloc 2014-02-17 15:35:11 +00:00
markj
9a6dcea530 MFC r260051:
When clearing relocations to __dtrace* symbols, handle both SHT_REL and
SHT_RELA sections properly instead of assuming that the relocation section
is of type SHT_REL.
2014-02-17 14:51:02 +00:00
avg
26096ba436 MFC r258632,258704: MFV r255255: 4045 zfs write throttle & i/o scheduler
performance work

Sponsored by:	HybridCluster [merge]
2014-01-16 15:57:39 +00:00
avg
a4ee4e9fb0 MFC r258630: 734 taskq_dispatch_prealloc() desired 2014-01-16 14:34:53 +00:00
jhibbits
04f97f6231 MFC r258362
Use 'int' to store the return value of getopt(), rather than char.

On some architectures (powerpc), char is unsigned by default, which means
comparisons against -1 always fail, so the programs get stuck in an
infinite loop.
2014-01-15 05:30:05 +00:00
jhibbits
b1391adbae MFC r256543,r259245,r259421,r259668,r259674
r256543:

Add fasttrap for PowerPC.  This is the last piece of the DTrace/ppc puzzle.
It's incomplete, it doesn't contain full instruction emulation, but it should be
sufficient for most cases.

r259245,r259421: (FBT)

FBT now does work fully on PowerPC.

Save r3 before using it for the trap check, else we end up saving the new r3,
containing the trap instruction encoding (0x7c810808), and restoring it back
with the frame on return.  This caused it to panic on my ppc32 machine.

r259668,r259674:
Fix a typo in the FBT code.
2014-01-15 05:19:37 +00:00
delphij
139f0a076d MFC r259811:
MFV r258373:

4168 ztest assertion failure in dbuf_undirty

4169 verbatim import causes zdb to segfa
4170 zhack leaves pool in ACTIVE state

illumos/illumos-gate@7fdd916c47
2014-01-14 01:28:08 +00:00
mav
cebac06b67 MFC r259168:
Don't even try to read vdev labels from devices smaller then SPA_MINDEVSIZE
(64MB).  Even if we would find one somehow, ZFS kernel code rejects such
devices.  It is funny to look on attempts to read 4 256K vdev labels from
1.44MB floppy, though it is not very practical and quite slow.
2014-01-05 22:14:12 +00:00
delphij
907f6b0d31 MFC r259131:
Don't panic when we get ZPOOL_STATUS_NON_NATIVE_ASHIFT
while listing importable pools.
2013-12-23 20:16:54 +00:00
markj
0b5457bf66 Convert the dtrace(1) man page to mdoc and fix up some aspects of it that
don't make sense on FreeBSD. In particular,

- remove the ATTRIBUTES section,
- remove references to the Solaris Dynamic Tracing Guide, except in the
  SEE ALSO section,
- update the description of the -A option for FreeBSD's implementation,
- remove references to Solaris-specific programs and configuration files,
  and replace them with FreeBSD equivalents where possible.

The content has not changed aside from this.

Approved by:	re (joel)
MFC after:	1 week
2013-10-10 03:50:23 +00:00
rmh
774f6b00bf Fix implicit declaration of jail_getid()
Approved by:	re
2013-10-07 14:22:19 +00:00
markj
8ff2d52009 Add a separate translator for headers passed to the TCP probes in the
input path. These probes get some of the fields in host order, whereas the
output probes get them in network order, so a single translator isn't
enough. This workaround ensures that the problem is essentially invisble
to users: none of the probe arguments or their fields have changed.

Approved by:	re (hrs)
2013-10-02 17:14:12 +00:00
delphij
d04a7f0144 MFV r254750:
Add support of Illumos dumps on zvol over RAID-Z.

Note that this only adds the features.  FreeBSD would
still need more work to support dumping on zvols.

Illumos ZFS issues:
  2932 support crash dumps to raidz, etc. pools

MFC after:	1 month
Approved by:	re (ZFS blanket)
2013-09-21 00:17:26 +00:00
markj
37ef3ce41e Use the address of the inpcb rather than the tcpcb to identify TCP
connections. This keeps the tcp provider consistent with the other network
providers.

Approved by:	re (delphij)
2013-09-15 21:38:46 +00:00
delphij
a23043347c MFV r247844 (illumos-gate 13975:ef6409bc370f)
Illumos ZFS issues:
  3582 zfs_delay() should support a variable resolution
  3584 DTrace sdt probes for ZFS txg states

Provide a compatibility shim for Solaris's cv_timedwait_hires
to help aid future porting.

Approved by:	re (ZFS blanket)
2013-09-10 01:46:47 +00:00
will
1b508b8cc8 Build all ZFS testing & debugging tools with -g.
These programs and everything using libzpool rely on the embedded asserts to
verify the correctness of operations.  Given that, the core dumps would be
useless without debug symbols.
2013-08-27 04:01:31 +00:00
pfg
615f223c07 Merge various CTF fixes from illumos
2942 CTF tools need to handle files which legitimately lack data
2978 ctfconvert still needs to ignore legitimately dataless files on SPARC

Illumos Revisions:	13745:6b3106b4250f
			13754:7231b684c18b

Reference:

https://www.illumos.org/issues/2942
https://www.illumos.org/issues/2978

MFC after:	3 weeks
2013-08-26 22:29:42 +00:00
markj
29e4661920 Implement the ip, tcp, and udp DTrace providers. The probe definitions use
dynamic translation so that their arguments match the definitions for
these providers in Solaris and illumos. Thus, existing scripts for these
providers should work unmodified on FreeBSD.

Tested by:	gnn, hiren
MFC after:	1 month
2013-08-25 21:54:41 +00:00
delphij
a4cf8ab508 MFV r254751:
Don't treat the parameter as a number (pool GUID) when there is
error converting it from string, instead, treat it as the pool
name.

Illumos ZFS issues:
  1765 assert triggered in libzfs_import.c trying to import pool
       name beginning with a number
2013-08-24 00:54:47 +00:00
delphij
40685a92a9 MFV r254748:
Fix memory leak in libzfs's iter_dependents_cb().

Illumos ZFS issues:
  4061 libzfs: memory leak in iter_dependents_cb()
2013-08-24 00:29:34 +00:00
delphij
677bfa2265 MFV r254746:
To quote original Illumos ticket:

libctf thinks that any ELF file containing more than 65536 sections is
corrupt, because it doesn't understand the SHN_XINDEX magic.

Illumos DTrace issues:
  4005 libctf can't deal with extended sections
2013-08-23 23:58:56 +00:00
delphij
5017a032d2 MFV r254422:
Illumos DTrace issues:
  3089 want ::typedef
  3094 libctf should support removing a dynamic type
  3095 libctf does not validate arrays correctly
  3096 libctf does not validate function types correctly
2013-08-23 23:21:24 +00:00
gibbs
bd47afb289 Enhance the ZFS vdev layer to maintain both a logical and a physical
minimum allocation size for devices.  Use this information to
automatically increase ZFS's minimum allocation size for new top-level
vdevs to a value that more closely matches the optimum device
allocation size.

Use GEOM's stripesize attribute, if set, as the physical sector
size of the GEOM.

Calculate the minimum blocksize of each metaslab class.  Use the
calculated value instead of SPA_MINBLOCKSIZE (512b) when determining
the likelyhood of compression yeilding a reduction in physical space
usage.

Report devices with sub-optimal block size configuration in "zpool
status".  Also properly fail attempts to attach devices with a
logical block size greater than 8kB, since this will cause corruption
to ZFS's label area.

Sponsored by:	Spectra Logic Corporaion
MFC after:	2 weeks

Background
==========
Many modern devices use physical allocation units that are much
larger than the minimum logical allocation size accessible by
external commands.  Two prevalent examples of this are 512e disk
drives (512b logical sector, 4K physical sector) and flash devices
(512b logical sector, 4K or larger allocation block size, and 128k
or larger erase block size).  Operations that modify less than the
physical sector size result in a costly read-modify-write or garbage
collection sequence on these devices.

Simply exporting the true physical sector of the device to ZFS would
yield optimal performance, but has two serious drawbacks:

1) Existing pools created with devices that have different logical
   and physical block sizes, but were configured to use the logical
   block size (e.g. because the OS version used for pool construction
   reported the logical block size instead of the physical block
   size) will suddenly find that the vdev allocation size has
   increased.  This can be easily tolerated for active members of
   the array, but ZFS would prevent replacement of a vdev with
   another identical device because it now appears that the smaller
   allocation size required by the pool is not supported by the new
   device.

2) The device's physical block size may be too large to be supported
   by ZFS.  The optimal allocation size for the vdev may be quite
   large.  For example, a RAID controller may export a vdev that
   requires read-modify-write cycles unless accessed using 64k
   aligned/sized requests.  ZFS currently has an 8k minimum block
   size limit.

Reporting both the logical and physical allocation sizes for vdevs
solves these problems.  A device may be used so long as the logical
block size is compatible with the configuration.  By comparing the
logical and physical block sizes, new configurations can be optimized
and administrators can be notified of any existing pools that are
sub-optimal.

sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/spa.h:
	Add the SPA_ASHIFT constant.  ZFS currently has a hard upper
	limit of 13 (8k) for ashift and this constant is used to
	both document and enforce this limit.

sys/cddl/contrib/opensolaris/uts/common/sys/fs/zfs.h:
	Add the VDEV_AUX_ASHIFT_TOO_BIG error code.

	Add fields for exporting the configured, logical, and
	physical ashift to the vdev_stat_t structure.

	Add VDEV_STAT_VALID() macro which can be used to verify the
	presence of required vdev_stat_t fields in nvlist data.

sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c:
	Provide a SYSCTL_PROC handler for "max_auto_ashift".  Since
	the limit is only referenced long after boot when a create
	operation occurs, there's no compelling need for it to be
	a boot time configurable tunable.  This also allows the
	validation code for the max_auto_ashift value to be contained
	within the sysctl handler.

	Populate the new fields in the vdev_stat_t structure.

	Fail vdev opens if the vdev reports an ashift larger than
	SPA_MAXASHIFT.

	Propogate vdev_logical_ashift and vdev_physical_ashift between
	child and parent vdevs as is done for vdev_ashift.

	In vdev_open(), restore code that fails opens for devices
	where vdev_ashift grows.  This can only happen now if the
	device's logical ashift grows, which means it really isn't
	safe to use the device.

sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev_impl.h:
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c:
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_file.c:
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c:
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_mirror.c:
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_missing.c:
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_raidz.c:
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_root.c:
	Update the vdev_open() API so that both logical (what was
	just ashift before) and physical ashift are reported.

sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev_impl.h:
	Add two new fields, vdev_physical_ashift and vdev_logical_ashift,
	to vdev_t.

sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c:
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_config.c:
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c:
	Add vdev_ashift_optimize().  Call it anytime a new top-level
	vdev is allocated.

cddl/contrib/opensolaris/cmd/zpool/zpool_main.c:
	Add text for the VDEV_AUX_ASHIFT_TOO_BIG error.

	For each sub-optimally configured leaf vdev, report configured
	and native block sizes.

cddl/contrib/opensolaris/cmd/zpool/zpool_main.c:
cddl/contrib/opensolaris/lib/libzfs/common/libzfs.h:
cddl/contrib/opensolaris/lib/libzfs/common/libzfs_status.c:
	Introduce a new zpool status: ZPOOL_STATUS_NON_NATIVE_ASHIFT.
	This status is reported on healthy pools containing vdevs
	configured to use a block size smaller than their reported
	physical block size.

cddl/contrib/opensolaris/lib/libzfs/common/libzfs_status.c:
	Update find_vdev_problem() and supporting functions to
	provide the full vdev_stat_t structure to problem checking
	routines, and to allow decent into replacing vdevs.

	Add a vdev_non_native_ashift() validator which is used on
	the full vdev tree to check for ZPOOL_STATUS_NON_NATIVE_ASHIFT.

cddl/contrib/opensolaris/lib/libzpool/common/kernel.c:
cddl/contrib/opensolaris/lib/libzpool/common/sys/zfs_context.h:
	Enhance sysctl userland stubs now that a SYSCTL_PROC handler
	is used in vdev.c.

sys/cddl/contrib/opensolaris/uts/common/fs/zfs/metaslab.c:
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/metaslab_impl.h:
	When the group membership of a metaslab class changes (i.e.
	when a vdev is added or removed from a pool), walk the group
	list to determine the smallest block size currently available
	and record this in the metaslab class.

sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/metaslab.h:
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/metaslab.c:
	Add the metaslab_class_get_minblocksize() accessor.

sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zio_compress.h:
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio_compress.c:
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:
	In zio_compress_data(), take the minimum blocksize as an
	input parameter instead of assuming SPA_MINBLOCKSIZE.

sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:
	In l2arc_compress_buf(), pass SPA_MINBLOCKSIZE as the minimum
	blocksize of the device.  The l2arc code performs has it's own
	code for deciding if compression is worth while, so this
	effectively disables zio_compress_data() from second guessing
	the original decision.

sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:
	In zio_write_bp_init(), use the minimum blocksize of the
	normal metaslab class when compressing data.
2013-08-21 04:10:24 +00:00
delphij
1b0e7b9e07 MFV r254421:
Illumos ZFS issues:
  3996 want a libzfs_core API to rollback to latest snapshot
2013-08-21 00:04:31 +00:00
rpaulo
10427f9082 Load the dtraceall module if /dev/dtrace/dtrace doesn't exist.
MFC after:	3 days
2013-08-10 23:17:09 +00:00
delphij
ccc3c4970e MFV r254079:
Illumos ZFS issues:
  3957 ztest should update the cachefile before killing itself
  3958 multiple scans can lead to partial resilvering
  3959 ddt entries are not always resilvered
  3960 dsl_scan can skip over dedup-ed blocks if
       physical birth != logical birth
  3961 freed gang blocks are not resilvered and can cause pool to suspend
  3962 ztest should print out zfs debug buffer before exiting
2013-08-08 23:38:31 +00:00
delphij
7291294314 MFV r254071:
Fix a regression introduced by fix for Illumos bug #3834.  Quote from
Matthew Ahrens on the Illumos issue:

ztest fails this assertion because ztest_dmu_read_write() does
        dmu_tx_hold_free(tx, bigobj, bigoff, bigsize);
and then
    dmu_object_set_checksum(os, bigobj,
        (enum zio_checksum)ztest_random_dsl_prop(ZFS_PROP_CHECKSUM), tx);

If the region to free is past the end of the file, the DMU assumes that there
will be nothing to do for this object.  However, ztest does set_checksum(),
which must modify the dnode.  The fix is for ztest to also call

    dmu_tx_hold_bonus(tx, bigobj);

so we can account for the dirty data associated with setting the checksum

Illumos ZFS issues:
  3955 ztest failure: assertion refcount_count(&tx->tx_space_written)
         + delta <= tx->tx_space_towrite
2013-08-07 22:21:00 +00:00
delphij
0171695909 MFV r254070:
Merge vendor bugfix for ZFS test suite that triggers false positives.

Illumos ZFS issues:
  3949 ztest fault injection should avoid resilvering devices
  3950 ztest: deadman fires when we're doing a scan
  3951 ztest hang when running dedup test
  3952 ztest: ztest_reguid test and ztest_fault_inject don't place nice together
2013-08-07 21:16:14 +00:00
rmh
5933e25518 Fix implicit declaration of warnx(). 2013-08-04 16:25:46 +00:00
delphij
c5affee6a3 MFV r253781 + r253871:
Illumos ZFS issues:
  3894 zfs should not allow snapshot of inconsistent dataset

MFC after:	2 weeks
2013-07-30 21:02:09 +00:00
smh
ef92cf9910 MFV r253784:
Fix zfs send -D hang after processing requiring a CTRL+C to interrupt due to
pthread_join prior to fd close.

This was introduced by r251646 (MFV r251644)

Illumos ZFS issue:
  3909 "zfs send -D" does not work

MFC after:	1 day
2013-07-30 20:45:27 +00:00
pfg
03e98d6f13 DTrace: re-apply r249426 now that the underlying issues have been solved.
Merge change from illumos:

3519 DTrace fails to resolve const types from fbt
3520 dtrace internal error -- token type 316 is not a valid D
     compilation token
3521 clean up dtrace unit tests

Illumos Revision:	e98f46c

Reference:
https://www.illumos.org/issues/3519
https://www.illumos.org/issues/3520
https://www.illumos.org/issues/3521

Tested by:	Fabian Keil
Obtained from:	Illumos
MFC after:	1 month
2013-07-28 01:02:17 +00:00
pfg
46097436dc DTrace: re-merge remainder of r249367 (original from Illumos).
Bring back some important fixes from Illumos:

3022 DTrace: keys should not affect the sort order when sorting by value
3023 it should be possible to dereference dynamic variables
3024 D integer narrowing needs some work

We particularly avoid the LD_NOLAZYLOAD changes that Illumos made
as those don't apply to FreeBSD and were causing problems in
interactive mode.

Illumos Revision:	13758:23432da34147

Reference:

https://www.illumos.org/issues/3022
https://www.illumos.org/issues/3023
https://www.illumos.org/issues/3024

MFC after:	1 month
Tested by:	markj
2013-07-28 00:45:20 +00:00
markj
401af7020e Use kern_ioctl() rather than ioctl() for testing the FBT provider, since the
latter doesn't exist in FreeBSD. All the tests under fbtprovider pass now.
2013-07-27 21:31:48 +00:00
pfg
923a920777 Style issue in r253661.
Pointed out by:	avg
MFC after:	1 month
2013-07-26 14:37:23 +00:00
pfg
34504eb8a9 Fix a segfault in ctfmerge due to a bug in gcc.
GCC can generate bogus dwarf attributes with DW_AT_byte_size
set to 0xFFFFFFFF.
The issue was originaly detected in NetBSD but it has been
adapted for portability and to avoid compiler warnings.

Reference:
https://www.illumos.org/issues/3776

Obtained from:	NetBSD
MFC after:	1 month
2013-07-26 00:28:19 +00:00
delphij
5a5f4c42fb Manually merge part of vendor import r238583 from Illumos.
Illumos changeset: 13680:2bd022a765e2
Illumos ZFS issue:

    2671 zpool import should not fail if vdev ashift has increased

MFC after:	3 days
2013-07-18 00:22:42 +00:00
mm
ddbb7a25f4 Fix misleading or remove irrelevant illumos messages and manpage references
in the zfs command.

PR:		bin/178996
Submitted by:	Peter Schaefer <peter.schaefer@wilhelmheinrichs.de>
MFC after:	3 days
2013-07-04 22:26:38 +00:00
delphij
4dfc3c75a2 MFV r252215:
Restore a previous behavior before r251646, where when destructing
ZFS snapshot, the ioctl would return ENOENT when it hit any of
them in the errlist (the new behavior was only return ENOENT when
all returns error).

Illumos ZFS issues:
  3829 fix for 3740 changed behavior of zfs destroy/hold/release ioctl

MFC after:	1 week
2013-06-25 22:14:32 +00:00
delphij
5240466227 Diff reduction against Illumos, no real change to code itself.
This marks vendor branch revision 252213 as merged, the actual code was
committed in r245479.

MFC after:	1 week
2013-06-25 21:51:52 +00:00
smh
08b5f9540c Fixed ZFS zpool freeze (debug command) not processing due to invalid ioctl call syntax.
MFC after:	1 week
2013-06-21 15:30:46 +00:00
delphij
ab3dbcb998 MFV r251644:
Poor ZFS send / receive performance due to snapshot
hold / release processing (by smh@)

Illumos ZFS issues:
  3740 Poor ZFS send / receive performance due to snapshot
       hold / release processing

MFC after:      2 weeks
2013-06-12 07:07:06 +00:00
delphij
421d823726 MFV r251624:
txg commit callbacks don't work

Illumos ZFS issues:
  3747 txg commit callbacks don't work

MFC after:      2 weeks
2013-06-11 19:29:31 +00:00
delphij
8fe6a06d58 MFV r251623:
zpool create should treat -O mountpoint and -m the same

Illumos ZFS issues:
  3745 zpool create should treat -O mountpoint and -m the same

MFC after:      2 weeks
2013-06-11 19:25:49 +00:00
delphij
9d0815fcd1 MFV r251619:
ZFS needs better comments.

Illumos ZFS issues:
  3741 zfs needs better comments

MFC after:      2 weeks
2013-06-11 19:02:36 +00:00
markj
f492c31639 Port the SDT test now that it's possible to create SDT probes that take
seven arguments.

The original test uses Solaris' uadmin system call to trigger the test
probe; this change adds a sysctl to the dtrace_test module and gets the test
program to trigger the test probe via the sysctl handler.

The test is currently failing on amd64 because of some bugs in the way that
probe arguments beyond the first five are obtained - these bugs will be
fixed in a separate change.
2013-06-02 00:33:36 +00:00
markj
b3c31e1254 Remove a block of code that was not intended to be part of the previous
revision.

X-MFC with:	r250812
2013-05-20 02:10:18 +00:00
markj
9e08262228 Re-introduce another part of r249367. This commit fixes a register leak in
dt_cg_ptrsize() and generally cleans up some of the error handling around
register allocation.

This change corresponds to part of illumos-gate commit e5803b76927480:
  3025 register leak in D code generation

Reviewed by:	pfg
Obtained from:	illumos
MFC after:	1 month
2013-05-19 17:14:36 +00:00
markj
e34342dafe Convert a couple of helper scripts used to test the ip provider to work on
FreeBSD. In the IPv6 case, try each interface before returning an error;
each IPv6-enabled interface will have a link-local address even if the link
isn't up.

MFC after:	1 week
2013-05-15 22:56:24 +00:00
markj
08da0b8fad head -n doesn't take negative arguments on FreeBSD, so instead use sed '$d'
to remove the last line of a file.
2013-05-12 16:29:09 +00:00
markj
8f40d761c1 Bring back part of r249367 by adding DTrace's temporal option, which allows
users to guarantee that the output of DTrace scripts will be time-ordered.
This option is enabled by adding the line

  #pragma D option temporal

to the beginning of a script, or by adding '-x temporal' to the arguments of
dtrace(1).

This change fixes a bug in the original port of the temporal option. This
bug was causing some assertions to fail, so they had been disabled; in this
revision the assertions are working properly and are enabled.

The DTrace version number has been bumped from 1.9.0 to 1.9.1 to reflect
the language change that's being introduced.

This change corresponds to part of illumos-gate commit e5803b76927480:
  3021 option for time-ordered output from dtrace(1M)

Reviewed by:	pfg
Obtained from:	illumos
MFC after:	1 month
2013-05-12 16:26:33 +00:00
brooks
dda6e06873 Work around the implementation of LIBRARIES_ONLY. It causes drti.o to
not be installed in /usr/lib32 on systems with compat-32 support.

This fix has two parts.  First, the build is forced by linking drti.o
into a dummy internal library.  Second, the object file is installed
manually in the LIBRARIES_ONLY case.

MFC after:	3 days
2013-05-07 22:05:57 +00:00
avg
8fa4324396 revert r248644 because of the regression for usdt probes
USDT probes are advertised to kernel by initialization code with
atrribute((constructor))).  It seems that on Solaris the .init-ish code
of the main object is executed before RD_PREINIT point is hit.  On
FreeBSD that is not the case.  And because on FreeBSD there is no other
well-defined point between RD_PREINIT and main() we have to parse a
DTrace script when main is hit, for time being.

A footnote: currently we actually post RD_POSTINIT event, but that's a
bug because the event is triggered by hitting r_debug_state which
happens before any init code is executed.

Reported by:	markj
2013-04-25 07:04:56 +00:00
mm
edefb5410f Respect the enoent_ok flag if reporting error for holding an non-existing
snapshot.

Related illumos ZFS issue:
  3699 zfs hold or release of a non-existent snapshot does not output error

Reported by:	Steven Hartland <smh@FreeBSD.org>
MFC after:	3 days
2013-04-25 06:55:57 +00:00
ed
e342f91158 Fix -Wmissing-variable-declarations compiler warnings.
References:
https://www.illumos.org/issues/3700
2013-04-19 19:38:39 +00:00
pfg
6f17eab492 DTrace: NFS translators should be split into client/server pieces
Belatedly update the provider with changes from OpenSolaris:

PSARC 2008/050 DTrace NFS v3 Provider
6696397 NFS v3 provider reports all UDP clients as 0.0.0.0
6711844 assert: vp->v_shrlocks

Obtained from:	Illumos
2013-04-18 01:20:08 +00:00
pfg
d3334c59c7 DTrace: Revert r249426
This change actually depends on r249367 which had to be reverted

Pointy Hat:	pfg
2013-04-17 02:40:07 +00:00
pfg
f9796e10e2 DTrace: Revert r249367
The following change from illumos brought caused DTrace to
pause in an interactive environment:

3026 libdtrace should set LD_NOLAZYLOAD=1 to help the pid provider

This was not detected during testing because it doesn't
affect scripts.

We shouldn't be changing the environment, especially since the
LD_NOLAZYLOAD option doesn't apply to our (GNU) ld.
Unfortunately the change from upstream was made in such a way
that it is very difficult to separate this change from the
others so, at least for now, it's better to just revert
everything.

Reference:
https://www.illumos.org/issues/3026

Reported by:	Navdeep Parhar and Mark Johnston
2013-04-17 02:20:17 +00:00
pfg
c31105ae2f DTrace: print() should try to resolve function pointers
Merge changes from illumos:

3675 DTrace print() should try to resolve function pointers
3676 dt_print_enum hardcodes a value of zero

Illumos Revision:	b1fa6326238973aeaf12c34fcda75985b6c06be1

Reference:
https://www.illumos.org/issues/3675
https://www.illumos.org/issues/3676

Obtained from:	Illumos
MFC after:	1 month
2013-04-16 19:39:27 +00:00
pjd
a87381e520 Correct error message.
Reported by:	Dirk Engling <erdgeist@erdgeist.org>
2013-04-16 12:31:16 +00:00
pfg
7eee57bac2 DTrace: NFS translators should be split into client/server pieces
Merge change from illumos:

1731 DTrace NFS translators should be split into client/server pieces

Illumos Revision:	13523:6763769941d2

This code seems to be currently unused on FreeBSD.

Reference:
https://www.illumos.org/issues/1731

Obtained from:	Illumos
MFC after:	1 week
2013-04-15 20:16:31 +00:00
pfg
0132d9f8dc Dtrace: resolve const types from fbt and other fixes.
Merge change from illumos:

3519 DTrace fails to resolve const types from fbt
3520 dtrace internal error -- token type 316 is not a valid D
     compilation token
3521 clean up dtrace unit tests

Illumos Revision:	e98f46c

Reference:
https://www.illumos.org/issues/3519
https://www.illumos.org/issues/3520
https://www.illumos.org/issues/3521

Tested by:	Fabian Keil
Obtained from:	Illumos
MFC after:	1 month
2013-04-12 20:10:27 +00:00
pfg
2554e81a37 DTrace: option for time-ordered output
Merge changes from illumos:

3021 option for time-ordered output from dtrace(1M)
3022 DTrace: keys should not affect the sort order when sorting by value
3023 it should be possible to dereference dynamic variables
3024 D integer narrowing needs some work
3025 register leak in D code generation
3026 libdtrace should set LD_NOLAZYLOAD=1 to help the pid provider

This brings yet another feature implemented in upstream DTrace.
A complete description is available here:
http://dtrace.org/blogs/ahl/2012/07/28/my-new-dtrace-favorite/

This change bumps the DT_VERS_* number to 1.9.1 in
accordance to what is done in illumos.

This change was somewhat complicated because upstream is mixed many
changes in an individual commit and some of the tests don't really
apply to us.

There are also appear to be differences in timestamping with Solaris
so we had to workaround some assertions making sure no regression
happened.

Special thanks to Fabian Keil for changes and testing.

Illumos Revisions:	13758:23432da34147

Reference:
https://www.illumos.org/issues/3021
https://www.illumos.org/issues/3022
https://www.illumos.org/issues/3023
https://www.illumos.org/issues/3024
https://www.illumos.org/issues/3025
https://www.illumos.org/issues/1694

Tested by:	Fabian Keil
Obtained from:	Illumos
MFC after:	1 months
2013-04-11 16:24:36 +00:00
mm
1a822a3e4d Fix libzfs to report error instead of returning zero if trying to hold or
release a non-existing snapshot of a existing dataset. In recursive case
error is reported if no snapshots with the requested name have been found.

Problem and proposed solution reported to illumos:
  3699 zfs hold or release of a non-existent snapshot does not output error

MFC after:	8 days
2013-04-11 07:49:16 +00:00
mm
6cbe43292e ZFS expects a copyout of zfs_cmd_t on an ioctl error. Our sys_ioctl()
doesn't copyout in this case.

To solve this issue a new struct zfs_iocparm_t is introduced consisting of:
- zfs_ioctl_version (future backwards compatibility purposes)
- user space pointer to zfs_cmd_t (copyin and copyout)
- size of zfs_cmd_t (verification purposes)

The copyin and copyout of zfs_cmd_t is now done the illumos (vendor) way
what makes porting of new changes easier and ensures correct behavior if
returning an error.

MFC after:	10 days
2013-04-09 22:27:44 +00:00
mm
b78a61d3ba MFV r249185:
Allow zdb to output a histogram of compressed block sizes.

Illumos ZFS issues:
  3641 want a histogram of compressed block sizes

MFC after:	3 weeks
2013-04-06 17:18:48 +00:00
mm
696d961cbd MFV r248217:
Merge change from vendor to reduce diff only.
ZFS dtrace probes are not supported on FreeBSD yet.

Illumos ZFS issues:
  3598 want to dtrace when errors are generated in zfs

MFC after:	3 weeks
2013-04-06 10:39:38 +00:00
dim
ed91507629 Follow up to r247960 and rr247960 by also amending ctfmerge. For the
only other case where STT_FILE symbols are used, in symit_next() in
cddl/contrib/opensolaris/tools/ctf/cvt/input.c, save the basename of the
symbol, instead of the full pathname.

Reported by:	avg
Tested by:	avg, jimharris
MFC after:	1 week
2013-04-01 21:16:32 +00:00
pfg
50c97d79d9 Dtrace: enablings on defunct providers prevent providers from unregistering
Merge change from illumos:

1368 enablings on defunct providers prevent providers from unregistering

We try to address some underlying differences between the Solaris
and FreeBSD implementations: dtrace_attach() / dtrace_detach() are
currently unimplemented in FreeBSD but the new code from illumos
makes use of taskq so some adaptations were made to dtrace_open()
and dtrace_close() to handle them appropriately.

Illumos Revision:	r13430:8e6add739e38

Reference:
https://www.illumos.org/issues/1368

Reviewed by:	gnn
Tested by:	Fabian Keil
Obtained from:	Illumos
MFC after:	3 weeks
2013-04-01 19:13:46 +00:00
gnn
9e7e2d6285 Commit a patch that fixes a problem in the #pragma statement when searching
for and loading dependent modules.  This addresses a bug seen with
io.d where it was being doubly included.

PR:		171678
Submitted by:	 Mark Johnston
MFC after:	2 weeks
2013-03-28 20:31:03 +00:00
gnn
d955ec99d7 Revert previous change in favor of a fix to the actual dtrace
libraries that addresses a #pragma issue.
2013-03-28 20:22:43 +00:00
gnn
3793680579 Remove dependency code that caused a double inclusion.
Pointed out by: rpaulo and others
2013-03-28 20:12:46 +00:00
pfg
996e9d6f74 Dtrace: dtrace.c erroneously checks for memory alignment on amd64.
Merge change from illumos:

3511 dtrace.c erroneously checks for memory alignment on amd64

Illumos Revision:	c93cc65

Reference:
https://www.illumos.org/issues/3511

Obtained from:	Illumos
MFC after:	3 weeks
2013-03-26 20:17:08 +00:00
pfg
5304ecc65d Dtrace: Add SUN MDB-like type-aware print() action.
Merge change from illumos:

1694 Add type-aware print() action

This is a very nice feature implemented in upstream Dtrace.
A complete description is available here:
http://dtrace.org/blogs/eschrock/2011/10/26/your-mdb-fell-into-my-dtrace/

This change bumps the DT_VERS_* number to 1.9.0 in
accordance to what is done in illumos.

While here also include some minor cleanups to ease further merging
and appease clang with a fix by Fabian Keil.

Illumos Revisions:	13501:c3a7090dbc16
			13483:f413e6c5d297

Reference:
https://www.illumos.org/issues/1560
https://www.illumos.org/issues/1694

Tested by:	Fabian Keil
Obtained from:	Illumos
MFC after:	1 month
2013-03-25 20:38:09 +00:00
pfg
fe83fcce74 Dtrace: add toupper()/tolower() and enhancements to lltostr().
Merge changes from illumos:

1451 DTrace needs toupper()/tolower() subroutines
1457 lltostr() D subroutine should take an optional base

This change bumps the DT_VERS_* number to 1.8.1 in
accordance to what is done in illumos.

The test suite we currently include is outdated and
doesnt support some updates in tst.subr.d which had to
be left out for now.

Illumos Revisions:	r13458 5e394d8db762
			r13459 c3454574dd1a

Reference:
https://www.illumos.org/issues/1451
https://www.illumos.org/issues/1457

Tested by:	Fabian Keil
Obtained from:	Illumos
MFC after:	1 month
2013-03-25 15:40:57 +00:00
pfg
61cbaa2ebd Dtrace: add optional size argument to tracemem().
Merge change from illumos:

1455 DTrace tracemem() should take an optional size argument

Our local enhancements to dt_print_bytes were equivalent to
those in illumos but we made it match the illumos version
to ease further code merges.

For now leave out tst.smallsize.d and tst.smallsize.d.out
since those don't seem to work cleanly on FreeBSD.

This change bumps the DT_VERS_* number to 1.7.1 in accordance
to what is done in illumos.

Illumos Revision:	13457:571b0355c2e3

Reference:
https://www.illumos.org/issues/1455

Tested by:	Fabian Keil
Obtained from:	Illumos
MFC after:	1 month
2013-03-24 19:12:08 +00:00
avg
880c47f8a7 dtrace: ensure that we can always catch a process (e.g. when -c is used)
It is not guaranteed that a program has a symbol table entry for main
and thus that it would be possible to set a breakpoint on it.

Reviewed by:	rpaulo
Discussed with:	rpaulo
MFC after:	13 days
2013-03-23 08:57:54 +00:00
mm
f1985e5cd7 Merge libzfs_core branch:
includes MFV 238590, 238592, 247580

MFV 238590, 238592:
  In the first zfs ioctl restructuring phase, the libzfs_core library was
  introduced. It is a new thin library that wraps around kernel ioctl's.
  The idea is to provide a forward-compatible way of dealing with new
  features. Arguments are passed in nvlists and not random zfs_cmd fields,
  new-style ioctls are logged to pool history using a new method of
  history logging.

  http://blog.delphix.com/matt/2012/01/17/the-future-of-libzfs/

MFV 247580 [1]:
  To address issues of several deadlocks and race conditions the locking
  code around dsl_dataset was rewritten and the interface to synctasks
  was changed.

User-Visible Changes:
  "zfs snapshot" can create more arbitrary snapshots at once (atomically)
  "zfs destroy" destroys multiple snapshots at once
  "zfs recv" has improved performance

Backward Compatibility:
  I have extended the compatibility layer to support full backward
  compatibility by remapping or rewriting the responsible ioctl arguments.
  Old utilities are fully supported by the new kernel module.

Forward Compatibility:
  New utilities work with old kernels with the following restrictions:
    - creating, destroying, holding and releasing of multiple snapshots
      at once is not supported, this includes recursive (-r) commands

Illumos ZFS issues:
  2882 implement libzfs_core
  2900 "zfs snapshot" should be able to create multiple,
       arbitrary snapshots at once
  3464 zfs synctask code needs restructuring

References:
  https://www.illumos.org/issues/2882
  https://www.illumos.org/issues/2900
  https://www.illumos.org/issues/3464 [1]

MFC after:	1 month
Sponsored by:	Hybrid Logic Inc. [1]
2013-03-21 08:38:03 +00:00
mm
7c87858955 MFV r247580:
Merge synctask code restructuring from vendor.

Modify forward and backward compatibility to support new change.

Illumos ZFS issues:
  3464 zfs synctask code needs restructuring

Sponsored by:	Hybrid Logic Ltd.
2013-03-19 12:51:18 +00:00
mm
09f7f9e4ff MFC @248461 2013-03-18 09:39:51 +00:00
mm
713c2d790d Move common zfs ioctl compatibility functions (userland) into libzfs_compat.c
Introduce additional constants for zfs ioctl versions
2013-03-18 09:32:29 +00:00
mm
da897e350b Fix working with zfs_ioctl_version in libzfs_compat.h and include mirror
lzc_ioctl_version in libzfs_core
2013-03-17 22:24:08 +00:00
mm
3d2567990a Add forwards compatibility for libzfs_core
Unsupported: creation of multiple snapshots including "zfs snapshot -r"
2013-03-17 18:33:06 +00:00
mm
6c7511b96e libzfs_core:
- provide complete backwards compatibility (old utility, new kernel)
  - add zfs_cmd_t compatibility mapping in both directions
  - determine ioctl address in zfs_ioctl_compat.c
2013-03-17 10:57:04 +00:00
mm
d1a8cea992 MFV r248266:
Import minor ZFS changes from vendor

Illumos ZFS issues:
  3604 zdb should print bpobjs more verbosely (fix zdb hang)
  3606 zpool status -x shouldn't warn about old on-disk format

MFC after:	3 days
2013-03-14 10:02:59 +00:00
mm
f7f87457f8 Update zfs.8 manpage date (missing in r247585)
MFC:	together with r247585
2013-03-14 08:18:40 +00:00
mm
ec13ab3f50 MFC @248093 2013-03-09 11:57:51 +00:00
dim
24bac1724c Fix error in r247960: actually assign the basename to match.iim_file.
Pointed out by:	avg
Pointy hat to:	dim
MFC after:	1 week
X-MFC-With:	r247960
2013-03-07 22:43:50 +00:00
dim
be01e90b71 Make ctfconvert work correctly on clang-compiled object files. Clang
puts the full original source filename in the STT_FILE entry of the ELF
symbol table, while gcc saves only the basename.

Since the DWARF DW_AT_name attribute contains the full source filename,
both for clang and gcc, ctfconvert takes just the basename of it, for
matching with the STT_FILE entry.  So when attempting to match with such
an entry, use its basename, if necessary.

Reported by:	avg
MFC after:	1 week
2013-03-07 22:16:35 +00:00