Commit Graph

377 Commits

Author SHA1 Message Date
Dimitry Andric
51ff6adc94 Follow up to r247960 and rr247960 by also amending ctfmerge. For the
only other case where STT_FILE symbols are used, in symit_next() in
cddl/contrib/opensolaris/tools/ctf/cvt/input.c, save the basename of the
symbol, instead of the full pathname.

Reported by:	avg
Tested by:	avg, jimharris
MFC after:	1 week
2013-04-01 21:16:32 +00:00
Pedro F. Giffuni
9f4c7ba460 Dtrace: enablings on defunct providers prevent providers from unregistering
Merge change from illumos:

1368 enablings on defunct providers prevent providers from unregistering

We try to address some underlying differences between the Solaris
and FreeBSD implementations: dtrace_attach() / dtrace_detach() are
currently unimplemented in FreeBSD but the new code from illumos
makes use of taskq so some adaptations were made to dtrace_open()
and dtrace_close() to handle them appropriately.

Illumos Revision:	r13430:8e6add739e38

Reference:
https://www.illumos.org/issues/1368

Reviewed by:	gnn
Tested by:	Fabian Keil
Obtained from:	Illumos
MFC after:	3 weeks
2013-04-01 19:13:46 +00:00
George V. Neville-Neil
03a9b7c085 Commit a patch that fixes a problem in the #pragma statement when searching
for and loading dependent modules.  This addresses a bug seen with
io.d where it was being doubly included.

PR:		171678
Submitted by:	 Mark Johnston
MFC after:	2 weeks
2013-03-28 20:31:03 +00:00
George V. Neville-Neil
02b3906712 Revert previous change in favor of a fix to the actual dtrace
libraries that addresses a #pragma issue.
2013-03-28 20:22:43 +00:00
George V. Neville-Neil
3e3d9dcd39 Remove dependency code that caused a double inclusion.
Pointed out by: rpaulo and others
2013-03-28 20:12:46 +00:00
Pedro F. Giffuni
f5678b698a Dtrace: dtrace.c erroneously checks for memory alignment on amd64.
Merge change from illumos:

3511 dtrace.c erroneously checks for memory alignment on amd64

Illumos Revision:	c93cc65

Reference:
https://www.illumos.org/issues/3511

Obtained from:	Illumos
MFC after:	3 weeks
2013-03-26 20:17:08 +00:00
Pedro F. Giffuni
5472787377 Dtrace: Add SUN MDB-like type-aware print() action.
Merge change from illumos:

1694 Add type-aware print() action

This is a very nice feature implemented in upstream Dtrace.
A complete description is available here:
http://dtrace.org/blogs/eschrock/2011/10/26/your-mdb-fell-into-my-dtrace/

This change bumps the DT_VERS_* number to 1.9.0 in
accordance to what is done in illumos.

While here also include some minor cleanups to ease further merging
and appease clang with a fix by Fabian Keil.

Illumos Revisions:	13501:c3a7090dbc16
			13483:f413e6c5d297

Reference:
https://www.illumos.org/issues/1560
https://www.illumos.org/issues/1694

Tested by:	Fabian Keil
Obtained from:	Illumos
MFC after:	1 month
2013-03-25 20:38:09 +00:00
Pedro F. Giffuni
730cecb05a Dtrace: add toupper()/tolower() and enhancements to lltostr().
Merge changes from illumos:

1451 DTrace needs toupper()/tolower() subroutines
1457 lltostr() D subroutine should take an optional base

This change bumps the DT_VERS_* number to 1.8.1 in
accordance to what is done in illumos.

The test suite we currently include is outdated and
doesnt support some updates in tst.subr.d which had to
be left out for now.

Illumos Revisions:	r13458 5e394d8db762
			r13459 c3454574dd1a

Reference:
https://www.illumos.org/issues/1451
https://www.illumos.org/issues/1457

Tested by:	Fabian Keil
Obtained from:	Illumos
MFC after:	1 month
2013-03-25 15:40:57 +00:00
Pedro F. Giffuni
f2e66d30b8 Dtrace: add optional size argument to tracemem().
Merge change from illumos:

1455 DTrace tracemem() should take an optional size argument

Our local enhancements to dt_print_bytes were equivalent to
those in illumos but we made it match the illumos version
to ease further code merges.

For now leave out tst.smallsize.d and tst.smallsize.d.out
since those don't seem to work cleanly on FreeBSD.

This change bumps the DT_VERS_* number to 1.7.1 in accordance
to what is done in illumos.

Illumos Revision:	13457:571b0355c2e3

Reference:
https://www.illumos.org/issues/1455

Tested by:	Fabian Keil
Obtained from:	Illumos
MFC after:	1 month
2013-03-24 19:12:08 +00:00
Andriy Gapon
083a3ffdec dtrace: ensure that we can always catch a process (e.g. when -c is used)
It is not guaranteed that a program has a symbol table entry for main
and thus that it would be possible to set a breakpoint on it.

Reviewed by:	rpaulo
Discussed with:	rpaulo
MFC after:	13 days
2013-03-23 08:57:54 +00:00
Martin Matuska
05f49d92ef Merge libzfs_core branch:
includes MFV 238590, 238592, 247580

MFV 238590, 238592:
  In the first zfs ioctl restructuring phase, the libzfs_core library was
  introduced. It is a new thin library that wraps around kernel ioctl's.
  The idea is to provide a forward-compatible way of dealing with new
  features. Arguments are passed in nvlists and not random zfs_cmd fields,
  new-style ioctls are logged to pool history using a new method of
  history logging.

  http://blog.delphix.com/matt/2012/01/17/the-future-of-libzfs/

MFV 247580 [1]:
  To address issues of several deadlocks and race conditions the locking
  code around dsl_dataset was rewritten and the interface to synctasks
  was changed.

User-Visible Changes:
  "zfs snapshot" can create more arbitrary snapshots at once (atomically)
  "zfs destroy" destroys multiple snapshots at once
  "zfs recv" has improved performance

Backward Compatibility:
  I have extended the compatibility layer to support full backward
  compatibility by remapping or rewriting the responsible ioctl arguments.
  Old utilities are fully supported by the new kernel module.

Forward Compatibility:
  New utilities work with old kernels with the following restrictions:
    - creating, destroying, holding and releasing of multiple snapshots
      at once is not supported, this includes recursive (-r) commands

Illumos ZFS issues:
  2882 implement libzfs_core
  2900 "zfs snapshot" should be able to create multiple,
       arbitrary snapshots at once
  3464 zfs synctask code needs restructuring

References:
  https://www.illumos.org/issues/2882
  https://www.illumos.org/issues/2900
  https://www.illumos.org/issues/3464 [1]

MFC after:	1 month
Sponsored by:	Hybrid Logic Inc. [1]
2013-03-21 08:38:03 +00:00
Martin Matuska
07091d8f14 MFV r247580:
Merge synctask code restructuring from vendor.

Modify forward and backward compatibility to support new change.

Illumos ZFS issues:
  3464 zfs synctask code needs restructuring

Sponsored by:	Hybrid Logic Ltd.
2013-03-19 12:51:18 +00:00
Martin Matuska
876a84e867 MFC @248461 2013-03-18 09:39:51 +00:00
Martin Matuska
6f4accc2de Move common zfs ioctl compatibility functions (userland) into libzfs_compat.c
Introduce additional constants for zfs ioctl versions
2013-03-18 09:32:29 +00:00
Martin Matuska
d97b41582d Fix working with zfs_ioctl_version in libzfs_compat.h and include mirror
lzc_ioctl_version in libzfs_core
2013-03-17 22:24:08 +00:00
Martin Matuska
67ebc12d49 Add forwards compatibility for libzfs_core
Unsupported: creation of multiple snapshots including "zfs snapshot -r"
2013-03-17 18:33:06 +00:00
Martin Matuska
e2b4467975 libzfs_core:
- provide complete backwards compatibility (old utility, new kernel)
  - add zfs_cmd_t compatibility mapping in both directions
  - determine ioctl address in zfs_ioctl_compat.c
2013-03-17 10:57:04 +00:00
Martin Matuska
0afa556b1b MFV r248266:
Import minor ZFS changes from vendor

Illumos ZFS issues:
  3604 zdb should print bpobjs more verbosely (fix zdb hang)
  3606 zpool status -x shouldn't warn about old on-disk format

MFC after:	3 days
2013-03-14 10:02:59 +00:00
Martin Matuska
1a5160106d Update zfs.8 manpage date (missing in r247585)
MFC:	together with r247585
2013-03-14 08:18:40 +00:00
Martin Matuska
a03fbc7ecf MFC @248093 2013-03-09 11:57:51 +00:00
Dimitry Andric
e86b103cc6 Fix error in r247960: actually assign the basename to match.iim_file.
Pointed out by:	avg
Pointy hat to:	dim
MFC after:	1 week
X-MFC-With:	r247960
2013-03-07 22:43:50 +00:00
Dimitry Andric
8b0d8e9f2f Make ctfconvert work correctly on clang-compiled object files. Clang
puts the full original source filename in the STT_FILE entry of the ELF
symbol table, while gcc saves only the basename.

Since the DWARF DW_AT_name attribute contains the full source filename,
both for clang and gcc, ctfconvert takes just the basename of it, for
matching with the STT_FILE entry.  So when attempting to match with such
an entry, use its basename, if necessary.

Reported by:	avg
MFC after:	1 week
2013-03-07 22:16:35 +00:00
Martin Matuska
db49d41d8a Move libzfs compat functions to libzfs_compat.h
This header is used by both libzfs_core and libzfs libraries
2013-03-06 10:40:50 +00:00
Martin Matuska
400c4069a5 MFV r247845:
Import ZFS bpobj bugfix from vendor.

Illumos ZFS issues:
  3603 panic from bpobj_enqueue_subobj()
  3604 zdb should print bpobjs more verbosely

References:
  https://www.illumos.org/issues/3603
  https://www.illumos.org/issues/3604

MFC after:	1 week
2013-03-05 18:54:41 +00:00
Martin Matuska
dce1a726f2 WiP merge of libzfs_core (MFV r238590, r238592)
not yet working, ioctl handling needs to be changed
2013-03-05 08:09:53 +00:00
Martin Matuska
4abd59512a MFV r247316:
Merge new read-only zfs properties from vendor (illumos)

Illumos ZFS issues:
  3588 provide zfs properties for logical (uncompressed) space used and
       referenced

References:
  https://www.illumos.org/issues/3588

MFC after:	2 weeks
2013-03-01 21:58:51 +00:00
Martin Matuska
bb508e7732 Fix the zfs_ioctl compat layer to support zfs_cmd size change introduced
in r247265 (ZFS deadman thread). Both new utilities now support the old
kernel and new kernel properly detects old utilities.

For future backwards compatibility, the vfs.zfs.version.ioctl read-only
sysctl has been introduced. With this sysctl zfs utilities will be able
to detect the ioctl interface version of the currently loaded zfs module.

As a side effect, the zfs utilities between r247265 and this revision don't
support the old kernel module. If you are using HEAD newer or equal than
r247265, install the new kernel module (or whole kernel) first.

MFC after:	10 days
2013-03-01 09:42:58 +00:00
Martin Matuska
24245e76ea MFV 247176, 247178, 247315:
Import metaslab_sync() speedup from vendor (illumos).

Illumos ZFS issues:
  3552 condensing one space map burns 3 seconds of CPU in spa_sync() thread
  3564 spa_sync() spends 5-10% of its time in metaslab_sync() (when not
       condensing)
  3578 transferring the freed map to the defer map should be constant time
  3579 ztest trips assertion in metaslab_weight()

References:
  https://www.illumos.org/issues/3552
  https://www.illumos.org/issues/3564
  https://www.illumos.org/issues/3578
  https://www.illumos.org/issues/3579

MFC after:	2 weeks
2013-02-27 14:45:23 +00:00
Martin Matuska
e70664bafc MFV v242732:
Merge the ZFS I/O deadman thread from vendor (illumos).
This feature panics the system on hanging ZFS I/O, helps debugging
and resumes failed service.

The panic behavior can be controlled with the loader-only tunables:
vfs.zfs.deadman_enabled (enable or disable panic on stalled ZFS I/O)
vfs.zfs.deadman_synctime (expiration time for stalled ZFS I/O)

By default, ZFS I/O deadman is enabled by default on amd64 and i386
excluding virtual guest machines.

Illumos ZFS issues:
  3246 ZFS I/O deadman thread

References:
  https://www.illumos.org/issues/3246

MFC after:	2 weeks
2013-02-25 12:33:31 +00:00
Justin T. Gibbs
cf4e0cc15e Orphaned processes that are being traced are killed by the
kernel.  Properly restore, continue, and detach from processes
being DTraced when DTrace exits with an error so the program
being inspected is not terminated.

cddl/contrib/opensolaris/cmd/dtrace/dtrace.c:
	In fatal(), the generic error handler, close the DTrace
	handle as is done in the "probe/script" error handler
	dfatal().  fatal() can be invoked after DTrace attaches
	to processes (e.g. a script specified by command line
	argument can't be found) and closing the handle will
	release them.

Submitted by:	Spectra Logic Corporation
Reviewed by:	rpaulo, gnn
2013-02-20 17:46:38 +00:00
Xin LI
f667ff300d Use the right year.
Noticed by:	Denis Ahrens <denis h3q com>
2013-02-11 18:22:26 +00:00
Martin Matuska
ff20578569 MFV r246392:
Import vendor ZFS bugfix fixing a possible deadlock in arc_read().

Illumos ZFS issues:
  3498 panic in arc_read(): !refcount_is_zero(&pbuf->b_hdr->b_refcnt)

References:
  https://www.illumos.org/issues/3498

MFC after:	2 weeks
2013-02-11 12:42:11 +00:00
Martin Matuska
fd9778c236 MFV r246388:
Import vendor bugfixes

Illumos ZFS issues:
  3422 zpool create/syseventd race yield non-importable pool
  3425 first write to a new zvol can fail with EFBIG

References:
  https://www.illumos.org/issues/3422
  https://www.illumos.org/issues/3425

MFC after:	2 weeks
2013-02-10 19:32:55 +00:00
Martin Matuska
1af420e000 Backport vendor changes in zfs(8) manual page (MFV r246389)
Illumos ZFS issues:
  3380 zfs man page: documentation for zfs allow is confusing

References:
  https://www.illumos.org/issues/3380

MFC after:	2 weeks
2013-02-10 17:10:07 +00:00
Gavin Atkinson
9a71ddccd0 Correct spelling of "daemon". No .Dd bump.
Noticed by:	Nathan Rich <Nathan.Rich dynastysystems com>
MFC after:	3 days
2013-02-10 14:28:07 +00:00
Joel Dahl
1848dd2aec mdoc: Remove EOL whitespace. 2013-02-09 07:01:05 +00:00
Xin LI
ef17620fc8 MFV r245512:
* Illumos zfs issue #3035 [1] LZ4 compression support in ZFS.

LZ4 is a new high-speed BSD-licensed compression algorithm created
by Yann Collet that delivers very high compression and decompression
performance compared to lzjb (>50% faster on compression, >80% faster
on decompression and around 3x faster on compression of incompressible
data), while giving better compression ratio [1].

This version of LZ4 corresponds to upstream's [2] revision 85.

Please note that for obvious reasons this is not backward read
compatible.  This means once a pool have LZ4 compressed data, these
data can no longer be read by older ZFS implementations.

Local changes:

 - On-stack hash table disabled and using kernel slab allocator
   instead, at this time.  This requires larger kernel thread stack
   for zio workers.  This may change in the future should we adjusted
   the zio workers' thread stack size.
 - likely and unlikely will be undefined if they are already defined,
   this is required for i386 XEN build.
 - Removed De Bruijn sequence based __builtin_ctz family of builtins
   in favor of the latter.  Both GCC and clang supports these builtins.
 - Changed the way the LZ4 code detects endianness.
 - Manual pages modifications to mention the feature based on Illumos
   counterpart.
 - Boot loader changes to make it support LZ4 decompression.

[1] https://www.illumos.org/issues/3035
[2] http://code.google.com/p/lz4/source/list

Obtained from:	Illumos (13921:9d721847e469)
Tested on:	FreeBSD/amd64
MFC after:	1 month
2013-02-09 06:39:28 +00:00
Brooks Davis
afd9b4631f Add a new LIBRARIES_ONLY make variable to disable the build and install
of files other than the actual libraries.

Use LIBRARIES_ONLY to supress the inclusion of files in the lib32
distribution that are duplicates of files in base.

Sponsored by:	DARPA, AFRL
Reviewed by:	emaste
2013-01-17 17:27:10 +00:00
Steven Hartland
0e8f70b4bc Reports pools which have a removed l2cache disk under -x as this is what
happens when a cache device is dropped for any reason.

Reviewed by:	pjd
Approved by:	pjd (mentor)
MFC after:	2 weeks
2013-01-15 23:34:24 +00:00
Xin LI
290a1ba9a4 The current ZFS code expects ddt_zap_count to always succeed by asserting
the underlying zap_count() to return no errors.  However, it is possible
that the pool reaches to such a state where zap_count would return error,
leading to panics when a pool is imported.

This commit changes the ddt_zap_count to return error returned from
zap_count and handle the error appropriately.  With this change, it's now
possible to let zpool rollback damaged transaction groups and import the
pool.

Obtained from:	ZFS on Linux github (e8fd45a0f9)
MFC after:	1 month
2013-01-10 19:26:56 +00:00
Pawel Jakub Dawidek
1bba6bdf13 Allow to create pool even if mount point directory is not empty if -f is given.
Obtained from:	WHEEL Systems
2012-12-30 13:47:40 +00:00
Martin Matuska
4d50603f6b MFV r244245:
Merge two zdb bugfixes from vendor (illumos)

illumos-gate 13894:f4af77f6bbd2
3397 zdb <pool> <objnum> output is too verbose
3398 zdb can't dump feature flags zap objects

References:
https://www.illumos.org/issues/3397
https://www.illumos.org/issues/3398

MFC after:	1 week
2012-12-15 10:02:11 +00:00
Steven Hartland
425fb6dbaf Fixes zfs receive errors caused by snapshot replication being processed in a
random order instead of creation order.

Eliminates needless filesystem renames caused by removed parent snapshots
which subsequently causes many more errors.

PR:		kern/172259
Submitted by:	Steven Hartland
Reviewed by:	pjd (mentor)
Approved by:	pjd (mentor)
MFC after:	2 weeks
2012-12-13 22:03:07 +00:00
Martin Matuska
6e5bba4ae1 Update manpage dates in zfs.8 and zpool.8
MFC after:	2 weeks
2012-11-26 13:16:55 +00:00
Martin Matuska
7faa32552f MFV r243395:
Introduce a new dataset aclmode setting "restricted" to protect ACL's
being destroyed or corrupted by a drive-by chmod.

illumos-gate 13889:a67716f16746
3254 add support in zfs for aclmode=restricted

References:
https://www.illumos.org/issues/3254

MFC after:	2 weeks
2012-11-26 12:24:39 +00:00
Martin Matuska
dd801aa546 MFV r243013 and r243267:
Import the zio nop-write improvement from Illumos. To reduce I/O,
nop-write omits overwriting data if the checksum (cryptographically
secure) of new data matches the checksum of existing data.
It also saves space if snapshots are in use.

It currently works only on datasets with enabled compression, disabled
deduplication and sha256 checksums.

IllumOS 13887:196932ec9e6a and 13888:7204b3392a58
3236 zio nop-write

References:
https://www.illumos.org/issues/3236

MFC after:	2 weeks
2012-11-25 16:32:07 +00:00
Martin Matuska
64ad6bf196 zfs sha256 checksum is missing in zfs.8 manpage
MFC after:	3 days
2012-11-25 11:26:36 +00:00
Martin Matuska
2f06dfc9a3 MFV r243012:
Illumos 13886:e3261d03efbf

3349 zpool upgrade -V bumps the on disk version number, but leaves
     the in core version

References:
https://www.illumos.org/issues/3349

MFC after:	1 week
2012-11-25 10:53:42 +00:00
Sergey Kandaurov
9098689d87 Sort SEE ALSO xrefs by the manual section number and add a missing comma. 2012-11-15 20:50:18 +00:00
Martin Matuska
82e5de4d20 Move zpool-features manual page from section 5 to section 7
and fix references

Reported by:	pluknet
MFC after:	1 week
2012-11-14 01:16:29 +00:00