Commit Graph

506 Commits

Author SHA1 Message Date
Andriy Gapon
6ed9e9f32f zfs: correctly calculate dn_bonuslen for saving SAs to disk
Since all attribute values start at 8-byte aligned boundary, we would
previously incorrectly calculate dn_bonuslen if any attribute but the
last had a variable-length value with length not multiple of 8.

Reported by:	Nicolas Rachinsky <fbsd-mas-0@ml.turing-complete.org>
Tested by:	Nicolas Rachinsky <fbsd-mas-0@ml.turing-complete.org>
Reviewed by:	Matthew Ahrens <mahrens@delphix.com> (for upstream)
MFC after:	2 weeks
2012-09-18 08:02:54 +00:00
Andriy Gapon
ea559fb573 zfs: allow both DEBUG and ZFS_DEBUG to be defined on command line
Discussed with:	pjd
MFC after:	10 days
2012-09-18 08:00:56 +00:00
Martin Matuska
4c5238d576 Merge recent zfs vendor changes, sync code and adjust userland DEBUG.
Illumos issued covered:
1884 Empty "used" field for zfs *space commands
3006 VERIFY[S,U,P] and ASSERT[S,U,P] frequently check if first argument
     is zero
3028 zfs {group,user}space -n prints (null) instead of numeric GID/UID
3048 zfs {user,group}space [-s|-S] is broken
3049 zfs {user,group}space -t doesn't really filter the results
3060 zfs {user,group}space -H output isn't tab-delimited
3061 zfs {user,group}space -o doesn't use specified fields order
3064 usr/src/cmd/zpool/zpool_main.c misspells "successful"
3093 zfs {user,group}space's -i is noop
3098 zfs userspace/groupspace fail without saying why when run as non-root

References:
  https://www.illumos.org/issues/ + [issue_id]

Obtained from:	illumos (vendor/illumos, vendor/illumos-sys)
MFC after:	2 weeks
2012-09-12 18:05:43 +00:00
Andriy Gapon
9c5c3cafbd zfs: fix sa_modify_attrs handling of variable-sized attributes
- skip length_idx index for a replaced variable-sized attribute
- skip length_idx index for a removed variable-sized attribute
- also re-arranged code to make sure that length_idx is always
  incremented for variable-sized attributes
- additionally add an assertion that the number of actually produced
  attributes is the same as the expected number of resulting
  attributes

In cooperation with:	Matthew Ahrens <mahrens@delphix.com>
Tested by:	Trent Nelson <trent@snakebite.org>
Reviewed by:	Matthew Ahrens <mahrens@delphix.com> (for upstream)
To do:		get this upstreamed
MFC after:	2 weeks
2012-09-11 07:07:52 +00:00
Martin Matuska
6643637f67 Add assfail() and assfail3() to the opensolaris module.
Remove obsoleted intermediate cddl/compat/opensolaris/sys/debug.h.

MFC after:	2 weeks
2012-09-10 10:24:57 +00:00
Martin Matuska
4a24a25b2f Merge recent vendor changes and sync code:
1862 incremental zfs receive fails for sparse file > 8PB
3112 ztest does not honor ZFS_DEBUG
3122 zfs destroy filesystem should prefetch blocks
3129 'zpool reopen' restarts resilvers
3130 ztest failure: Assertion failed:
       0 == dmu_objset_destroy(name, B_FALSE) (0x0 == 0x10)

References:
  https://www.illumos.org/issues/1862
  https://www.illumos.org/issues/3112
  https://www.illumos.org/issues/3122
  https://www.illumos.org/issues/3129
  https://www.illumos.org/issues/3130

Obtained from:	illumos (vendor/illumos, vendor/illumos-sys)
MFC after:	2 weeks
2012-09-05 12:02:09 +00:00
Ed Schouten
3f0fb35417 Use a proper destructor function.
When calling a revoke(2) on a dtrace device, dtrace_close() could be
called, even if threads are still stuck in the device. Defer the actual
deallocation of datastructures to the cdevpriv destructor.

While there, remove the unneeded D_TRACKCLOSE and D_NEEDMINOR flags. For
the helper device, we never need it. For the regular dtrace devices, we
only need these flags on FreeBSD pre-8.

MFC after:	1 month
2012-08-28 18:33:12 +00:00
Martin Matuska
6e767def16 Merge recent vendor changes:
3100 zvol rename fails with EBUSY when dirty
3104 eliminate empty bpobjs
3120 zinject hangs in zfsdev_ioctl() due to uninitialized zc

References:
  https://www.illumos.org/issues/3100
  https://www.illumos.org/issues/3104
  https://www.illumos.org/issues/3120

Obtained from:	illumos (vendor/illumos, vendor/illumos-sys)
MFC after:	2 weeks
2012-08-28 12:25:37 +00:00
Martin Matuska
671303c6d5 Merge recent vendor changes:
3086 unnecessarily setting DS_FLAG_INCONSISTENT on async destroyed datasets
3090 vdev_reopen() during reguid causes vdev to be treated as corrupt
3102 vdev_uberblock_load() and vdev_validate() may read the wrong label

Referenes:
  https://www.illumos.org/issues/3086
  https://www.illumos.org/issues/3090
  https://www.illumos.org/issues/3102

PR:		kern/170912, kern/170914
Obtained from:	illumos (changeset #13776, #13777)
MFC after:	2 weeks
2012-08-23 19:32:57 +00:00
Martin Matuska
bb9b1f7a8b Backport fix for vendor issue #3085
3085 zfs diff panics, then panics in a loop on booting

References:
  https://www.illumos.org/issues/3085

PR:		kern/170763
Obtained from:	ssh://anonhg@hg.illumos.org/illumos-gate (r13772)
MFC after:	1 week
2012-08-19 09:59:41 +00:00
Hans Petter Selasky
07da61a6cc Streamline use of cdevpriv and correct some corner cases.
1) It is not useful to call "devfs_clear_cdevpriv()" from
"d_close" callbacks, hence for example read, write, ioctl and
so on might be sleeping at the time of "d_close" being called
and then then freed private data can still be accessed.
Examples: dtrace, linux_compat, ksyms (all fixed by this patch)

2) In sys/dev/drm* there are some cases in which memory will
be freed twice, if open fails, first by code in the open
routine, secondly by the cdevpriv destructor. Move registration
of the cdevpriv to the end of the drm open routines.

3) devfs_clear_cdevpriv() is not called if the "d_open" callback
registered cdevpriv data and the "d_open" callback function
returned an error. Fix this.

Discussed with:	phk
MFC after:	2 weeks
2012-08-15 16:19:39 +00:00
Marius Strobl
787c338407 Include <vm/vm_param.h> for PA_LOCK_COUNT in order to fix kernel build
with options ZFS after r239065.
2012-08-05 20:19:27 +00:00
Martin Matuska
e9832bb1da Partial MFV (illumos-gate 13753:2aba784c276b)
2762 zpool command should have better support for feature flags

References:
https://www.illumos.org/issues/2762

MFC after:	2 weeks
2012-07-30 23:14:24 +00:00
Edward Tomasz Napierala
b75ca29147 Make ZVOL resizing ('zfs set volsize') properly resize the GEOM provider.
Sponsored by:	FreeBSD Foundation
2012-07-20 16:56:34 +00:00
Pawel Jakub Dawidek
92484a2615 vdev_io_done stage is not used for ioctls.
MFC after:	1 week
2012-07-04 17:39:29 +00:00
Martin Matuska
a6a8d8377f Expose scrub and resilver tunables.
This allows the user to tune the priority trade-off between scrub/resilver
and other ZFS I/O.

MFC after:	2 weeks
Discussed with:	pjd
2012-07-02 07:27:14 +00:00
Pedro F. Giffuni
9a9df34345 Bump dtrace_helper_actions_max from 32 to 128
Dave Pacheco from Joyent (and Dtrace.org) bumped the cap to 1024 but,
according to his blog, 128 is the recommended minimum.

For now bump it safely to 128 although we may have to bump it further
if there is demand in the future.

Reference:

http://www.illumos.org/issues/2558
http://dtrace.org/blogs/dap/2012/01/50/where-does-your-node-program-spend-its-time/
2012-06-29 18:49:14 +00:00
Pedro F. Giffuni
675cf9154b Bring llquantize support into Dtrace.
Bryan Cantrill implemented the equivalent of semi-log graph
paper for Dtrace so llquantize will use one logarithmic and
one linear scale.

Special thanks to Mark Peek for providing fix to an
assertion and to Fabian Keill for testing the port.

Illumos Revision:	13355:15b74a2a9a9d

Reference:
https://www.illumos/issues/905

Obtained from:	Illumos
Tested by:	Fabian Keill, mp
MFC after:	4 days
2012-06-27 04:39:30 +00:00
Martin Matuska
de37372f73 Import Illumos revision 13736:9f1d48e1681f
2901 ZFS receive fails for exabyte sparse files

References:
https://www.illumos.org/issues/2901

Obtained from:	illumos (issue #2901)
MFC after:	1 week
2012-06-22 20:42:11 +00:00
Martin Matuska
2d9cf57e18 Introduce "feature flags" for ZFS pools (bump SPA version to 5000).
Add first feature "com.delphix:async_destroy" (asynchronous destroy
of ZFS datasets).
Implement features support in ZFS boot code.

Illumos revisions merged:
13700:2889e2596bd6
13701:1949b688d5fb
2619 asynchronous destruction of ZFS file systems
2747 SPA versioning with zfs feature flags

References:
https://www.illumos.org/issues/2619
https://www.illumos.org/issues/2747

Obtained from:	illumos (issue #2619, #2747)
MFC after:	1 month
2012-06-11 11:35:22 +00:00
Pawel Jakub Dawidek
25892bfc61 ds_guid of 0 is special, as it is used by snapshot receive code to
differentiate between an incremental and full stream.
Be sure not to generate guid equal to 0.

Reported by:	someone who saw 0 being generated as 64bit random guid
MFC after:	3 days
2012-06-09 20:16:19 +00:00
Pawel Jakub Dawidek
97e9ad8ec4 Tighten up the assertion: because size can't be 0 and even if sm_space is equal
to sm_size, any 'sm_space - size' will be less than sm_size.

MFC after:	3 days
2012-05-29 18:11:45 +00:00
Pawel Jakub Dawidek
837a617728 Eliminate 'where' argument, we don't use it.
MFC after:	3 days
2012-05-29 18:09:14 +00:00
Pawel Jakub Dawidek
8ac2669cc8 Remove unused variable.
MFC after:	3 days
2012-05-29 18:05:24 +00:00
Pawel Jakub Dawidek
e21c77d804 Remove unused sysctl.
MFC after:	3 days
2012-05-29 17:53:11 +00:00
Martin Matuska
2182d44714 Import illumos changeset 13570:3411fd5f1589
1948 zpool list should show more detailed pool information

Display per-vdev information with "zpool list -v".
The added expandsize property has currently no value on FreeBSD.
This changeset allows adding expansion support to individual vdevs
in the future.

References:
https://www.illumos.org/issues/1948

Obtained from:	illumos (issue #1948)
MFC after:	2 weeks
2012-05-27 16:00:00 +00:00
Martin Matuska
d9727dc29c Import illumos changeset 13605:b5c2b5db80d6 (partial)
763 FMD msg URLs should refer to something visible

Replace sun.com URL's with illumos.org

References:
https://www.illumos.org/issues/763

Obtained from:	illumos (issue #763)
MFC after:	1 week
2012-05-27 12:31:57 +00:00
Edward Tomasz Napierala
9280affe16 Fix enforcement of file size limit with O_APPEND on ZFS.
vn_rlimit_fsize takes uio->uio_offset and uio->uio_resid into account
when determining whether given write would exceed RLIMIT_FSIZE.

When APPEND flag is specified, ZFS updates uio->uio_offset to point to the
end of file.

But this happens after a call to vn_rlimit_fsize, so vn_rlimit_fsize check
can be rendered ineffective by thread that opens some file with O_APPEND
and lseeks below RLIMIT_FSIZE before calling write.

Submitted by:	Mateusz Guzik <mjguzik at gmail dot com>
MFC after:	2 weeks
2012-05-22 10:54:42 +00:00
Martin Matuska
a837775a9e Import illumos changeset 13686:4bc0783f6064
2703 add mechanism to report ZFS send progress

If the zfs send command is used with the -v flag, the amount of bytes
transmitted is reported in per second updates.

References:
https://www.illumos.org/issues/2703

Obtained from:	illumos (issue #2703)
MFC after:	2 weeks
2012-05-10 10:39:45 +00:00
Marius Strobl
35225ae651 Partially revert r232938; ZFS only requires nfs4 but not posix1e.
Submitted by:	jhb
2012-04-29 16:21:47 +00:00
Ryan Stone
c6024848dd Implement the D "cpu" variable, which returns curcpu. I have chosen not
to follow the example of OpenSolaris and its descendants, which implemented
cpu as an inline that took a value out of curthread.  At certain points in
the FreeBSD scheduler curthread->td_oncpu will no longer be valid (in
particukar, just before the thread gets descheduled) so instead I have
implemented this as its own built-in variable.

Sponsored by:	Sandvine Inc.
MFC after:	1 week
2012-04-26 01:07:03 +00:00
Edward Tomasz Napierala
af6e6b87ad Remove unused thread argument to vrecycle().
Reviewed by:	kib
2012-04-23 14:10:34 +00:00
Attilio Rao
a0f2c37b6f - Introduce a cache-miss optimization for consistency with other
accesses of the cache member of vm_object objects.
- Use novel vm_page_is_cached() for checks outside of the vm subsystem.

Reviewed by:	alc
MFC after:	2 weeks
X-MFC:		r234039
2012-04-09 17:05:18 +00:00
Andriy Gapon
70542ee01f zfs_ioctl: no need for ddi_copyin/out here because sys_ioctl handles that
On FreeBSD the direct ioctl argument is automatically copied in/out
as necesary by the kernel ioctl entry point.

PR:		kern/164445
Submitted by:	Luis Garces-Erice <lge@ieee.org>
Tested by:	Attila Nagy <bra@fsn.hu>
MFC after:	5 days
2012-04-05 07:59:59 +00:00
Oleksandr Tymoshenko
5083ce5c09 Add MIPS support to cddl/contrib part:
- header and stub .c file for fasttrap module. It's not supported on
    MIPS yet, but there is no way to disable support completely
- Do as amd64 trying to limit allocated memory
2012-03-24 04:52:18 +00:00
Adrian Chadd
368a79ddd0 Add dependencies onto acl_posix1e and acl_nfs4. 2012-03-13 20:29:04 +00:00
Martin Matuska
e7af90ab00 Analogous to r232059, add a parameter for the ZFS file system:
allow.mount.zfs:
	allow mounting the zfs filesystem inside a jail

This way the permssions for mounting all current VFCF_JAIL filesystems
inside a jail are controlled wia allow.mount.* jail parameters.

Update sysctl descriptions.
Update jail(8) and zfs(8) manpages.

TODO:	document the connection of allow.mount.* and VFCF_JAIL for kernel
	developers

MFC after:	10 days
2012-02-26 16:30:39 +00:00
Martin Matuska
4ee8a13704 Revert r230913 and r230914.
The initialization was correct, the problem needs deeper analysis.
2012-02-03 13:40:51 +00:00
Martin Matuska
8b152ded1c Add copyright information on last commits to comply with CDDL.
Discussed with:	pluknet@
MFC after:	3 days
2012-02-02 16:33:58 +00:00
Martin Matuska
3ce26884ba Fix out of bounds write causing random panics,
uncovered by the change in r230256

Reviewed by:	pluknet@
MFC after:	3 days
2012-02-02 16:18:40 +00:00
Kip Macy
ad69d4e266 always exclude data bufs regardless of debug settings 2012-01-29 00:19:19 +00:00
Kip Macy
cc0021eb34 add tunable for developers working on areas outside of ZFS
to further reduce core size by excluding ARC metadata buffers
from core dumps
2012-01-28 17:41:42 +00:00
Kip Macy
263811f724 exclude kmem_alloc'ed ARC data buffers from kernel minidumps on amd64
excluding other allocations including UMA now entails the addition of
a single flag to kmem_alloc or uma zone create

Reviewed by:	alc, avg
MFC after:	2 weeks
2012-01-27 20:18:31 +00:00
Martin Matuska
538251bbf6 Merge illumos revisions 13572, 13573, 13574:
Rev. 13572:
disk sync write perf regression when slog is used post oi_148 [1]

Rev. 13573:
crash during reguid causes stale config [2]
allow and unallow missing from zpool history since removal of pyzfs [5]

Rev. 13574:
leaking a vdev when removing an l2cache device [3]
memory leak when adding a file-based l2arc device [4]
leak in ZFS from metaslab_group_create and zfs_ereport_checksum [6]

References:
https://www.illumos.org/issues/1909 [1]
https://www.illumos.org/issues/1949 [2]
https://www.illumos.org/issues/1951 [3]
https://www.illumos.org/issues/1952 [4]
https://www.illumos.org/issues/1953 [5]
https://www.illumos.org/issues/1954 [6]

Obtained from:	illumos (issues #1909, #1949, #1951, #1952, #1953, #1954)
MFC after:	2 weeks
2012-01-24 23:09:54 +00:00
Pawel Jakub Dawidek
1698a6aec9 Dramatically optimize listing snapshots when user requests only snapshot
names and wants to sort them by name, ie. when executes:

	# zfs list -t snapshot -o name -s name

Because only name is needed we don't have to read all snapshot properties.

Below you can find how long does it take to list 34509 snapshots from a single
disk pool before and after this change with cold and warm cache:

before:

	# time zfs list -t snapshot -o name -s name > /dev/null
	cold cache: 525s
	warm cache: 218s

after:

	# time zfs list -t snapshot -o name -s name > /dev/null
	cold cache: 1.7s
	warm cache: 1.1s

MFC after:	1 week
2012-01-21 21:12:53 +00:00
Pawel Jakub Dawidek
b636ebaa6a By default turn off prefetch when listing snapshots.
In my tests it makes listing snapshots 19% faster with cold cache and
47% faster with warm cache.

MFC after:	1 week
2012-01-20 22:04:59 +00:00
Sergey Kandaurov
37c2842272 Fix the "lock &zrl->zr_mtx already initialized" assertion by initializing
the allocated memory before calling mtx_init(9) on mtx pointing to it.
Otherwize, random contents of uninitialized memory might occasionally
trigger the assertion.

Reported by:	Pavel Polyakov <bsd kobyla org>
Reviewed by:	pjd
MFC after:	1 week
2012-01-17 06:23:25 +00:00
Pawel Jakub Dawidek
62859c9061 - Allow to change vfs.zfs.arc_meta_limit at runtime.
- Change vfs.zfs.arc_meta_used from CTLFLAG_RDTUN to CTLFLAG_RD, as it is
  not a tunable.

MFC after:	3 days
2012-01-05 22:16:41 +00:00
Dimitry Andric
a5988eb997 In sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c, check the
the number of links against LINK_MAX (which is INT16_MAX), not against
UINT32_MAX.  Otherwise, the constant would implicitly be converted to
-1.

Reviewed by:	pjd
MFC after:	1 week
2012-01-03 20:53:07 +00:00
Pawel Jakub Dawidek
bb265163b2 From time to time people report space map corruption resulting in panic
(ss == NULL) on pool import. I had such a panic recently. With current version
of ZFS it is still possible to import the pool in readonly mode and backup
all the data, but in case it is impossible for some reason add tunable
vfs.zfs.space_map_last_hope, which when set to '1' will tell ZFS to remove
colliding range and retry. This seems to have worked for me, but I consider
it highly risky to use.

MFC after:	1 week
2011-12-18 12:27:45 +00:00