Commit Graph

488 Commits

Author SHA1 Message Date
delphij
91643324a9 MFC r268079: MFV r267566:
4390 i/o errors when deleting filesystem/zvol can lead to space map
     corruption
2014-07-15 05:00:46 +00:00
delphij
9d1dc5bcc9 MFC r268075: MFV r267565:
4757 ZFS embedded-data block pointers ("zero block compression")
4913 zfs release should not be subject to space checks
2014-07-15 04:53:34 +00:00
delphij
5ed76404fe MFC r260142: MFV r258972:
4373 add block contents print to zstreamdump
2014-07-15 04:44:06 +00:00
delphij
38554d6064 MFC r266771: MFV r266766:
Add a new zfs property, "redundant_metadata" which can have values "all" or
"most".  The default will be "all", which is the current behavior.  When set
to all, ZFS stores an extra copy of all metadata.  If a single on-disk block
is corrupt, at worst a single block of user data (which is recordsize bytes
long) can be lost.

Setting to "most" will cause us to only store 1 copy of level-1 indirect
blocks of user data files.  This can improve performance of random writes,
because less metadata has to be written.  In practice,  at worst about
100 blocks (of recordsize bytes each) of user data can be lost if a single
on-disk block is corrupt.

The exact behavior of which metadata blocks are stored redundantly may change
in future releases.

Illumos issue: 3835 zfs need not store 2 copies of all metadata
2014-07-15 04:39:55 +00:00
delphij
7afd1db032 MFC r267572: MFV r249332 (illumos-gate 14005:55fc53126003)
Illumos ZFS issues:
  3654 zdb should print number of ganged blocks
2014-07-15 04:33:11 +00:00
rpaulo
0cff4b03dd MFC 267929, 267937, 267939, 267940, 267941, 267942, 267987, 268006:
2915 DTrace in a zone should see "cpu", "curpsinfo", et al
 2916 DTrace in a zone should be able to access fds[]
 2917 DTrace in a zone should have limited provider access
 4477 DTrace should speak JSON
 Add stubs for CTF functions which are not yet implemented.
 4474 DTrace Userland CTF Support
 4475 DTrace userland Keyword
 4476 DTrace tests should be better citizens
 4479 pid provider types
 4480 dof emulation is missing checks
 4471 DTrace count() with histogram
 4472 DTrace full width distribution histograms
 4473 DTrace frequency trails
2014-07-12 22:56:41 +00:00
pfg
93d41081e3 MFC r267513:
Merge from r258379 missed the tests.

4248 dtrace(1M) should never create DOF with empty probes section
4249 Only probes from the first DTrace object file will be included

Illumos Revision:	54a20ab41aadcb81c53e72fc65886e964e9add59
2014-06-20 15:40:13 +00:00
delphij
c49d771309 MFC r266520:
Explicitly link libzfs against libavl as it is done in OpenSolaris
(4543:12bb2876a62e).  Without this, some third party applications
may break because the lack of AVL related symbols.

FreeBSD base system are not affected because the FreeBSD ZFS command
line tools were all linked against libavl and thus hide the underlying
issue.

PR:	bin/183081
2014-05-27 18:22:52 +00:00
markj
644a04942b MFC r262329:
Define the KM_NORMALPRI flag for kmem_alloc(), as it is used in some
upstream DTrace code.

MFC r262330:
1452 DTrace buffer autoscaling should be less violent

illumos/illumos-gate@6fb4854bed
2014-05-25 18:19:57 +00:00
mav
058b9b78b9 MFC r265689:
Import adapted OpenSolaris' thread pool API implementation.

The thread pool is used by libzfs to implement parallel disk scanning.
Without this change our dummy wrapper made `zpool import ZZZ` command to
scan all disks sequentially from the single thread when searching for pools.
This change makes it use two threads per CPU, same as in OpenSolaris.

On system with 200 HDDs this change reduces ZFS pool import time from 35
to 22 seconds.
2014-05-24 10:44:40 +00:00
mav
54ed85cbfe MFC r265821:
Comment out some pointless device open/close around reading device IDs.

FreeBSD ZFS port unlike OpenSolaris does not use device IDs, and does not
implement respective devid_*() fuctions.  It is pointless to open devices
just to close them back immediately.
2014-05-24 10:41:37 +00:00
delphij
7cb0f49ed2 MFC r264835 (MFV r264829):
3897 zfs filesystem and snapshot limits
2014-05-09 07:12:31 +00:00
delphij
0fce5e81cc MFC r264669: MFV r264666:
4374 dn_free_ranges should use range_tree_t

illumos/illumos-gate@bf16b11e8d
2014-05-09 06:56:26 +00:00
mav
44963562b0 MFC r264145:
Add property and sysctl to control how ZVOLs are exposed to OS.

New ZFS property volmode and sysctl vfs.zfs.vol.mode allow switching ZVOL
between three modes:
 geom -- existing fully functional behavior (default);
 dev -- exposing volumes only as raw disk device file in devfs;
 none -- not exposing volumes outside ZFS.

The "dev" mode is less functional (can't be partitioned, mounted, etc),
but it is faster, and in some scenarios with untrusted consumers safer.
It can be useful for NAS, VM block storages, etc.
The "none" mode may be convenient for backup servers, etc. that don't
need direct data access.

Due to the way ZVOL is integrated with main ZFS code, those property
and sysctl are checked only during pool import and volume creation.
2014-05-08 13:12:24 +00:00
smh
c84c4e2d80 MFC r264851
Eliminated optarg global being used outside of the function which called getopt

Sponsored by:	Multiplay
2014-05-08 08:17:12 +00:00
markj
607e8b47f9 MFC r262542:
Move some files that are identical on i386 and amd64 to an x86 subdirectory
rather than keeping duplicate copies.
2014-05-03 16:08:52 +00:00
pfg
7c82f25917 MFC r264040:
4248 dtrace(1M) should never create DOF with empty probes section
4249 Only probes from the first DTrace object file will be included

Illumos Revision:	4a20ab41aadcb81c53e72fc65886e964e9add59

Reference:
https://www.illumos.org/issues/4248
https://www.illumos.org/issues/4249

Obtained from:	Illumos
2014-05-02 20:12:31 +00:00
delphij
e852cd6938 MFC r264467:
Take into account when zpool history block grows exceeding 128KB in zpool(8)
and zdb(8) by growing the buffer on demand with a cap of 1GB (specified in
spa_history_create_obj()).

PR:		bin/186574
Submitted by:	Andrew Childs <lorne cons org nz> (with changes)
2014-04-28 06:11:03 +00:00
jmmv
797209d767 MFC r264741: Add placeholder Kyuafiles for various top-level hierarchies.
This is "make tinderbox" clean.
2014-04-28 04:20:14 +00:00
markj
ec059ac886 MFC r262596:
4478 dtrace_dof_maxsize is far too small

illumos/illumos-gate@d339a29bb4
2014-04-23 03:26:29 +00:00
delphij
d336d68dfb MFC r263889 (MFV r263887):
3993 zpool(1M) and zfs(1M) should support -p for "list" and "get"
4700 "zpool get" doesn't support -H or -o options
2014-04-11 01:27:33 +00:00
delphij
f4b8b0fe7c MFC r263459: MFV 263436-263438:
3947 zpool(1M) references nonexistent zfs-features(5)
4540 zpool(1M) man page doesn't describe "readonly" property
3948 zfs sync=default is not accepted
4611 zfs(1M) still mentions 'send -r' in synopsis
4415 zpool(1M) man page missing "import -m" description
4570 Document dedupditto pool property
4572 Dedup-related documentation additions for zpool and zdb.
1371 Add -D option description to zpool(1M) manpage
4571 Add documentation for -T and interval to "zpool list"
2014-04-11 01:23:46 +00:00
delphij
60e600a585 MFC r263385:
Remove unused option -r from zpool.

Submitted by:	Richard Yao <ryao gentoo org>
2014-04-03 01:00:51 +00:00
dim
ebd9ec5dee MFC r260880 (by kaiw, from projects/elftoolchain):
* Make die_mem_offset() be able to handle DW_AT_data_member_location
    attributes generated by Clang 3.4.
  * Document how different compilers generate DW_AT_data_member_location
    attributes differently.
  * Document the quirks about DW_FORM_data[48].

This is a slightly modified version, adapted to work with the old
libdwarf in stable/9 and stable/10.  It should fix DTrace on these
branches, when the kernel is compiled with clang 3.4.

Note that you have to build *and* install the CTF tools first, before
building the kernel.  Otherwise you can possibly still get error
messages similar to "failed to copy type of 'pr_uid': Type information
is in parent and unavailable", when attempting to run dtrace(1).

Submitted by:	kaiw
2014-03-29 17:18:23 +00:00
asomers
31717f7c2c MFC r262912
cddl/contrib/opensolaris/lib/libuutil/common/uu_avl.c
	Fix a memory leak in uu_avl_pool_create: pthread_mutex_init without
	a corresponding pthread_mutex_destroy.  It shows up, among other
	places, when doing "zfs list".
2014-03-28 15:09:35 +00:00
delphij
ec5d67aa46 MFC r260183: MFV r260154 + 260182:
4369 implement zfs bookmarks
4368 zfs send filesystems from readonly pools

Illumos/illumos-gate@78f1710053
2014-03-20 00:28:53 +00:00
delphij
11b0e21c75 MFC r259850: MFV r258384:
2583 Add -p (parsable) option to zfs list

illumos/illumos-gate@43d68d68c1
2014-03-20 00:24:35 +00:00
delphij
18a0059ca2 MFC r256999 (smh):
Added support for the 'zfs list -t snap' and 'zfs snap' aliases which are
available under Oracle Solaris 11.

This includes an update to the ZFS(8) man page to reflect all the
available alias (snap, umount, and recv).

Initial changes obtained from ZFS On Linux + fixes for man page and cmd
help:
10b75496bb
cf81b00a73

Obtained from:  https://github.com/zfsonlinux/zfs
2014-03-20 00:10:58 +00:00
delphij
ede49d4106 MFC r260150: MFV r259170:
4370 avoid transmitting holes during zfs send

4371 DMU code clean up

illumos/illumos-gate@43466aae47

NOTE: Make sure the boot code is updated if a zpool upgrade is
done on boot zpool.
2014-03-19 23:55:03 +00:00
delphij
874f949f14 MFC r260138: MFV r242733:
3306 zdb should be able to issue reads in parallel
3321 'zpool reopen' command should be documented in the man page
and help message

illumos/illumos-gate@31d7e8fa33

FreeBSD porting notes: the kernel part of this changeset depends
on Solaris buf(9S) interfaces and are not really applicable for
our use.  vdev_disk.c is patched as-is to reduce diverge from
upstream, but vdev_file.c is left intact.
2014-03-19 23:44:03 +00:00
delphij
5b9b9ced50 MFC r259813 + r259813: MFV r258374:
4171 clean up spa_feature_*() interfaces

4172 implement extensible_dataset feature for use by other zpool
features

illumos/illumos-gate@2acef22db7
2014-03-19 23:36:12 +00:00
delphij
dda91b34ac MFC r262577: MFV r262570:
4626 libzfs memleak in zpool_in_use()
2014-03-14 01:09:42 +00:00
eadler
1f967fac12 MFC r261774 by feld:
Add caveat to zpool manpage indicating that we do not automatically activate
	hot spares. This should be MFC'd to all STABLE branches.

	Upon the availability of zfsd, the zpool manpage on relevant branches should
	be updated to remove this caveat and document hot spare's reliance on zfsd.

Requested by:	feld
2014-02-24 17:01:27 +00:00
avg
444af87821 MFC r260811: zdb -R: do not treat numeric parameters to a flag as more flags 2014-02-17 17:48:38 +00:00
avg
e67db98e94 MFC r260703: zinject must use ioctl(2) compatibility wrapper 2014-02-17 17:46:15 +00:00
avg
51714698c3 MFC r258717: MFV r258371,r258372: 4101 metaslab_debug should allow for
fine-grained control
2014-02-17 17:11:38 +00:00
avg
6f834c6747 MFC r261122: dtrace: remove unexplained 16MB limitation from dt_alloc/dt_zalloc 2014-02-17 15:35:11 +00:00
markj
9a6dcea530 MFC r260051:
When clearing relocations to __dtrace* symbols, handle both SHT_REL and
SHT_RELA sections properly instead of assuming that the relocation section
is of type SHT_REL.
2014-02-17 14:51:02 +00:00
avg
26096ba436 MFC r258632,258704: MFV r255255: 4045 zfs write throttle & i/o scheduler
performance work

Sponsored by:	HybridCluster [merge]
2014-01-16 15:57:39 +00:00
avg
a4ee4e9fb0 MFC r258630: 734 taskq_dispatch_prealloc() desired 2014-01-16 14:34:53 +00:00
jhibbits
04f97f6231 MFC r258362
Use 'int' to store the return value of getopt(), rather than char.

On some architectures (powerpc), char is unsigned by default, which means
comparisons against -1 always fail, so the programs get stuck in an
infinite loop.
2014-01-15 05:30:05 +00:00
jhibbits
b1391adbae MFC r256543,r259245,r259421,r259668,r259674
r256543:

Add fasttrap for PowerPC.  This is the last piece of the DTrace/ppc puzzle.
It's incomplete, it doesn't contain full instruction emulation, but it should be
sufficient for most cases.

r259245,r259421: (FBT)

FBT now does work fully on PowerPC.

Save r3 before using it for the trap check, else we end up saving the new r3,
containing the trap instruction encoding (0x7c810808), and restoring it back
with the frame on return.  This caused it to panic on my ppc32 machine.

r259668,r259674:
Fix a typo in the FBT code.
2014-01-15 05:19:37 +00:00
delphij
139f0a076d MFC r259811:
MFV r258373:

4168 ztest assertion failure in dbuf_undirty

4169 verbatim import causes zdb to segfa
4170 zhack leaves pool in ACTIVE state

illumos/illumos-gate@7fdd916c47
2014-01-14 01:28:08 +00:00
mav
cebac06b67 MFC r259168:
Don't even try to read vdev labels from devices smaller then SPA_MINDEVSIZE
(64MB).  Even if we would find one somehow, ZFS kernel code rejects such
devices.  It is funny to look on attempts to read 4 256K vdev labels from
1.44MB floppy, though it is not very practical and quite slow.
2014-01-05 22:14:12 +00:00
delphij
907f6b0d31 MFC r259131:
Don't panic when we get ZPOOL_STATUS_NON_NATIVE_ASHIFT
while listing importable pools.
2013-12-23 20:16:54 +00:00
markj
0b5457bf66 Convert the dtrace(1) man page to mdoc and fix up some aspects of it that
don't make sense on FreeBSD. In particular,

- remove the ATTRIBUTES section,
- remove references to the Solaris Dynamic Tracing Guide, except in the
  SEE ALSO section,
- update the description of the -A option for FreeBSD's implementation,
- remove references to Solaris-specific programs and configuration files,
  and replace them with FreeBSD equivalents where possible.

The content has not changed aside from this.

Approved by:	re (joel)
MFC after:	1 week
2013-10-10 03:50:23 +00:00
rmh
774f6b00bf Fix implicit declaration of jail_getid()
Approved by:	re
2013-10-07 14:22:19 +00:00
markj
8ff2d52009 Add a separate translator for headers passed to the TCP probes in the
input path. These probes get some of the fields in host order, whereas the
output probes get them in network order, so a single translator isn't
enough. This workaround ensures that the problem is essentially invisble
to users: none of the probe arguments or their fields have changed.

Approved by:	re (hrs)
2013-10-02 17:14:12 +00:00
delphij
d04a7f0144 MFV r254750:
Add support of Illumos dumps on zvol over RAID-Z.

Note that this only adds the features.  FreeBSD would
still need more work to support dumping on zvols.

Illumos ZFS issues:
  2932 support crash dumps to raidz, etc. pools

MFC after:	1 month
Approved by:	re (ZFS blanket)
2013-09-21 00:17:26 +00:00
markj
37ef3ce41e Use the address of the inpcb rather than the tcpcb to identify TCP
connections. This keeps the tcp provider consistent with the other network
providers.

Approved by:	re (delphij)
2013-09-15 21:38:46 +00:00