Commit Graph

839 Commits

Author SHA1 Message Date
Pedro F. Giffuni
87e109c3e0 Merge from OpenSolaris (22-Apr-2008):
6823388 DTrace ioctl handlers must validate all structure members

MFC after:	1 week
2014-07-03 19:07:37 +00:00
Pedro F. Giffuni
e099b3a948 Merge from OpenSolaris (20-Apr-2008):
6822482 DOF validation needs to handle loadable sections flagged as unloadable

MFC after:	1 week
2014-07-03 17:36:59 +00:00
Alexander Motin
5a178afd41 Fix bug in sync control in new "dev" mode of ZVOL (r265678).
Don't check ZVOL_WCE flag, used in Solaris to control device "write cache".
It is not applicable on FreeBSD and by default set to "disable".

MFC after:	3 days
2014-07-02 21:25:32 +00:00
Pedro F. Giffuni
0b8f286e83 Merge from OpenSolaris (15-Sep-2008):
6735480 race between probe enabling and provider registration

MFC after:	1 week
2014-07-01 23:37:24 +00:00
Xin LI
30324e945a MFV r268122:
4929 want prevsnap property

illumos/illumos-gate@b461c7460e

MFC after:	2 weeks
2014-07-01 22:42:53 +00:00
Xin LI
9cc8a15b2e MFV r268121:
4924 LZ4 Compression for metadata

illumos/illumos-gate@b8289d24d8

MFC after:	2 weeks
2014-07-01 22:31:09 +00:00
Pedro F. Giffuni
f384ec379c Small merges from OpenSolaris:
These have no effect on FreeBSD, in fact they are ifdef'ed,
but make easier future merges:

6699767 panic in spec_open()

6718877 crgetzoneid() use can cause problems when forking processes with
USDT providers in a non global zone

MFC after:	3 days
2014-07-01 22:16:44 +00:00
Xin LI
aa882b9048 MFV r268119:
4914 zfs on-disk bookmark structure should be named *_phys_t

illumos/illumos-gate@7802d7bf98

MFC after:	2 weeks
2014-07-01 21:51:30 +00:00
Xin LI
55f6421982 - Fix handling of "new" style of ioctl in compatiblity mode [1];
- Reorganize code and reduce diff from upstream;
 - Improve forward compatibility shims for previous kernel;

Reported by:	sbruno [1]
X-MFC-With:	r268075
2014-07-01 20:57:39 +00:00
Pedro F. Giffuni
c6d712caf3 Revert r268007, and re-adapt MFV r260708:
4427 pid provider rejects probes with valid UTF-8 names

Use of u8_textprep.c required -Wno-cast-qual for powerpc.

MFC after:	2 weeks
2014-07-01 15:36:05 +00:00
Xin LI
be78a8db97 MFV r267570:
4756 metaslab_group_preload() could deadlock

illumos/illumos-gate@30beaff42d

MFC after:	2 weeks
2014-07-01 08:36:56 +00:00
Xin LI
3a0f8ff95e MFV r267569:
4897 Space accounting mismatch in L2ARC/zpool

illumos/illumos-dist@3038a2b421

MFC after:	2 weeks
2014-07-01 08:28:49 +00:00
Xin LI
93b8d53c09 MFV r267567:
4881 zfs send performance degradation when embedded block pointers are
     encountered

illumos/illumos-gate@06315b795c

MFC after:	2 weeks
2014-07-01 07:56:07 +00:00
Xin LI
71eaf0fda7 MFV r267566:
4390 i/o errors when deleting filesystem/zvol can lead to space map corruption

MFC after:	2 weeks
2014-07-01 07:29:42 +00:00
Xin LI
29441ba3fa MFV r267565:
4757 ZFS embedded-data block pointers ("zero block compression")
4913 zfs release should not be subject to space checks

MFC after:	2 weeks
2014-07-01 06:43:15 +00:00
Pedro F. Giffuni
0135aadfc3 Reduce some warnings in the Solaris unicode support.
Clean some warnings from parenthesis and minor style issues.

MFC after:	3 days
2014-06-29 02:28:05 +00:00
Pedro F. Giffuni
f34dd28f7d Revert r267869:
MFV	r260708
4427 pid provider rejects probes with valid UTF-8 names

Use of u8_textprep.c broke the build on powerpc.

Reported by:	bz, rpaulo and tinderbox.
Pointyhat:	me
2014-06-28 19:59:12 +00:00
Hans Petter Selasky
af3b2549c4 Pull in r267961 and r267973 again. Fix for issues reported will follow. 2014-06-28 03:56:17 +00:00
Glen Barber
37a107a407 Revert r267961, r267973:
These changes prevent sysctl(8) from returning proper output,
such as:

 1) no output from sysctl(8)
 2) erroneously returning ENOMEM with tools like truss(1)
    or uname(1)
 truss: can not get etype: Cannot allocate memory
2014-06-27 22:05:21 +00:00
Hans Petter Selasky
3da1cf1e88 Extend the meaning of the CTLFLAG_TUN flag to automatically check if
there is an environment variable which shall initialize the SYSCTL
during early boot. This works for all SYSCTL types both statically and
dynamically created ones, except for the SYSCTL NODE type and SYSCTLs
which belong to VNETs. A new flag, CTLFLAG_NOFETCH, has been added to
be used in the case a tunable sysctl has a custom initialisation
function allowing the sysctl to still be marked as a tunable. The
kernel SYSCTL API is mostly the same, with a few exceptions for some
special operations like iterating childrens of a static/extern SYSCTL
node. This operation should probably be made into a factored out
common macro, hence some device drivers use this. The reason for
changing the SYSCTL API was the need for a SYSCTL parent OID pointer
and not only the SYSCTL parent OID list pointer in order to quickly
generate the sysctl path. The motivation behind this patch is to avoid
parameter loading cludges inside the OFED driver subsystem. Instead of
adding special code to the OFED driver subsystem to post-load tunables
into dynamically created sysctls, we generalize this in the kernel.

Other changes:
- Corrected a possibly incorrect sysctl name from "hw.cbb.intr_mask"
to "hw.pcic.intr_mask".
- Removed redundant TUNABLE statements throughout the kernel.
- Some minor code rewrites in connection to removing not needed
TUNABLE statements.
- Added a missing SYSCTL_DECL().
- Wrapped two very long lines.
- Avoid malloc()/free() inside sysctl string handling, in case it is
called to initialize a sysctl from a tunable, hence malloc()/free() is
not ready when sysctls from the sysctl dataset are registered.
- Bumped FreeBSD version to indicate SYSCTL API change.

MFC after:	2 weeks
Sponsored by:	Mellanox Technologies
2014-06-27 16:33:43 +00:00
Rui Paulo
a43f0be9fe MFV illumos
4471 DTrace count() with histogram
4472 DTrace full width distribution histograms
4473 DTrace frequency trails

MFC after:	2 weeks
2014-06-26 23:24:59 +00:00
Rui Paulo
8e648814b0 MFV illumos
4474 DTrace Userland CTF Support
4475 DTrace userland Keyword
4476 DTrace tests should be better citizens
4479 pid provider types
4480 dof emulation is missing checks

MFC after:	2 weeks
2014-06-26 23:21:11 +00:00
Rui Paulo
b1f9167f94 MFV illumos
4477 DTrace should speak JSON

MFC after:	2 weeks
2014-06-26 21:45:49 +00:00
Rui Paulo
0c2b601953 MFV illumos r266986:
2915 DTrace in a zone should see "cpu", "curpsinfo", et al
2916 DTrace in a zone should be able to access fds[]
2917 DTrace in a zone should have limited provider access

MFC after:	2 weeks
2014-06-26 19:38:16 +00:00
Rui Paulo
dd9b2abed8 Revert r267898. 2014-06-26 17:34:42 +00:00
Rui Paulo
d8e37c5f72 Bring the following change from the illumos-joyent repository:
commit 78e24ab6803bbe11ba37642624e1498ede5b239d
Author: Bryan Cantrill <bryan@joyent.com>
Date:   Thu Oct 31 01:20:54 2013

    OS-1688 DTrace count() with histogram
    OS-2360 DTrace full width distribution histograms
    OS-2361 DTrace frequency trails

MFC after:	2 weeks
2014-06-26 07:06:43 +00:00
Pedro F. Giffuni
af8bd6e468 MFV r260708
4427 pid provider rejects probes with valid UTF-8 names

This make use of Solaris' u8_validate() which we happen to
use since r185029 for ZFS.

Illumos Revision:	1444d846b126463eb1059a572ff114d51f7562e5

Reference:
https://www.illumos.org/issues/4427

Obtained from:	Illumos
MFC after:	2 weeks
2014-06-25 14:23:30 +00:00
Davide Italiano
a99098e2ba Continue the crusade towards a dev_clone()-free kernel, removing its
usage from dtrace. The dtrace code already uses cdevpriv(9) since FreeBSD
8, so this change should be quite harmless.

Reviewed by:	markj
Approved by:	markj
MFC after:	never
2014-06-25 03:54:02 +00:00
Mark Johnston
efa1aff675 Fix some bugs when fetching probe arguments in i386. Firstly ensure that
the 4 byte-aligned dtrace_invop_callsite can be found and that it
immediately follows the call to dtrace_invop(). Secondly, fix some pointer
arithmetic to account for differences between struct i386_frame and illumos'
struct frame. Finally, ensure that dtrace_getarg() isn't inlined. It works
by following a fixed number of frame pointers to the probe site, so inlining
breaks it.

MFC after:	3 weeks
2014-06-23 02:00:14 +00:00
Steven Hartland
74ddec2b18 Removed stale comment about multi-vdev root pool config not working
MFC after:	1 week
2014-06-09 13:04:58 +00:00
Bryan Drewery
f3a7518361 - Naively fix build by partially reverting r267029 to still use
gethrtime() when building libzpool.

X-MFC-With:	267029
2014-06-04 05:04:15 +00:00
Alexander Motin
4220ebcf71 Replace gethrtime() with cpu_ticks(), as source of random for the taskqueue
selection.  gethrtime() in our port updated with HZ rate, so unusable for
this specific purpose, completely draining benefit of multiple taskqueues.

MFC after:	2 weeks
2014-06-03 21:06:03 +00:00
Xin LI
f4c7dd6dd0 MFV 266913+266914:
3897 zfs filesystem and snapshot limits (fix leak)
4901 zfs filesystem/snapshot limit leaks

MFC after:	3 days
2014-05-31 01:00:22 +00:00
Xin LI
2bdf7f79bc MFV r266766:
Add a new zfs property, "redundant_metadata" which can have values "all" or
"most".  The default will be "all", which is the current behavior.  When set
to all, ZFS stores an extra copy of all metadata.  If a single on-disk block
is corrupt, at worst a single block of user data (which is recordsize bytes
long) can be lost.

Setting to "most" will cause us to only store 1 copy of level-1 indirect
blocks of user data files.  This can improve performance of random writes,
because less metadata has to be written.  In practice,  at worst about
100 blocks (of recordsize bytes each) of user data can be lost if a single
on-disk block is corrupt.

The exact behavior of which metadata blocks are stored redundantly may change
in future releases.

Illumos issue: 3835 zfs need not store 2 copies of all metadata

MFC after:	2 weeks
2014-05-27 19:46:11 +00:00
Allan Jude
ecd9567c1a Improve sysctl descriptions for new ZFS sysctls:
vfs.zfs.dirty_data_max
vfs.zfs.dirty_data_max_max
vfs.zfs.dirty_data_sync

Reviewed by:	smh
Approved by:	wblock (mentor)
2014-05-22 05:30:38 +00:00
Steven Hartland
df23182a62 Added sysctls / tunables for ZFS dirty data tuning
Added the following new sysctls / tunables:
* vfs.zfs.dirty_data_max
* vfs.zfs.dirty_data_max_max
* vfs.zfs.dirty_data_max_percent
* vfs.zfs.dirty_data_sync
* vfs.zfs.delay_min_dirty_percent
* vfs.zfs.delay_scale

PR:		kern/189865
MFC after:	2 weeks
2014-05-21 13:36:04 +00:00
Xin LI
b8cdcb8ad8 Import George Wilson's change for Illumos #4730:
4730 metaslab group taskq should be destroyed in metaslab_group_destroy()
	Reviewed by: Alex Reece <alex.reece@delphix.com>
	Reviewed by: Matthew Ahrens <mahrens@delphix.com>
	Reviewed by: Sebastien Roy <sebastien.roy@delphix.com>

	Original author: George Wilson

MFC after:	3 days
2014-05-06 19:03:04 +00:00
Steven Hartland
4f64781818 Use a zio flag to prevent recursion of vdev_queue_io_done which can
cause stack overflow for IO's which return ZIO_PIPELINE_CONTINUE
from the zio_vdev_io_start stage and hence don't suspend and complete
in a different thread.

This prevents double fault panic on slow machines running ZFS on
GELI volumes which return EOPNOTSUPP directly to BIO_DELETE requests.

MFC after:	1 month
X-MFC-With:	r265152
2014-05-04 14:05:14 +00:00
Steven Hartland
573621a6d6 Don't treat TRIM requests returning ENOTSUP as an unexpected error.
MFC after:	1 month
X-MFC-With:	r265152
2014-05-03 02:30:01 +00:00
Steven Hartland
10138166cf Removed pointless / duplicated call to trim_map_first.
MFC after:	1 month
X-MFC-With:	r265152
2014-05-02 09:31:21 +00:00
Steven Hartland
82ce008538 Reintroduce priority for the TRIM ZIOs instead of using the "NOW" priority
The changes how TRIM requests are generated to use ZIO_TYPE_FREE + a priority
instead of ZIO_TYPE_IOCTL, until processed by vdev_geom; only then is it
translated the required geom values. This reduces the amount of changes
required for FREE requests to be supported by the new IO scheduler. This
also eliminates the need for a specific DKIOCTRIM.

Also fixed FREE vdev child IO's from running ZIO_STAGE_VDEV_IO_DONE as part
of their schedule.

As the new IO scheduler can result in a request to execute one type of IO to
actually run a different type of IO it requires that zio_trim requests are
processed without holding the trim map lock (tm->tm_lock), as the free request
execute call may result in write request running hence triggering a
trim_map_write_start call, which takes the trim map lock and hence would result
in recused on no-recursive sx lock.

This is based off avg's original work, so credit to him.

MFC after:	1 month
2014-04-30 17:46:29 +00:00
Steven Hartland
101dfa0ed4 Fix ZIO reordering done by vdev_queue_io causing panics when zio_vdev_io_start
returns ZIO_PIPELINE_CONTINUE from vdev_op_io_start to zio_execute resulting
in the wrong ZIO continuing its pipeline.

This is a serious issue which could cause data loss / corruption but appears
to be limited to error handling such as when vdev_readable(vd) returns false.

MFC after:	2 days
2014-04-28 09:00:00 +00:00
Steven Hartland
c2b2c5fc76 Eliminate duplicate checks in vdev_geom_io_intr error handling
MFC after:	1 month
2014-04-24 15:36:00 +00:00
Steven Hartland
5b245b8ae0 Add the ability to set a minimum ashift size for ZFS pool creation or root level
vdev addition.

Change max_auto_ashift sysctl to error when an invalid value is requested instead
of silently limiting it.
2014-04-24 01:06:03 +00:00
Xin LI
754180f4ae MFV r264830:
4745 fix AVL code misspellings

MFC after:	2 weeks
2014-04-23 20:32:39 +00:00
Xin LI
f8587167e4 MFV r264829:
3897 zfs filesystem and snapshot limits

MFC after:	2 weeks
2014-04-23 20:29:46 +00:00
Xin LI
18ab4bd8d9 MFV r264668:
4754 io issued to near-full luns even after setting noalloc threshold
4755 mg_alloc_failures is no longer needed

illumos/illumos@b6240e830b

MFC after:	2 weeks
2014-04-18 22:04:58 +00:00
Xin LI
d301d390a7 MFV r264667:
4752 fan out read zio taskqs

illumos/illumos-gate@1b497ab83e
2014-04-18 21:35:23 +00:00
Xin LI
613074ec08 MFV r264666:
4374 dn_free_ranges should use range_tree_t

illumos/illumos-gate@bf16b11e8d

MFC after:	2 weeks
2014-04-18 21:15:12 +00:00
Mark Johnston
0626f3e435 DTrace's pid provider works by inserting breakpoint instructions at probe
sites and installing a hook at the kernel's trap handler. The fasttrap code
will emulate the overwritten instruction in some common cases, but otherwise
copies it out into some scratch space in the traced process' address space
and ensures that it's executed after returning from the trap.

In Solaris and illumos, this (per-thread) scratch space comes from some
reserved space in TLS, accessible via the fs segment register. This
approach is somewhat unappealing on FreeBSD since it would require some
modifications to rtld and jemalloc (for static TLS) to ensure that TLS is
executable, and would thus introduce dependencies on their implementation
details. I think it would also be impossible to safely trace static binaries
compiled without these modifications.

This change implements the functionality in a different way, by having
fasttrap map pages into the target process' address space on demand. Each
page is divided into 64-byte chunks for use by individual threads, and
fasttrap's process descriptor struct has been extended to keep track of
any scratch space allocated for the corresponding process.

With this change it's possible to trace all libc functions in a program,
e.g. with

  pid$target:libc.so.*::entry {@[probefunc] = count();}

Previously this would generally cause the victim process to crash, as
tracing memcpy on amd64 requires the functionality described above.

Tested by:	Prashanth Kumar <pra_udupi@yahoo.co.in> (earlier version)
MFC after:	6 weeks
2014-04-14 00:22:42 +00:00