Commit Graph

1783 Commits

Author SHA1 Message Date
Alexander Motin
83d0c3846d Allocate buffers smaller then ABD chunk size as linear.
This allows to reduce memory waste by letting UMA to put multiple small
buffers into one memory page slab.  The page sharing means that UMA
may not be able to free memory page when some of buffers are freed, but
alternatively memory used by that buffer would just be wasted from the
beginning.

This change follows alike change in ZoL, but unlike Linux (according to
my understanding of it from comments) FreeBSD never shares slabs bigger
then one memory page, so this should be even less invasive then there.

MFC after:	2 weeks
Sponsored by:	iXsystems, Inc.
2019-05-22 18:43:48 +00:00
Allan Jude
57e9b361ba Fix typo in r348068 2019-05-21 21:39:03 +00:00
Allan Jude
4e7d8292ec ZFS: Make deadman tunables no longer read-only
This allows the user to enable, disable, and adjust the I/O deadman at
runtime. This can be especially useful when a pool is backed by remote
storage (such as iscsi, ggated, etc).

PR:		221906
Submitted by:	Fabian Keil <fk@fabiankeil.de>
Obtained from:	ElectroBSD
MFC after:	1 week
Sponsored by:	Klara Systems
Event:		Waterloo Hackathon 2019
2019-05-21 21:26:18 +00:00
Alexander Motin
7763842174 Add mutex_destroy() missed in r334844.
MFC after:	1 week
Sponsored by:	iXsystems, Inc.
2019-04-26 19:02:21 +00:00
Alexander Motin
32d8034f77 Fix minor mismerges.
No functional change.

MFC after:	1 week
2019-04-26 18:25:59 +00:00
Alexander Motin
48ecceba1e Change the way FreeBSD GID inheritance is hacked.
I believe previous ifdef caused NULL dereference in later zfs_log_create()
on attempt to create file inside directory belonging to ephemeral group
created on illumos, trying to write to log information about GID domain
of the newly created file, inheriting the ephemeral GID.

This patch reuses original illumos SGID code with exception that due to
lack of ID mapping code on FreeBSD ephemeral GID will turn into GID_NOBODY
by another ifdef inside zfs_fuid_map_id().

MFC after:	1 month
Sponsored by:	iXsystems, Inc.
2019-04-19 15:44:45 +00:00
Conrad Meyer
a8a16c7128 Replace read_random(9) with more appropriate arc4rand(9) KPIs
Reviewed by:	ae, delphij
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D19760
2019-04-04 01:02:50 +00:00
Pawel Jakub Dawidek
af4f9e5f00 If the autoexpand pool property is turned on and vdev is healthy try to
expand the pool automatically when we detect underlying GEOM provider
size change.

Obtained from:	Fudo Security
Tested in:	AWS
2019-03-30 07:29:20 +00:00
Andriy Gapon
76a63ee510 Revert r345410, VOP_FSYNC change in ZFS vdev_file
I overlooked the fact that that VOP_FSYNC() call is not a FreeBSD VFS
call, but a macro that provides an illumos-compatible wrapper for the
FreeBSD operation.

PR:		236475
Reported by:	lwhsu
Pointyhat to:	avg
2019-03-22 17:44:47 +00:00
Andriy Gapon
c1581f4f4d ZFS vdev_file: use correct value for waitfor parameter of VOP_FSYNC
PR:		236475
Reported by:	asomers
MFC after:	2 weeks
2019-03-22 09:11:45 +00:00
Alexander Motin
6bb46107d8 MFV r336930: 9284 arc_reclaim_thread has 2 jobs
`arc_reclaim_thread()` calls `arc_adjust()` after calling
`arc_kmem_reap_now()`; `arc_adjust()` signals `arc_get_data_buf()` to
indicate that we may no longer be `arc_is_overflowing()`.

The problem is, `arc_kmem_reap_now()` can take several seconds to
complete, has no impact on `arc_is_overflowing()`, but due to how the
code is structured, can impact how long the ARC will remain in the
`arc_is_overflowing()` state.

The fix is to use seperate threads to:

1. keep `arc_size` under `arc_c`, by calling `arc_adjust()`, which
    improves `arc_is_overflowing()`

2. keep enough free memory in the system, by calling
 `arc_kmem_reap_now()` plus `arc_shrink()`, which improves
 `arc_available_memory()`.

illumos/illumos-gate@de753e34f9

Reviewed by: Matt Ahrens <mahrens@delphix.com>
Reviewed by: Serapheim Dimitropoulos <serapheim@delphix.com>
Reviewed by: Pavel Zakharov <pavel.zakharov@delphix.com>
Reviewed by: Dan Kimmel <dan.kimmel@delphix.com>
Reviewed by: Paul Dagnelie <pcd@delphix.com>
Reviewed by: Dan McDonald <danmcd@joyent.com>
Reviewed by: Tim Kordas <tim.kordas@joyent.com>
Approved by: Garrett D'Amore <garrett@damore.org>
Author: Brad Lewis <brad.lewis@delphix.com>
2019-03-15 18:59:04 +00:00
Simon J. Gerraty
f5fdf82d82 Add _PC_ACL_* to vop_stdpathconf
This avoid EINVAL from tmpfs etc.

Reviewed by:	kib
Differential Revision:	https://reviews.freebsd.org/D19512
2019-03-11 20:40:56 +00:00
Alexander Motin
aa8676f25d Revert minor part of r344934.
I tried to save some CPU time on hopeless aggregation attempts, but it seems
the condition I added is overly strict, blocking also aggregation of optional
I/Os in cases which previously were possible.  Revert just to be safe.

MFC after:	1 month
2019-03-11 17:39:09 +00:00
Alexander Motin
5ca679e3c4 MFV/ZoL: Disable LBA weighting on files and SSDs
The LBA weighting makes sense on rotational media where the outer tracks
have twice the bandwidth of the inner tracks. However, it is detrimental
on nonrotational media such as solid state disks, where the only effect
is to ensure that metaslabs enter the best-fit allocation behavior
sooner, which is detrimental to performance. It also makes no sense on
files where the underlying filesystem can arrange things however it
wants.

Author: Richard Yao <ryao@gentoo.org>
Signed-off-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3712
zfsonlinux/zfs@fb40095f5f

To reduce code divergence this merge replaces equivalent but different
FreeBSD code detecting non-rotating medium vdevs.

MFC after:	1 month
2019-03-08 21:13:45 +00:00
Alexander Motin
673544c3dd Add separate aggregation limit for non-rotating media.
Before sequential scrub patches ZFS never aggregated I/Os above 128KB.
Sequential scrub bumped that to 1MB, which motivation I understand for
spinning disks, since it should reduce number of head seeks.  But for
SSDs it makes much less sense to me, especially on FreeBSD, where due
to MAXPHYS limitation device will likely still see bunch of 128KB I/Os
instead of one large.  Having more strict aggregation limit allows to
avoid allocation of large memory buffer and memcpy to/from it, that is
a serious problem when bandwidth reaches few GB/s.

MFC after:	1 month
Sponsored by:	iXsystems, Inc.
2019-03-08 19:38:52 +00:00
Alexander Motin
3a3ba532e7 MFV/ZoL: Fix zfs_vdev_aggregation_limit bounds checking
Update the bounds checking for zfs_vdev_aggregation_limit so that
it has a floor of zero and a maximum value of the supported block
size for the pool.

Additionally add an early return when zfs_vdev_aggregation_limit
equals zero to disable aggregation.  For very fast solid state or
memory devices it may be more expensive to perform the aggregation
than to issue the IO immediately.

Author: Brian Behlendorf <behlendorf1@llnl.gov>
zfsonlinux/zfs@a58df6f536

MFV/ZoL: Cap maximum aggregate IO size

Commit 8542ef8 allowed optional IOs to be aggregated beyond
the specified aggregation limit.  Since the aggregation limit
was also used to enforce the maximum block size, setting
`zfs_vdev_aggregation_limit=16777216` could result in an
attempt to allocate an ABD larger than 16M.

Author: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed-by: George Melikov <mail@gmelikov.ru>
Reviewed-by: Giuseppe Di Natale <dinatale2@llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #6259
Closes #6270
zfsonlinux/zfs@2d678f779a
2019-03-08 18:49:27 +00:00
Alexander Motin
ede8782611 Improve entropy for ZFS taskqueue selection.
I just found that at least on Skylake CPUs cpu_ticks() never returns odd
values, only even, and possibly has even bigger step (176/2?), that makes
its lower bits very bad entropy source, leaving half of taskqueues unused.
Switch to sbinuptime(), closer to upstreams, mitigates the problem by the
rate conversion working as kind of hash function.  In case that is somehow
not enough (timer rate is too low or too divisible) mix in curcpu.

MFC after:	1 week
2019-03-07 22:56:39 +00:00
Alexander Motin
551b7d3a29 Add respective tunables to few ZFS sysctls.
MFC after:	1 week
2019-03-07 01:24:08 +00:00
Pawel Jakub Dawidek
b8da50d526 Improve readability of the code by making it explicit where the 'c' variable
starts. It is also more consistent with similar code in this file.
2019-03-01 05:54:13 +00:00
Mark Johnston
8e7127fd91 Fix fasttrap_sig{trap,segv}().
- Don't leak the ksiginfo structure.
- Hold the proc lock when sending a signal in fasttrap_sigsegv().

MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2019-02-26 18:20:41 +00:00
Mark Johnston
5563c675b3 Revert r344587.
The fasttrap_isa.h header is needed by libdtrace, not just the kernel.
2019-02-26 17:33:56 +00:00
Mark Johnston
df59ed0787 Remove illumos-specific code from the x86 fasttrap_isa.c.
The file has not been touched upstream in over a decade, and the nature
of the code means that a lot of FreeBSD-specific bits are required.  Remove
the dead code to improve readability.  No functional change intended.

Discussed with:	cem
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2019-02-26 16:34:43 +00:00
Mark Johnston
6829dae12b Remove stub fasttrap implementations.
No platforms except i386, amd64 and powerpc implement fasttrap; the
fasttrap files for other arches do not contain any code and bloat
the output from cscope, so just remove them.

MFC after:	1 week
2019-02-26 16:31:47 +00:00
Mark Johnston
f23e684bbf Commit a missing piece of r344452.
MFC with:	r344452
2019-02-21 22:56:54 +00:00
Mark Johnston
4f1b715c84 Fix a tracepoint lookup race in fasttrap_pid_probe().
fasttrap hooks the userspace breakpoint handler; the hook looks up the
breakpoint address in a hash table of tracepoints.  It is possible for
the tracepoint to be removed by a different thread in between the
breakpoint trap and the hash table lookup, in which case SIGTRAP gets
delivered to the target process.  Fix the problem by adding a
per-process generation counter that gets incremented when a tracepoint
belonging to that process is removed.  Then, when a lookup fails, the
trapping instruction is restarted if the thread's counter doesn't match
that of the process.

Reviewed by:	cem
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D19273
2019-02-21 22:54:17 +00:00
Pawel Jakub Dawidek
2691ae3230 Simplify the code. No functional changes.
Reviewed by:	rpokala
2019-02-20 00:25:45 +00:00
Pawel Jakub Dawidek
91853b8546 Simplify the code. 2019-02-19 23:53:33 +00:00
Pawel Jakub Dawidek
01e21ead90 Correct typo in the comment. 2019-02-19 23:44:00 +00:00
Pawel Jakub Dawidek
99ab63b69d Change assertion to log the incorrect io_type we've got. 2019-02-19 23:43:15 +00:00
Pawel Jakub Dawidek
36d43b5dfe Grabage-collect no longer used variable. 2019-02-19 23:41:23 +00:00
Pawel Jakub Dawidek
11c8759337 The way ZFS searches for its vdevs is the following: first it looks for
a vdev that has the same name as the one stored in metadata and that has
all VDEV labels in place. If it cannot find a GEOM provider with the given
name and all VDEV labels it will scan all GEOM providers for the best match
(the most VDEV labels available), but here the name is ignored.

In case the ZFS pool is created, eg. using GPT partition label:

	# zpool create tank /dev/gpt/tank

everything works, and on every import ZFS will pick /dev/gpt/tank and
not /dev/da0p4.

The problem occurs when da0p4 is extended and ZFS is unable to find all
VDEV labels in /dev/gpt/tank anymore (the VDEV labels stored at the end
of the partition are now somewhere else). In this case it will scan all
GEOM providers and will pick the first one with the best match, ie. da0p4.

Fix this problem by checking the VDEV/provider name even if we get the same
match. If the name is the same as the one we have in pool's metadata, prefer
this GEOM provider.

Reported by:	oshogbo, Michal Mroz <m.mroz@fudosecurity.com>
Tested by:	Michal Mroz <m.mroz@fudosecurity.com>
Obtained from:	Fudo Security
2019-02-19 23:35:55 +00:00
Pawel Jakub Dawidek
d793cf7019 In the vdev_geom_open_by_path() function we assume that vdev path starts
with "/dev/". Make sure this is the case.
2019-02-19 23:22:39 +00:00
Alexander Motin
ed0a3e8637 s/Maximal/Maximum/ in sysctl description.
Submitted by:	smh
MFC after:	1 week
2019-02-04 20:09:22 +00:00
Alexander Motin
ef08154150 Add missed tunables/sysctls for some new vdev variables.
While there, make few existing sysctls writeable, since there is no reason
not to.

MFC after:	1 week
2019-02-04 16:13:41 +00:00
Alexander Motin
54cde30f92 Remove BIO_ORDERED flag from BIO_FLUSH sent by ZFS.
In all cases where ZFS sends BIO_FLUSH, it first waits for all related
writes to complete, so its BIO_FLUSH does not care about strict ordering.
Removal of one makes life much easier at least for NVMe driver, which
hardware has no concept of request ordering, relying completely on software.

MFC after:	2 weeks
Sponsored by:	iXsystems, Inc.
2019-01-30 17:39:44 +00:00
Mariusz Zaborski
db009dddfd zfs: allow to change cache flush sysctl
There is no reason for this variable to be tunable.
This variable is used as a barrier in few places.

Discussed with:	pjd
MFC after:	2 weeks
Sponsored by:	Fudo Security
2019-01-26 13:53:00 +00:00
Sean Eric Fagan
82e20c0a72 Change ZFS quotas to return EINVAL when not present (matches man page).
UFS will return EINVAL when quotas are not enabled on a filesystem; ZFS'
equivalent involves not having quotas (there is not way to enable or disable
quotas as such).  My initial implementation had it return ENOENT, but
quotactl(2) indicates EINVAL is more appropriate.

MFC after:	2 weeks
Approved by:	mav
Reviewed by:	markj
Reported by:	Emrion <kmachine@free.fr>
Sponsored by: iXsystems Inc
PR: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=234413
2019-01-11 02:53:46 +00:00
Andriy Gapon
4c325393f3 MFV r342532: 5882 Temporary pool names
Note that this commit brings only formatting changes that were done
during the final review of the illumos change, because FreeBSD got the
main changes before illumos.

illumos/illumos-gate@04e5635652
04e5635652

https://www.illumos.org/issues/5882
  This is an import of the temporary pool names functionality from ZoL:
  e2282ef57e
  26b42f3f9d
  2f3ec90061
  00d2a8c92f
  83e9986f6e
  023bbe6f01
  It is intended to assist the creation and management of virtual machines
  that have their rootfs on ZFS on hosts that also have their rootfs on
  ZFS. These situations cause SPA namespace collisions when the standard
  name rpool is used in both cases. The solution is either to give each
  guest pool a name unique to the host, which is not always desireable, or
  boot a VM environment containing an ISO image to install it, which is
  cumbersome.

MFC after:	1 week
Sponsored by:	Panzura
2018-12-26 11:03:14 +00:00
Andriy Gapon
f050611e7f MFV r342469: 9630 add lzc_rename and lzc_destroy to libzfs_core
illumos/illumos-gate@049ba636fa
049ba636fa

https://www.illumos.org/issues/9630
  Rename and destroy are very useful operations that deserve to be in
  libzfs_core.  And they are not hard to implement too.

MFC after:	2 weeks
Relnotes:	maybe
2018-12-26 10:37:41 +00:00
Mateusz Guzik
cc426dd319 Remove unused argument to priv_check_cred.
Patch mostly generated with cocinnelle:

@@
expression E1,E2;
@@

- priv_check_cred(E1,E2,0)
+ priv_check_cred(E1,E2)

Sponsored by:	The FreeBSD Foundation
2018-12-11 19:32:16 +00:00
Toomas Soome
7aaf685ba7 zfs: we can boot from dataset with large_dnode enabled
loader has been supporting large_dnode for some time, no need to block the
feature for boot dataset.

Reviewed by:	avg
MFC after:	1 month
Differential Revision:	https://reviews.freebsd.org/D18391
2018-12-03 19:35:21 +00:00
Mark Johnston
6d2e2df764 Ensure that directory entry padding bytes are zeroed.
Directory entries must be padded to maintain alignment; in many
filesystems the padding was not initialized, resulting in stack
memory being copied out to userspace.  With the ino64 work there
are also some explicit pad fields in struct dirent.  Add a subroutine
to clear these bytes and use it in the in-tree filesystems.  The
NFS client is omitted for now as it was fixed separately in r340787.

Reported by:	Thomas Barabosch, Fraunhofer FKIE
Reviewed by:	kib
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2018-11-23 22:24:59 +00:00
Alexander Motin
eecd0a1856 Revert r340096: 9952 Block size change during zfs receive drops spill block
It was reported, and I easily reproduced it, that this change triggers panic
when receiving replication stream with enabled embedded blocks, when short
file compressing into one embedded block changes its block size.  I am not
sure that the problem is in this particuler patch, not just triggered by it,
but since investigation and fix will take some time, I've decided to revert
this for now.

PR:		198457, 233277
2018-11-21 18:18:57 +00:00
Justin Hibbits
cfebc0faa7 DTrace/powerpc: Fix FBT return probes
The FBT fuction boundary prober was setting one return probe marker value,
but the dtrace handler was expecting another.  This causes a hang when
tracing return probes.
2018-11-21 16:47:11 +00:00
Konstantin Belousov
1c4ca77890 Add d_off support for multiple filesystems.
The d_off field has been added to the dirent structure recently.
Currently filesystems don't support this feature.  Support has been
added and tested for zfs, ufs, ext2fs, fdescfs, msdosfs and unionfs.
A stub implementation is available for cd9660, nandfs, udf and
pseudofs but hasn't been tested.

Motivation for this feature: our usecase is for a userspace nfs server
(nfs-ganesha) with zfs.  At the moment we cache direntry offsets by
calling lseek once per entry, with this patch we can get the offset
directly from getdirentries(2) calls which provides a significant
speedup.

Submitted by:	Jack Halford <jack@gandi.net>
Reviewed by:	mckusick, pfg, rmacklem (previous versions)
Sponsored by:	Gandi.net
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D17917
2018-11-14 14:18:35 +00:00
Alexander Motin
1fcdb58634 Do not ignore arc_adjust() return value.
This covers scenario when ARC may not shrink as fast as it could:
1. arc_size < arc_c and arc_adjust() does not evict anything, returning
   zero to arc_reclaim_thread();
2. arc_available_memory() reports memory pressure, which can not be
   satisfied by arc_kmem_reap_now();
3. arc_shrink() reduces arc_c and calls arc_adjust(), return of which is
   ignored;
4. even if the last arc_adjust() could not satisfy arc_size < arc_c,
   arc_reclaim_thread() will still go to sleep, since the first one
   returned zero.

Reviewed by:	allanjude, markj, sef
MFC after:	2 weeks
Sponsored by:	iXsystems, Inc.
Differential Revision:	https://reviews.freebsd.org/D17927
2018-11-10 01:58:37 +00:00
Alexander Motin
b4d66a1739 9952 Block size change during zfs receive drops spill block
Replication code in receive_object() falsely assumes that if received
object block size is different from local, then it must be a new object
and calls dmu_object_reclaim() to wipe it out. In most cases it is not a
problem, since all dnode, bonus buffer and data block(s) are immediately
rewritten any way, but the problem is that spill block (if used) is not.
This means loss of ACLs, extended attributes, etc.

This issue can be triggered in very simple way:
1. create 4KB file with 10+ ACL entries;
2. take snapshot and send it to different dataset;
3. append another 4KB to the file;
4. take another snapshot and send incrementally;
5. witness ACL loss on receive side.

PR:		198457
Discussed with:	mahrens
MFC after:	2 weeks
Sponsored by:	iXsystems, Inc.
2018-11-03 03:10:06 +00:00
Brooks Davis
1493c2ee62 Make vop_symlink take a const target path.
This will enable callers to take const paths as part of syscall
decleration improvements.

Where doing so is easy and non-distruptive carry the const through
implementations. In UFS the value is passed to an interface that must
take non-const values. In ZFS, const poisoning would touch code shared
with upstream and it's not worth adding diffs.

Bump __FreeBSD_version for external API consumers.

Reviewed by:	kib (prior version)
Obtained from:	CheriBSD
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D17805
2018-11-02 14:42:36 +00:00
Alexander Motin
2cb74ed856 Skip VDEV_IO_DONE stage only for ZIO_TYPE_FREE.
Device removal code uses zio_vdev_child_io() with ZIO_TYPE_NULL parent,
that never happened before.  It confused FreeBSD-specific TRIM code,
which does not use VDEV_IO_DONE for logical ZIO_TYPE_FREE ZIOs.  As
result of that stage being skipped device removal ZIOs leaked references
and memory that supposed to be freed by VDEV_IO_DONE, making it stuck.

It is a quick patch rather then a nice fix, but hopefully we'll be able
to drop it all together when alternative TRIM implementation finally get
landed.

PR:		228750, 229007
Discussed with:	allanjude, avg, smh
Approved by:	re (delphij)
MFC after:	5 days
Sponsored by:	iXsystems, Inc.
2018-10-15 21:59:24 +00:00
Mateusz Guzik
bca84f54ce zfs: fix a panic after failed mount
r338927("zfs: depessimize zfs_root with rmlocks") failed to error check
the mount before caching root vnode.

Results in crashes in rrw_enter_read_impl tracing back to zfs_mount.

Reported by:	Mike Tancsa
Tested by:	allanjude
Approved by:	re (kib)
2018-10-14 16:14:01 +00:00
Alexander Motin
178777f516 Avoid zero-sized kmem_alloc() in vdev_compact_children().
The device evacuation code adds a dependency that
vdev_compact_children() be able to properly empty the vdev_child
array by setting it to NULL and zeroing vdev_children.  Under Linux,
kmem_alloc() and related functions return a sentinel pointer rather
than NULL for zero-sized allocations.

This is a part of ZoL port of device removal patch:

commit a1d477c24c
Author: Matthew Ahrens <mahrens@delphix.com>
Ported-by: Tim Chase <tim@chase2k.com>

Approved by:	re (kib)
MFC after:	1 week
2018-10-12 16:55:28 +00:00
Alexander Motin
770ce5c3bf Add ZIO_TYPE_FREE support for indirect vdevs.
Upstream code expects only ZIO_TYPE_READ and some ZIO_TYPE_WRITE
requests to removed (indirect) vdevs, while on FreeBSD there is also
ZIO_TYPE_FREE (TRIM).  ZIO_TYPE_FREE requests do not have the data
buffers, so don't need the pointer adjustment.

PR:		228750, 229007
Reviewed by:	allanjude, sef
Approved by:	re (kib)
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D17523
2018-10-12 15:14:22 +00:00
Allan Jude
c79b58ccc5 Pull in a follow-on commit to resolve a deadlock in ZFS sequential
resilver (r334844)

MFV/ZoL: Fix deadlock in IO pipeline

commit a76f3d0437
Author: Brian Behlendorf <behlendorf1@llnl.gov>
Date:   Fri Mar 16 16:46:06 2018 -0700

    Fix deadlock in IO pipeline

    In vdev_queue_aggregate() the zio_execute() bypass should not be
    called under the vdev queue lock.  This can result in a deadlock
    as shown in the stack traces below.

    Drop the vdev queue lock then walk the parents of the aggregate IO
    to determine the list of component IOs to be bypassed.  This can
    be done safely without holding the io_lock since the new aggregate
    IO has not yet been returned and its parents cannot change.

    ---  THREAD 1 ---
    arc_read()
      zio_nowait()
        zio_vdev_io_start()
          vdev_queue_io() <--- mutex_enter(vq->vq_lock)
            vdev_queue_io_to_issue()
              vdev_queue_aggregate()
                zio_execute()
            vdev_queue_io_to_issue()
              vdev_queue_aggregate()
                zio_execute()
                  zio_vdev_io_assess()
                    zio_wait_for_children() <- mutex_enter(zio->io_lock)

    --- THREAD 2 --- (inverse order)
    arc_read()
      zio_change_priority() <- mutex_enter(zio->zio_lock)
        vdev_queue_change_io_priority() <- mutex_enter(vq->vq_lock)

    Reviewed-by: Tom Caputi <tcaputi@datto.com>
    Reviewed-by: Don Brady <don.brady@delphix.com>
    Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>

Reported by:	ZFS Leadership Meeting
Reviewed by:	mav
Approved by:	re (kib)
Obtained from:	ZFS-on-Linux
MFC after:	2 weeks
Sponsored by:	Klara Systems
Differential Revision:	https://reviews.freebsd.org/D17495
2018-10-10 22:59:15 +00:00
Allan Jude
bee8a18986 Add missing sysctls for tuning vdev queue depths for new I/O types
This connects new tunables that were added but not exposed in:
r329502 (zpool remove)
r337007 (zpool initialize)

Reviewed by:	avg
Approved by:	re (kib)
MFC after:	2 weeks
Sponsored by:	Klara Systems
Differential Revision:	https://reviews.freebsd.org/D17494
2018-10-10 22:55:31 +00:00
Allan Jude
cd00e3e1af Resolve a hang in ZFS during vnode reclaimation
This is caused by a deadlock between zil_commit() and zfs_zget()

Add a way for zfs_zget() to break out of the retry loop in the common case

PR:		229614
Reported by:	grembo, Andreas Sommer, many others
Tested by:	Andreas Sommer, Vicki Pfau
Reviewed by:	avg (no objection)
Approved by:	re (gjb)
MFC after:	2 months
Sponsored by:	Klara Systems
Differential Revision:	https://reviews.freebsd.org/D17460
2018-10-10 19:39:47 +00:00
Alexander Motin
7a492ba93c Remove extra thread_exit() call left after r329802.
spa_condense_indirect_thread() is no longer a thread function, but just
a callback for new zthr KPI.

Submitted by:	allanjude
Approved by:	re (gjb)
MFC after:	3 days
2018-10-10 16:34:53 +00:00
Alexander Motin
f3b515aea5 Fix r336951 mismerge -- use of uninitialized variable.
Reported by:	tsoome
Approved by:	re (gjb)
MFC after:	3 days
2018-10-08 15:19:03 +00:00
Alexander Motin
1f55b2a4b5 Add sysctls for dbuf metadata cache variables added in r336959.
Approved by:	re (gjb)
MFC after:	1 week
2018-10-05 16:05:59 +00:00
Allan Jude
9d967dd27d Avoid panic when adjusting priority of a read in the face of an IO error
PR:		231516
Reported by:	sbruno
Approved by:	re (rgrimes)
Obtained from:	ZFS-on-Linux
X-MFC-with:	334844
Sponsored by:	Klara Systems

MFV/ZoL:	Fix zio->io_priority failed (7 < 6) assert

commit c26cf0966d
Author: Tony Hutter <hutter2@llnl.gov>
Date:   Tue May 29 18:13:48 2018 -0700

  Fix zio->io_priority failed (7 < 6) assert

  This fixes an assert in vdev_queue_change_io_priority():

    VERIFY3(zio->io_priority < ZIO_PRIORITY_NUM_QUEUEABLE) failed (7 < 6)
    PANIC at vdev_queue.c:832:vdev_queue_change_io_priority()

  Reviewed-by: Tom Caputi <tcaputi@datto.com>
  Reviewed-by: George Melikov <mail@gmelikov.ru>
  Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
  Signed-off-by: Tony Hutter <hutter2@llnl.gov>
2018-09-29 01:26:07 +00:00
Mateusz Guzik
af534f8d99 zfs: depessimize zfs_root with rmlocks
Currently vfs calls the root method on each absolute lookup and when
crossing mount points.

zfs_root ends up looking up the inode internally as if it was not
instantianted which results in significant lock contention on systems
like EPYC.

Store the vnode in the mount point and protect the access with rmlocks.
This is a temporary hack for 12.0.

Sample result:

before:
make -s -j 128 buildkernel 2778.09s user 3319.45s system 8370% cpu 1:12.85 total

after:
make -s -j 128 buildkernel 3199.57s user 1772.78s system 8232% cpu 1:00.40 total

Tested by:	pho (zfs mount/unmount tests)
Reviewed by:	kib, mav, sef (different parts)
Approved by:	re (gjb)
Differential Revision:	https://reviews.freebsd.org/D17233
2018-09-25 17:58:06 +00:00
Alexander Motin
ae1b0b825a MFV r338866: 9700 ZFS resilvered mirror does not balance reads
illumos/illumos-gate@82f63c3c2b

Reviewed by: Toomas Soome <tsoome@me.com>
Reviewed by: Sanjay Nadkarni <sanjay.nadkarni@nexenta.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Approved by: Matthew Ahrens <mahrens@delphix.com>
Author:     Jerry Jelinek <jerry.jelinek@joyent.com>

Approved by:	re (delphij)
2018-09-21 21:56:00 +00:00
Konstantin Belousov
a6ade1a07b Fix ZFS VFS op quotactl to follow busy protocol.
Reviewed by:	avg, mckusick
Tested by:	pho
Sponsored by:	The FreeBSD Foundation
Approved by:	re (gjb)
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D17208
2018-09-19 14:38:01 +00:00
Mark Johnston
6368b4e471 Fix an nvpair leak in vdev_geom_read_config().
Also change the behaviour slightly: instead of freeing "config" if the
last nvlist doesn't pass the tests, return the last config that did pass
those tests.  This matches the comment at the beginning of the function.

PR:		230704
Diagnosed by:	avg
Reviewed by:	asomers, avg
Tested by:	Mark Martinec <Mark.Martinec@ijs.si>
Approved by:	re (gjb)
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
Differential revision:  https://reviews.freebsd.org/D17202
2018-09-17 16:16:57 +00:00
Eric van Gyzen
73511c241b Set zfs_arc_meta_strategy to metadata only
The previous default of "balanced" appears to have caused pathological
behavior, including very poor performance and 100% CPU load in the
arc_reclaim_thread.

The symptoms appeared when the daily periodic run started.
With this change, the system--and the ARC in particular--behaved
normally during a manual daily periodic run.

From Mark Johnston:  The port of the balanced strategy is incomplete,
since arc_prune_async() is a no-op on FreeBSD.  (This also seems
to imply that r337653 is a no-op.)  After 12 is branched we can
port the remaining bits and consider changing the default back.

Submitted by:	markj (essentially)
Reviewed by:	markj
Approved by:	re (gjb)
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D17156
2018-09-13 17:56:48 +00:00
Ruslan Bukin
378a495661 Add support for 'C'-compressed ISA extension to DTrace FBT provider.
Approved by:	re (kib)
Sponsored by:	DARPA, AFRL
2018-09-03 14:34:09 +00:00
Mark Johnston
d7965243c1 Re-compute the ARC size before computing the MFU target.
This fixes an upstream regression introduced in r331404, causing overly
aggressive reclamation of the ARC when under pressure.

Diagnosed by:	Paul <devgs@ukr.net>
Approved by:	re (gjb)
MFC after:	3 days
2018-08-31 21:45:05 +00:00
Mark Johnston
f554293615 Re-add kstat.zfs.misc.arcstats.other_size under COMPAT_FREEBSD11.
It is used by a number of applications, notably top(1).

Reported by:	netchild
Reviewed by:	allanjude
Approved by:	re (delphij)
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D16943
2018-08-30 13:42:01 +00:00
Konstantin Belousov
f0165b1ca6 Remove {max/min}_offset() macros, use vm_map_{max/min}() inlines.
Exposing max_offset and min_offset defines in public headers is
causing clashes with variable names, for example when building QEMU.

Based on the submission by:	royger
Reviewed by:	alc, markj (previous version)
Sponsored by:	The FreeBSD Foundation (kib)
MFC after:	1 week
Approved by:	re (marius)
Differential revision:	https://reviews.freebsd.org/D16881
2018-08-29 12:24:19 +00:00
Mark Johnston
394e8d20d9 Add a sysctl for the ZFS abd_scatter_enabled setting.
Submitted by:	Yamagi Burmeister <lists@yamagi.org> (original version)
Approved by:	re (rgrimes)
MFC after:	3 days
2018-08-29 02:49:18 +00:00
Alexander Motin
cb892a4117 Unblock speculative prefetcher also on pool creation.
Fix at r331950 appeared to be incomplete, fixing only case of pool
import, but not pool creation, leaving prefetcher still blocked for
newly created pools.

Approved by:	re (gjb)
MFC after:	1 week
2018-08-24 01:59:25 +00:00
Alexander Motin
2efb7660b5 Add dmu_tx_assign() error handling in zfs_unlinked_drain().
The error handling got lost during r334810, while according to the report
error there may happen in case of dataset being over quota.  In such case
just leave the node in the unlinked list to be freed sometimes later.

PR:		229887
Sponsored by:	iXsystems, Inc.
2018-08-22 16:32:53 +00:00
Alexander Motin
6128ca8683 Create separate taskqueue to call zfs_unlinked_drain().
r334810 introduced zfs_unlinked_drain() dispatch to taskqueue on every
deletion of a file with extended attributes.  Using system_taskq for that
with its multiple threads in case of multiple files deletion caused all
available CPU threads to uselessly spin on busy locks, completely blocking
the system.

Use of single dedicated taskqueue is the only easy solution I've found,
while in would be great if we could specify that some task should be
executed only once at a time, but never in parallel, while many tasks
could use different threads same time.

Sponsored by:	iXsystems, Inc.
2018-08-22 16:27:24 +00:00
Mark Johnston
b1a90834bb Set arc_kmem_cache_reap_retry_ms to 0 and make it configurable.
r329759 introduced this parameter, which controls the rate at which ZFS
UMA zones are drained when the ARC reclaim thread is shrinking the ARC.
The reclamation target is derived from the global free page count, and
arc_shrink() only frees buffers back to UMA, so the free page count is
not updated until the zones are drained.  Thus, back-to-back calls to
arc_shrink() within the arc_kmem_cache_reap_retry_ms interval do not
provide immediate feedback to the arc_reclaim control loop, so we may
free more of the ARC than needed to address a transient page shortage.

As we do not implement the asynchronous zone draining added in r329759,
disable the retry interval, restoring pre-r329759 behaviour.  That is,
we will drain the ZFS UMA zones before each attempt to shrink the ARC.

Reviewed by:	mav
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2018-08-21 16:37:37 +00:00
Matt Macy
d12e91d584 Make dnode definition uniform on !x86
gcc4 requires -fms-extensions to accept anonymous union members
2018-08-21 03:45:09 +00:00
Alexander Motin
cd2315086a 9751 Allocation throttling misplacing ditto blocks
Relax allocation throttling for ditto blocks.  Due to random imbalances
in allocation it tends to push block copies to one vdev, that looks
slightly better at the moment.  Slightly less strict policy allows both
improve data security and surprisingly write performance, since we don't
need to touch extra metaslabs on each vdev to respect the min distance.

Sponsored by:	iXsystems, Inc.
2018-08-17 15:17:09 +00:00
Alexander Motin
a8e93e3cd7 9738 Fix third block copy allocations, broken at 9112.
Use METASLAB_WEIGHT_CLAIM weight to allocate tertiary blocks.
Previous use of METASLAB_WEIGHT_SECONDARY for that caused errors
later on metaslab_activate_allocator() call, leading to massive
load of unneeded metaslabs and write freezes.

Reviewed by:	Paul Dagnelie <pcd@delphix.com>
2018-08-17 15:00:41 +00:00
Alexander Motin
6d14f2c48f Make vfs.zfs.zio.dva_throttle_enabled sysctl writable.
Not sure what I thought originally, but as I see now runtime changes are
working fine, and the code seems like even designed for this.
2018-08-16 18:44:50 +00:00
Jamie Gritton
284001a222 Put jail(2) under COMPAT_FREEBSD11. It has been the "old" way of creating
jails since FreeBSD 7.

Along with the system call, put the various security.jail.allow_foo and
security.jail.foo_allowed sysctls partly under COMPAT_FREEBSD11 (or
BURN_BRIDGES).  These sysctls had two disparate uses: on the system side,
they were global permissions for jails created via jail(2) which lacked
fine-grained permission controls; inside a jail, they're read-only
descriptions of what the current jail is allowed to do.  The first use
is obsolete along with jail(2), but keep them for the second-read-only use.

Differential Revision:	D14791
2018-08-16 18:40:16 +00:00
Alexander Motin
edc391e922 Add couple tunables/sysctl, missed in r336949. 2018-08-16 00:50:14 +00:00
Alexander Motin
8ce70dfcfa Fix mismerge in r337196.
ZoL did the same mistake, and fixed it with separate commit 863522b1f9:

dsl_scan_scrub_cb: don't double-account non-embedded blocks

We were doing count_block() twice inside this function, once
unconditionally at the beginning (intended to catch the embedded block
case) and once near the end after processing the block.

The double-accounting caused the "zpool scrub" progress statistics in
"zpool status" to climb from 0% to 200% instead of 0% to 100%, and
showed double the I/O rate it was actually seeing.

This was apparently a regression introduced in commit 00c405b4b5,
which was an incorrect port of this OpenZFS commit:

    https://github.com/openzfs/openzfs/commit/d8a447a7

Reviewed by: Thomas Caputi <tcaputi@datto.com>
Reviewed by: Matt Ahrens <matt@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: George Melikov <mail@gmelikov.ru>
Signed-off-by: Steven Noonan <steven@uplinklabs.net>
Closes #7720
Closes #7738

Reported by:	sef
2018-08-15 21:01:57 +00:00
Matt Macy
fb8f55f586 MFV/ZoL: Add dbuf hash and dbuf cache kstats
TODO: KSTAT_TYPE_NAMED support

commit 5e021f56d3
Author: Giuseppe Di Natale <dinatale2@users.noreply.github.com>
Date:   Mon Jan 29 10:24:52 2018 -0800

    Add dbuf hash and dbuf cache kstats

    Introduce kstats about the dbuf hash and dbuf cache
    to make it easier to inspect state. This should help
    with debugging and understanding of these portions
    of the codebase.

    Correct format of dbuf kstat file.

    Introduce a dbc column to dbufs kstat to indicate if
    a dbuf is in the dbuf cache.

    Introduce field filtering in the dbufstat python script.

    Introduce a no header option to the dbufstat python script.

    Introduce a test case to test basic mru->mfu list movement
    in the ARC.

    Reviewed-by: Tony Hutter <hutter2@llnl.gov>
    Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Giuseppe Di Natale <dinatale2@llnl.gov>
    Closes #6906
2018-08-12 03:15:30 +00:00
Matt Macy
13ae5c6ba8 MFV/ZoL: Fix stack dbuf_hold_impl()
commit fc5bb51f08
Author: Brian Behlendorf <behlendorf1@llnl.gov>
Date:   Thu Aug 26 10:52:00 2010 -0700

    Fix stack dbuf_hold_impl()

    This commit preserves the recursive function dbuf_hold_impl() but moves
    the local variables and function arguments to the heap to minimize
    the stack frame size.  Enough space is initially allocated on the
    stack for 20 levels of recursion.  This technique was based on commit
    34229a2f2ac07363f64ddd63e014964fff2f0671 which reduced stack usage of
    traverse_visitbp().

    Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
2018-08-12 02:24:18 +00:00
Matt Macy
6e3d1345d9 fix build DN_MAX_BONUSLEN -> DN_OLD_MAX_BONUSLEN 2018-08-12 02:12:44 +00:00
Matt Macy
0f5add2566 Restore legacy dnode_phys layout on tier 2 arches
Evidently gcc4 doesn't support anonymous union members
2018-08-12 02:09:06 +00:00
Matt Macy
104ed324dd MFV/ZoL: Fix stack noinline
commit 60948de1ef
Author: Brian Behlendorf <behlendorf1@llnl.gov>
Date:   Thu Aug 26 10:58:36 2010 -0700

    Fix stack noinline

    Certain function must never be automatically inlined by gcc because
    they are stack heavy or called recursively.  This patch flags all
    such functions I've found as 'noinline' to prevent gcc from making
    the optimization.

    Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
2018-08-12 01:29:30 +00:00
Matt Macy
71d48dbda3 MFV/ZoL: Fix PANIC: metaslab_free_dva(): bad DVA X:Y:Z
commit 81edd3e834
Author: Peng <peng.hse@xtaotech.com>
Date:   Wed Jun 8 15:22:07 2016 +0800

    Fix PANIC: metaslab_free_dva(): bad DVA X:Y:Z

    The following scenario can result in garbage in the dn_spill field.
    The db->db_blkptr must be set to NULL when DNODE_FLAG_SPILL_BLKPTR
    is clear to ensure the dn_spill field is cleared.

    Current txg = A.
    * A new spill buffer is created. Its dbuf is initialized with
      db_blkptr = NULL and it's dirtied.

    Current txg = B.
    * The spill buffer is modified. It's marked as dirty in this txg.
    * Additional changes make the spill buffer unnecessary because the
      xattr fits into the bonus buffer, so it's removed. The dbuf is
      undirtied in this txg, but it's still referenced and cannot be
      destroyed.

    Current txg = C.
    * Starts syncing of txg A
    * dbuf_sync_leaf() is called for the spill buffer. Since db_blkptr
      is NULL, dbuf_check_blkptr() is called.
    * The dbuf starts being written and it reaches the ready state
      (not done yet).
    * A new change makes the spill buffer necessary again.
      sa_build_layouts() ends up calling dbuf_find() to locate the
      dbuf.  It finds the old dbuf because it has not been destroyed yet
      (it will be destroyed when the previous write is done and there
      are no more references). The old dbuf has db_blkptr != NULL.
    * txg A write is complete and the dbuf released. However it's still
      referenced, so it's not destroyed.

    Current txg = D.
    * Starts syncing of txg B
    * dbuf_sync_leaf() is called for the bonus buffer. Its contents are
      directly copied into the dnode, overwriting the blkptr area because,
      in txg B, the bonus buffer was big enough to hold the entire xattr.
    * At this point, the db_blkptr of the spill buffer used in txg C
      gets corrupted.

    Signed-off-by: Peng <peng.hse@xtaotech.com>
    Signed-off-by: Tim Chase <tim@chase2k.com>
    Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Closes #3937
2018-08-12 01:17:32 +00:00
Matt Macy
6f06a36d47 MFV/ZoL: add dbuf stats
NB: disabled pending the addition of KSTAT_TYPE_RAW support to the
SPL

commit e0b0ca983d
Author: Brian Behlendorf <behlendorf1@llnl.gov>
Date:   Wed Oct 2 17:11:19 2013 -0700

    Add visibility in to cached dbufs

    Currently there is no mechanism to inspect which dbufs are being
    cached by the system.  There are some coarse counters in arcstats
    by they only give a rough idea of what's being cached.  This patch
    aims to improve the current situation by adding a new dbufs kstat.

    When read this new kstat will walk all cached dbufs linked in to
    the dbuf_hash.  For each dbuf it will dump detailed information
    about the buffer.  It will also dump additional information about
    the referenced arc buffer and its related dnode.  This provides a
    more complete view in to exactly what is being cached.

    With this generic infrastructure in place utilities can be written
    to post-process the data to understand exactly how the caching is
    working.  For example, the data could be processed to show a list
    of all cached dnodes and how much space they're consuming.  Or a
    similar list could be generated based on dnode type.  Many other
    ways to interpret the data exist based on what kinds of questions
    you're trying to answer.

    Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Signed-off-by: Prakash Surya <surya1@llnl.gov>
2018-08-12 01:10:18 +00:00
Matt Macy
cc0fbbb92e MFV/ZoL: Implement large_dnode pool feature
commit 50c957f702
Author: Ned Bass <bass6@llnl.gov>
Date:   Wed Mar 16 18:25:34 2016 -0700

    Implement large_dnode pool feature

    Justification
    -------------

    This feature adds support for variable length dnodes. Our motivation is
    to eliminate the overhead associated with using spill blocks.  Spill
    blocks are used to store system attribute data (i.e. file metadata) that
    does not fit in the dnode's bonus buffer. By allowing a larger bonus
    buffer area the use of a spill block can be avoided.  Spill blocks
    potentially incur an additional read I/O for every dnode in a dnode
    block. As a worst case example, reading 32 dnodes from a 16k dnode block
    and all of the spill blocks could issue 33 separate reads. Now suppose
    those dnodes have size 1024 and therefore don't need spill blocks.  Then
    the worst case number of blocks read is reduced to from 33 to two--one
    per dnode block. In practice spill blocks may tend to be co-located on
    disk with the dnode blocks so the reduction in I/O would not be this
    drastic. In a badly fragmented pool, however, the improvement could be
    significant.

    ZFS-on-Linux systems that make heavy use of extended attributes would
    benefit from this feature. In particular, ZFS-on-Linux supports the
    xattr=sa dataset property which allows file extended attribute data
    to be stored in the dnode bonus buffer as an alternative to the
    traditional directory-based format. Workloads such as SELinux and the
    Lustre distributed filesystem often store enough xattr data to force
    spill bocks when xattr=sa is in effect. Large dnodes may therefore
    provide a performance benefit to such systems.

    Other use cases that may benefit from this feature include files with
    large ACLs and symbolic links with long target names. Furthermore,
    this feature may be desirable on other platforms in case future
    applications or features are developed that could make use of a
    larger bonus buffer area.

    Implementation
    --------------

    The size of a dnode may be a multiple of 512 bytes up to the size of
    a dnode block (currently 16384 bytes). A dn_extra_slots field was
    added to the current on-disk dnode_phys_t structure to describe the
    size of the physical dnode on disk. The 8 bits for this field were
    taken from the zero filled dn_pad2 field. The field represents how
    many "extra" dnode_phys_t slots a dnode consumes in its dnode block.
    This convention results in a value of 0 for 512 byte dnodes which
    preserves on-disk format compatibility with older software.

    Similarly, the in-memory dnode_t structure has a new dn_num_slots field
    to represent the total number of dnode_phys_t slots consumed on disk.
    Thus dn->dn_num_slots is 1 greater than the corresponding
    dnp->dn_extra_slots. This difference in convention was adopted
    because, unlike on-disk structures, backward compatibility is not a
    concern for in-memory objects, so we used a more natural way to
    represent size for a dnode_t.

    The default size for newly created dnodes is determined by the value of
    a new "dnodesize" dataset property. By default the property is set to
    "legacy" which is compatible with older software. Setting the property
    to "auto" will allow the filesystem to choose the most suitable dnode
    size. Currently this just sets the default dnode size to 1k, but future
    code improvements could dynamically choose a size based on observed
    workload patterns. Dnodes of varying sizes can coexist within the same
    dataset and even within the same dnode block. For example, to enable
    automatically-sized dnodes, run

     # zfs set dnodesize=auto tank/fish

    The user can also specify literal values for the dnodesize property.
    These are currently limited to powers of two from 1k to 16k. The
    power-of-2 limitation is only for simplicity of the user interface.
    Internally the implementation can handle any multiple of 512 up to 16k,
    and consumers of the DMU API can specify any legal dnode value.

    The size of a new dnode is determined at object allocation time and
    stored as a new field in the znode in-memory structure. New DMU
    interfaces are added to allow the consumer to specify the dnode size
    that a newly allocated object should use. Existing interfaces are
    unchanged to avoid having to update every call site and to preserve
    compatibility with external consumers such as Lustre. The new
    interfaces names are given below. The versions of these functions that
    don't take a dnodesize parameter now just call the _dnsize() versions
    with a dnodesize of 0, which means use the legacy dnode size.

    New DMU interfaces:
      dmu_object_alloc_dnsize()
      dmu_object_claim_dnsize()
      dmu_object_reclaim_dnsize()

    New ZAP interfaces:
      zap_create_dnsize()
      zap_create_norm_dnsize()
      zap_create_flags_dnsize()
      zap_create_claim_norm_dnsize()
      zap_create_link_dnsize()

    The constant DN_MAX_BONUSLEN is renamed to DN_OLD_MAX_BONUSLEN. The
    spa_maxdnodesize() function should be used to determine the maximum
    bonus length for a pool.

    These are a few noteworthy changes to key functions:

    * The prototype for dnode_hold_impl() now takes a "slots" parameter.
      When the DNODE_MUST_BE_FREE flag is set, this parameter is used to
      ensure the hole at the specified object offset is large enough to
      hold the dnode being created. The slots parameter is also used
      to ensure a dnode does not span multiple dnode blocks. In both of
      these cases, if a failure occurs, ENOSPC is returned. Keep in mind,
      these failure cases are only possible when using DNODE_MUST_BE_FREE.

      If the DNODE_MUST_BE_ALLOCATED flag is set, "slots" must be 0.
      dnode_hold_impl() will check if the requested dnode is already
      consumed as an extra dnode slot by an large dnode, in which case
      it returns ENOENT.

    * The function dmu_object_alloc() advances to the next dnode block
      if dnode_hold_impl() returns an error for a requested object.
      This is because the beginning of the next dnode block is the only
      location it can safely assume to either be a hole or a valid
      starting point for a dnode.

    * dnode_next_offset_level() and other functions that iterate
      through dnode blocks may no longer use a simple array indexing
      scheme. These now use the current dnode's dn_num_slots field to
      advance to the next dnode in the block. This is to ensure we
      properly skip the current dnode's bonus area and don't interpret it
      as a valid dnode.

    zdb
    ---
    The zdb command was updated to display a dnode's size under the
    "dnsize" column when the object is dumped.

    For ZIL create log records, zdb will now display the slot count for
    the object.

    ztest
    -----
    Ztest chooses a random dnodesize for every newly created object. The
    random distribution is more heavily weighted toward small dnodes to
    better simulate real-world datasets.

    Unused bonus buffer space is filled with non-zero values computed from
    the object number, dataset id, offset, and generation number.  This
    helps ensure that the dnode traversal code properly skips the interior
    regions of large dnodes, and that these interior regions are not
    overwritten by data belonging to other dnodes. A new test visits each
    object in a dataset. It verifies that the actual dnode size matches what
    was stored in the ztest block tag when it was created. It also verifies
    that the unused bonus buffer space is filled with the expected data
    patterns.

    ZFS Test Suite
    --------------
    Added six new large dnode-specific tests, and integrated the dnodesize
    property into existing tests for zfs allow and send/recv.

    Send/Receive
    ------------
    ZFS send streams for datasets containing large dnodes cannot be received
    on pools that don't support the large_dnode feature. A send stream with
    large dnodes sets a DMU_BACKUP_FEATURE_LARGE_DNODE flag which will be
    unrecognized by an incompatible receiving pool so that the zfs receive
    will fail gracefully.

    While not implemented here, it may be possible to generate a
    backward-compatible send stream from a dataset containing large
    dnodes. The implementation may be tricky, however, because the send
    object record for a large dnode would need to be resized to a 512
    byte dnode, possibly kicking in a spill block in the process. This
    means we would need to construct a new SA layout and possibly
    register it in the SA layout object. The SA layout is normally just
    sent as an ordinary object record. But if we are constructing new
    layouts while generating the send stream we'd have to build the SA
    layout object dynamically and send it at the end of the stream.

    For sending and receiving between pools that do support large dnodes,
    the drr_object send record type is extended with a new field to store
    the dnode slot count. This field was repurposed from unused padding
    in the structure.

    ZIL Replay
    ----------
    The dnode slot count is stored in the uppermost 8 bits of the lr_foid
    field. The bits were unused as the object id is currently capped at
    48 bits.

    Resizing Dnodes
    ---------------
    It should be possible to resize a dnode when it is dirtied if the
    current dnodesize dataset property differs from the dnode's size, but
    this functionality is not currently implemented. Clearly a dnode can
    only grow if there are sufficient contiguous unused slots in the
    dnode block, but it should always be possible to shrink a dnode.
    Growing dnodes may be useful to reduce fragmentation in a pool with
    many spill blocks in use. Shrinking dnodes may be useful to allow
    sending a dataset to a pool that doesn't support the large_dnode
    feature.

    Feature Reference Counting
    --------------------------
    The reference count for the large_dnode pool feature tracks the
    number of datasets that have ever contained a dnode of size larger
    than 512 bytes. The first time a large dnode is created in a dataset
    the dataset is converted to an extensible dataset. This is a one-way
    operation and the only way to decrement the feature count is to
    destroy the dataset, even if the dataset no longer contains any large
    dnodes. The complexity of reference counting on a per-dnode basis was
    too high, so we chose to track it on a per-dataset basis similarly to
    the large_block feature.

    Signed-off-by: Ned Bass <bass6@llnl.gov>
    Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Closes #3542
2018-08-12 00:45:53 +00:00
Matt Macy
9f3a171221 Enable balanced arc pruning
Taken from:
ommit f604673836
Author: Brian Behlendorf <behlendorf1@llnl.gov>
Date:   Sat May 30 09:57:53 2015 -0500

    Make arc_prune() asynchronous

    As described in the comment above arc_adapt_thread() it is critical
    that the arc_adapt_thread() function never sleep while holding a hash
    lock.  This behavior was possible in the Linux implementation because
    the arc_prune() logic was implemented to be synchronous.  Under
    illumos the analogous dnlc_reduce_cache() function is asynchronous.

    To address this the arc_do_user_prune() function is has been reworked
    in to two new functions as follows:

    * arc_prune_async() is an asynchronous implementation which dispatches
    the prune callback to be run by the system taskq.  This makes it
    suitable to use in the context of the arc_adapt_thread().

    * arc_prune() is a synchronous implementation which depends on the
    arc_prune_async() implementation but blocks until the outstanding
    callbacks complete.  This is used in arc_kmem_reap_now() where it
    is safe, and expected, that memory will be freed.

    This patch additionally adds the zfs_arc_meta_strategy module option
    while allows the meta reclaim strategy to be configured.  It defaults
    to a balanced strategy which has been proved to work well under Linux
    but the illumos meta-only strategy can be enabled.

    Signed-off-by: Tim Chase <tim@chase2k.com>
    Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
2018-08-11 22:01:52 +00:00
Matt Macy
d815f5ba09 buildworld fix: private appears to have special meaning on FreeBSD - revert to priv 2018-08-11 20:41:42 +00:00
Matt Macy
6b55e6fb04 Limit the amount of dnode metadata in the ARC
In addition import most recent arc_prune_async implementation as dependency

commit 25458cbef9
Author: Tim Chase <tim@chase2k.com>
Date:   Wed Jul 13 07:42:40 2016 -0500

    Limit the amount of dnode metadata in the ARC

    Metadata-intensive workloads can cause the ARC to become permanently
    filled with dnode_t objects as they're pinned by the VFS layer.
    Subsequent data-intensive workloads may only benefit from about
    25% of the potential ARC (arc_c_max - arc_meta_limit).

    In order to help track metadata usage more precisely, the other_size
    metadata arcstat has replaced with dbuf_size, dnode_size and bonus_size.

    The new zfs_arc_dnode_limit tunable, which defaults to 10% of
    zfs_arc_meta_limit, defines the minimum number of bytes which is desirable
    to be consumed by dnodes.  Attempts to evict non-metadata will trigger
    async prune tasks if the space used by dnodes exceeds this limit.

    The new zfs_arc_dnode_reduce_percent tunable specifies the amount by
    which the excess dnode space is attempted to be pruned as a percentage of
    the amount by which zfs_arc_dnode_limit is being exceeded.  By default,
    it tries to unpin 10% of the dnodes.

    The problem of dnode metadata pinning was observed with the following
    testing procedure (in this example, zfs_arc_max is set to 4GiB):

        - Create a large number of small files until arc_meta_used exceeds
          arc_meta_limit (3GiB with default tuning) and arc_prune
          starts increasing.

        - Create a 3GiB file with dd.  Observe arc_mata_used.  It will still
          be around 3GiB.

        - Repeatedly read the 3GiB file and observe arc_meta_limit as before.
          It will continue to stay around 3GiB.

    With this modification, space for the 3GiB file is gradually made
    available as subsequent demands on the ARC are made.  The previous behavior
    can be restored by setting zfs_arc_dnode_limit to the same value as the
    zfs_arc_meta_limit.

    Signed-off-by: Tim Chase <tim@chase2k.com>
    Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Issue #4345
    Issue #4512
    Issue #4773
    Closes #4858
2018-08-11 19:45:04 +00:00
Matt Macy
90df93417e ZFS/MFV: Use cached feature info in spa_add_feature_stats()
commit 417104bdd3
Author: Ned Bass <bass6@llnl.gov>
Date:   Thu Feb 26 12:24:11 2015 -0800

    Use cached feature info in spa_add_feature_stats()

    Avoid issuing I/O to the pool when retrieving feature flags information.
    Trying to read the ZAPs from disk means that zpool clear would hang if
    the pool is suspended and recovery would require a reboot. To keep the
    feature stats resident in memory, we hang a cached nvlist off of the
    spa.  It is built up from disk the first time spa_add_feature_stats() is
    called, and refreshed thereafter using the cached feature reference
    counts. spa_add_feature_stats() gets called at pool import time so we
    can be sure the cached nvlist will be available if the pool is later
    suspended.

    Signed-off-by: Ned Bass <bass6@llnl.gov>
    Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Closes #3082
2018-08-10 23:42:11 +00:00
Matt Macy
648cfe57fd Performance optimization of AVL tree comparator functions
MFV:
commit ee36c709c3
Author: Gvozden Neskovic <neskovic@gmail.com>
Date:   Sat Aug 27 20:12:53 2016 +0200

    perf: 2.75x faster ddt_entry_compare()
        First 256bits of ddt_key_t is a block checksum, which are expected
    to be close to random data. Hence, on average, comparison only needs to
    look at first few bytes of the keys. To reduce number of conditional
    jump instructions, the result is computed as: sign(memcmp(k1, k2)).

    Sign of an integer 'a' can be obtained as: `(0 < a) - (a < 0)` := {-1, 0, 1} ,
    which is computed efficiently.  Synthetic performance evaluation of
    original and new algorithm over 1G random keys on 2.6GHz Intel(R) Xeon(R)
    CPU E5-2660 v3:

    old     6.85789 s
    new     2.49089 s

    perf: 2.8x faster vdev_queue_offset_compare() and vdev_queue_timestamp_compare()
        Compute the result directly instead of using conditionals

    perf: zfs_range_compare()
        Speedup between 1.1x - 2.5x, depending on compiler version and
    optimization level.

    perf: spa_error_entry_compare()
        `bcmp()` is not suitable for comparator use. Use `memcmp()` instead.

    perf: 2.8x faster metaslab_compare() and metaslab_rangesize_compare()
    perf: 2.8x faster zil_bp_compare()
    perf: 2.8x faster mze_compare()
    perf: faster dbuf_compare()
    perf: faster compares in spa_misc
    perf: 2.8x faster layout_hash_compare()
    perf: 2.8x faster space_reftree_compare()
    perf: libzfs: faster avl tree comparators
    perf: guid_compare()
    perf: dsl_deadlist_compare()
    perf: perm_set_compare()
    perf: 2x faster range_tree_seg_compare()
    perf: faster unique_compare()
    perf: faster vdev_cache _compare()
    perf: faster vdev_uberblock_compare()
    perf: faster fuid _compare()
    perf: faster zfs_znode_hold_compare()

    Signed-off-by: Gvozden Neskovic <neskovic@gmail.com>
    Signed-off-by: Richard Elling <richard.elling@gmail.com>
    Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Closes #5033
2018-08-10 06:42:08 +00:00
Alexander Motin
f58c851d32 Reduce taskq and context-switch cost of zio pipe
When doing a read from disk, ZFS creates 3 ZIO's: a zio_null(), the
logical zio_read(), and then a physical zio. Currently, each of these
results in a separate taskq_dispatch(zio_execute).

On high-read-iops workloads, this causes a significant performance
impact. By processing all 3 ZIO's in a single taskq entry, we reduce the
overhead on taskq locking and context switching.  We accomplish this by
allowing zio_done() to return a "next zio to execute" to zio_execute().

This results in a ~12% performance increase for random reads, from
96,000 iops to 108,000 iops (with recordsize=8k, on SSD's).

Reviewed by: Pavel Zakharov <pavel.zakharov@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed by: George Wilson <george.wilson@delphix.com>
Signed-off-by: Matthew Ahrens <mahrens@delphix.com>
External-issue: DLPX-59292
Closes #7736

zfsonlinux/zfs@62840030a7
2018-08-03 02:16:45 +00:00
Alexander Motin
07ddc55096 MFV r337223:
9580 Add a hash-table on top of nvlist to speed-up operations

illumos/illumos-gate@2ec7644aab

Reviewed by: Matt Ahrens <matt@delphix.com>
Reviewed by: Sebastien Roy <sebastien.roy@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author: Serapheim Dimitropoulos <serapheim@delphix.com>
2018-08-03 01:52:25 +00:00
Alexander Motin
420a1ab349 MFV r337220: 8375 Kernel memory leak in nvpair code
illumos/illumos-gate@843c2111b1

Reviewed by: Pavel Zakharov <pavel.zakharov@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Prashanth Sreenivasa <pks@delphix.com>
Reviewed by: Robert Mustacchi <rm@joyent.com>
Approved by: Dan McDonald <danmcd@joyent.com>
Author: Matthew Ahrens <mahrens@delphix.com>
2018-08-03 01:30:03 +00:00
Alexander Motin
33bdecc5d0 MFV r337218: 7261 nvlist code should enforce name length limit
illumos/illumos-gate@48dd5e630c

Reviewed by: Sebastien Roy <sebastien.roy@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Robert Mustacchi <rm@joyent.com>
Approved by: Dan McDonald <danmcd@omniti.com>
Author: Matthew Ahrens <mahrens@delphix.com>
2018-08-03 01:26:07 +00:00
Alexander Motin
c8c51346f5 MFV r337216: 7263 deeply nested nvlist can overflow stack
illumos/illumos-gate@9ca527c3d3

Reviewed by: Adam Leventhal <ahl@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Robert Mustacchi <rm@joyent.com>
Approved by: Dan McDonald <danmcd@omniti.com>
Author: Matthew Ahrens <mahrens@delphix.com>
2018-08-03 01:09:12 +00:00
Alexander Motin
0285589b38 MFV 337214:
9621 Make createtxg and guid properties public

illumos/illumos-gate@e8d4a73c86

Reviewed by: Andy Stormont <astormont@racktopsystems.com>
Reviewed by: Paul Dagnelie <pcd@delphix.com>
Reviewed by: Matt Ahrens <matt@delphix.com>
Reviewed by: Yuri Pankov <yuripv@yuripv.net>
Approved by: Robert Mustacchi <rm@joyent.com>
Author:     Josh Paetzel <josh@tcbug.org>
2018-08-03 00:24:27 +00:00
Alexander Motin
54b838a67c MFV r337212:
9465 ARC check for 'anon_size > arc_c/2' can stall the system

illumos/illumos-gate@abe1fd01ce

Reviewed by: Sebastien Roy <sebastien.roy@delphix.com>
Reviewed by: Matt Ahrens <matt@delphix.com>
Reviewed by: Prashanth Sreenivasa <pks@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author:     Don Brady <don.brady@delphix.com>
2018-08-03 00:14:36 +00:00