d/freebsd-dev
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
2bce8049c3
freebsd-dev/module/zfs
History
Matthew Ahrens 2bce8049c3 OpenZFS 7004 - dmu_tx_hold_zap() does dnode_hold() 7x on same object 
Using a benchmark which has 32 threads creating 2 million files in the
same directory, on a machine with 16 CPU cores, I observed poor
performance. I noticed that dmu_tx_hold_zap() was using about 30% of
all CPU, and doing dnode_hold() 7 times on the same object (the ZAP
object that is being held).

dmu_tx_hold_zap() keeps a hold on the dnode_t the entire time it is
running, in dmu_tx_hold_t:txh_dnode, so it would be nice to use the
dnode_t that we already have in hand, rather than repeatedly calling
dnode_hold(). To do this, we need to pass the dnode_t down through
all the intermediate calls that dmu_tx_hold_zap() makes, making these
routines take the dnode_t* rather than an objset_t* and a uint64_t
object number. In particular, the following routines will need to have
analogous *_by_dnode() variants created:

dmu_buf_hold_noread()
dmu_buf_hold()
zap_lookup()
zap_lookup_norm()
zap_count_write()
zap_lockdir()
zap_count_write()

This can improve performance on the benchmark described above by 100%,
from 30,000 file creations per second to 60,000. (This improvement is on
top of that provided by working around the object allocation issue. Peak
performance of ~90,000 creations per second was observed with 8 CPUs;
adding CPUs past that decreased performance due to lock contention.) The
CPU used by dmu_tx_hold_zap() was reduced by 88%, from 340 CPU-seconds
to 40 CPU-seconds.

Sponsored by: Intel Corp.

Signed-off-by: Matthew Ahrens <mahrens@delphix.com>
Signed-off-by: Ned Bass <bass6@llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>

OpenZFS-issue: https://www.illumos.org/issues/7004
OpenZFS-commit: https://github.com/openzfs/openzfs/pull/109
Closes #4641
Closes #4972
2016-08-19 12:48:03 -07:00
..
arc.c arc_meta_limit should be updated when arc_max is changed. 2016-08-02 13:43:36 -07:00
blkptr.c Illumos 4757, 4913 2014-08-01 14:28:05 -07:00
bplist.c Change KM_PUSHPAGE -> KM_SLEEP 2015-01-16 14:41:26 -08:00
bpobj.c Illumos 5810 - zdb should print details of bpobj 2015-05-11 15:10:24 -07:00
bptree.c Illumos 5960, 5925 2016-01-08 15:08:19 -08:00
bqueue.c Allow 16M send/recv blocks 2016-01-08 20:23:23 -05:00
dbuf_stats.c Fix dbuf_stats_hash_table_data race 2016-07-14 16:25:04 -07:00
dbuf.c OpenZFS 7004 - dmu_tx_hold_zap() does dnode_hold() 7x on same object 2016-08-19 12:48:03 -07:00
ddt_zap.c Change KM_PUSHPAGE -> KM_SLEEP 2015-01-16 14:41:26 -08:00
ddt.c Handle zap_lookup() failure in ddt_object_load() 2015-08-19 14:32:50 -07:00
dmu_diff.c Illumos 5960, 5925 2016-01-08 15:08:19 -08:00
dmu_object.c Implement large_dnode pool feature 2016-06-24 13:13:21 -07:00
dmu_objset.c OpenZFS 6314 - buffer overflow in dsl_dataset_name 2016-06-28 13:47:03 -07:00
dmu_send.c OpenZFS 6314 - buffer overflow in dsl_dataset_name 2016-06-28 13:47:03 -07:00
dmu_traverse.c Add tunable to ignore hole_birth 2016-08-15 09:52:56 -07:00
dmu_tx.c OpenZFS 7004 - dmu_tx_hold_zap() does dnode_hold() 7x on same object 2016-08-19 12:48:03 -07:00
dmu_zfetch.c Illumos 6281 - prefetching should apply to 1MB reads 2016-01-12 13:51:27 -08:00
dmu.c OpenZFS 7004 - dmu_tx_hold_zap() does dnode_hold() 7x on same object 2016-08-19 12:48:03 -07:00
dnode_sync.c Implement large_dnode pool feature 2016-06-24 13:13:21 -07:00
dnode.c Limit the amount of dnode metadata in the ARC 2016-07-25 15:26:38 -07:00
dsl_bookmark.c OpenZFS 6314 - buffer overflow in dsl_dataset_name 2016-06-28 13:47:03 -07:00
dsl_dataset.c Rework of fletcher_4 module 2016-08-16 14:11:55 -07:00
dsl_deadlist.c Handle damaged blk_birth in dsl_deadlist_insert() 2015-12-15 16:12:31 -08:00
dsl_deleg.c OpenZFS 6314 - buffer overflow in dsl_dataset_name 2016-06-28 13:47:03 -07:00
dsl_destroy.c OpenZFS 2605, 6980, 6902 2016-06-28 13:47:02 -07:00
dsl_dir.c OpenZFS 6314 - buffer overflow in dsl_dataset_name 2016-06-28 13:47:03 -07:00
dsl_pool.c Fix self-healing IO prior to dsl_pool_init() completion 2016-05-27 14:11:25 -07:00
dsl_prop.c OpenZFS 6314 - buffer overflow in dsl_dataset_name 2016-06-28 13:47:03 -07:00
dsl_scan.c OpenZFS 5997 - FRU field not set during pool creation and never updated 2016-08-12 13:06:48 -07:00
dsl_synctask.c Illumos 4951 - ZFS administrative commands should use reserved space 2015-05-04 09:41:10 -07:00
dsl_userhold.c OpenZFS 6314 - buffer overflow in dsl_dataset_name 2016-06-28 13:47:03 -07:00
fm.c Fix indefinite article 2016-08-11 11:23:49 -07:00
gzip.c cstyle: Resolve C style issues 2013-12-18 16:46:35 -08:00
lz4.c Change KM_PUSHPAGE -> KM_SLEEP 2015-01-16 14:41:26 -08:00
lzjb.c Change KM_PUSHPAGE -> KM_SLEEP 2015-01-16 14:41:26 -08:00
Makefile.in Add RAID-Z routines for SSE2 instruction set, in x86_64 mode. 2016-07-13 10:24:55 -07:00
metaslab.c Fix gcc self-comparison warning 2016-08-02 13:14:18 -07:00
multilist.c Identify locks flagged by lockdep 2015-12-22 10:21:33 -08:00
pathname.c Add pn_alloc()/pn_free() functions 2016-04-21 09:49:25 -07:00
policy.c Fix NFS credential 2016-06-21 09:58:37 -07:00
range_tree.c Illumos 5163 - arc should reap range_seg_cache 2015-06-25 08:58:16 -07:00
refcount.c Change KM_PUSHPAGE -> KM_SLEEP 2015-01-16 14:41:26 -08:00
rrwlock.c Illumos 5008 - lock contention (rrw_exit) while running a read only load 2015-07-06 09:34:13 -07:00
sa.c Implement large_dnode pool feature 2016-06-24 13:13:21 -07:00
sha256.c Add linux sha2 support 2010-08-31 13:41:59 -07:00
spa_boot.c Add linux kernel module support 2010-08-31 13:41:58 -07:00
spa_config.c OpenZFS 5997 - FRU field not set during pool creation and never updated 2016-08-12 13:06:48 -07:00
spa_errlog.c Illumos 4914 - zfs on-disk bookmark structure should be named *_phys_t 2014-08-06 14:48:41 -07:00
spa_history.c Fix indefinite article 2016-08-11 11:23:49 -07:00
spa_misc.c Implement large_dnode pool feature 2016-06-24 13:13:21 -07:00
spa_stats.c Illumos 5369 - arc flags should be an enum 2015-06-11 10:27:25 -07:00
spa.c Fix spa config generate memory leak in spa_load_best function 2016-08-19 11:17:12 -07:00
space_map.c Illumos 5960, 5925 2016-01-08 15:08:19 -08:00
space_reftree.c Change KM_PUSHPAGE -> KM_SLEEP 2015-01-16 14:41:26 -08:00
trace.c OpenZFS 6531 - Provide mechanism to artificially limit disk performance 2016-05-26 10:11:51 -07:00
txg.c txg visibility code should not execute under tc_open_lock 2016-07-27 14:11:13 -07:00
uberblock.c Illumos 5347 - idle pool may run itself out of space 2015-07-14 10:35:21 -07:00
unique.c Change KM_PUSHPAGE -> KM_SLEEP 2015-01-16 14:41:26 -08:00
vdev_cache.c Illumos 5045 - use atomic_{inc,dec}_* instead of atomic_add_* 2016-01-15 15:38:36 -08:00
vdev_disk.c Remove custom root pool import code 2016-08-11 11:19:34 -07:00
vdev_file.c OpenZFS 6531 - Provide mechanism to artificially limit disk performance 2016-05-26 10:11:51 -07:00
vdev_label.c Fix memleak in vdev_config_generate_stats 2016-05-31 16:05:21 -07:00
vdev_mirror.c FreeBSD r256956: Improve ZFS N-way mirror read performance by using load and locality information. 2016-02-26 11:24:35 -08:00
vdev_missing.c Illumos #5244 - zio pipeline callers should explicitly invoke next stage 2015-04-30 15:07:47 -07:00
vdev_queue.c Fix self-healing IO prior to dsl_pool_init() completion 2016-05-27 14:11:25 -07:00
vdev_raidz_math_avx2.c Fixes and enhancements of SIMD raidz parity 2016-07-19 16:43:07 -07:00
vdev_raidz_math_impl.h SIMD implementation of vdev_raidz generate and reconstruct routines 2016-06-21 09:27:26 -07:00
vdev_raidz_math_scalar.c Fixes and enhancements of SIMD raidz parity 2016-07-19 16:43:07 -07:00
vdev_raidz_math_sse2.c Add RAID-Z routines for SSE2 instruction set, in x86_64 mode. 2016-07-13 10:24:55 -07:00
vdev_raidz_math_ssse3.c Fixes and enhancements of SIMD raidz parity 2016-07-19 16:43:07 -07:00
vdev_raidz_math.c RAIDZ parity kstat rework 2016-07-19 16:43:07 -07:00
vdev_raidz.c Build user-space with different gcc optimization levels 2016-08-09 14:40:35 -07:00
vdev_root.c Illumos #3598 2013-10-31 14:58:04 -07:00
vdev.c Fix incorrect pool state after import 2016-08-12 13:46:51 -07:00
zap_leaf.c Illumos 5314 - Remove "dbuf phys" db->db_data pointer aliases in ZFS 2015-04-28 16:25:20 -07:00
zap_micro.c OpenZFS 7004 - dmu_tx_hold_zap() does dnode_hold() 7x on same object 2016-08-19 12:48:03 -07:00
zap.c OpenZFS 7004 - dmu_tx_hold_zap() does dnode_hold() 7x on same object 2016-08-19 12:48:03 -07:00
zfeature_common.c Implement large_dnode pool feature 2016-06-24 13:13:21 -07:00
zfeature.c Revert "zhack: Add 'feature disable' command" 2016-05-17 11:52:07 -07:00
zfs_acl.c Fix interaction between userns uid/gid and SA 2016-08-08 10:47:43 -07:00
zfs_byteswap.c Add linux kernel module support 2010-08-31 13:41:58 -07:00
zfs_ctldir.c Remove znode's z_uid/z_gid member 2016-07-25 13:21:49 -07:00
zfs_debug.c Add dbgmsg kstat 2015-09-04 16:08:14 -07:00
zfs_dir.c Remove znode's z_uid/z_gid member 2016-07-25 13:21:49 -07:00
zfs_fm.c OpenZFS 5997 - FRU field not set during pool creation and never updated 2016-08-12 13:06:48 -07:00
zfs_fuid.c Remove znode's z_uid/z_gid member 2016-07-25 13:21:49 -07:00
zfs_ioctl.c Fix zfs_allow_log_destroy() NULL dereference 2016-07-29 15:34:12 -07:00
zfs_log.c Remove znode's z_uid/z_gid member 2016-07-25 13:21:49 -07:00
zfs_onexit.c zfsdev_getminor() should check for invalid file handles 2015-06-22 17:02:13 -07:00
zfs_replay.c Implement large_dnode pool feature 2016-06-24 13:13:21 -07:00
zfs_rlock.c Remove dummy znode from zvol_state 2016-05-17 10:29:02 -07:00
zfs_sa.c Use native inode->i_nlink instead of znode->z_links 2016-07-14 16:25:34 -07:00
zfs_vfsops.c Remove znode's z_uid/z_gid member 2016-07-25 13:21:49 -07:00
zfs_vnops.c Fix interaction between userns uid/gid and SA 2016-08-08 10:47:43 -07:00
zfs_znode.c Move assignment of i_blkbits field 2016-07-29 15:34:12 -07:00
zil.c OpenZFS 6314 - buffer overflow in dsl_dataset_name 2016-06-28 13:47:03 -07:00
zio_checksum.c Illumos 4757, 4913 2014-08-01 14:28:05 -07:00
zio_compress.c Illumos 5661 - ZFS: "compression = on" should use lz4 if feature is enabled 2015-07-10 12:11:45 -07:00
zio_inject.c OpenZFS 6531 - Provide mechanism to artificially limit disk performance 2016-05-26 10:11:51 -07:00
zio.c Fix zdb crash with 4K-only devices 2016-07-27 13:38:46 -07:00
zle.c Update core ZFS code from build 121 to build 141. 2010-05-28 13:45:14 -07:00
zpl_ctldir.c Use file_dentry and file_inode wrappers 2016-08-11 12:06:37 -07:00
zpl_export.c zfsctl: No need to sync ctldir inodes 2015-08-31 13:54:39 -07:00
zpl_file.c Use file_dentry and file_inode wrappers 2016-08-11 12:06:37 -07:00
zpl_inode.c OpenZFS 6314 - buffer overflow in dsl_dataset_name 2016-06-28 13:47:03 -07:00
zpl_super.c Fix memleak in zpl_parse_options 2016-05-31 16:04:26 -07:00
zpl_xattr.c Linux 4.7 compat: fix zpl_get_acl returns invalid acl pointer 2016-08-09 10:03:04 -07:00
zrlock.c Illumos 5812 - assertion failed in zrl_tryenter(): zr_owner==NULL 2015-04-30 14:43:40 -07:00
zvol.c Linux 4.8 compat: Fix removal of bio->bi_rw member 2016-08-11 11:19:34 -07:00