d/freebsd-nq
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
aa755b3549
freebsd-nq/module/zfs
History
Matthew Ahrens aa755b3549
Set aside a metaslab for ZIL blocks 
Mixing ZIL and normal allocations has several problems:

1. The ZIL allocations are allocated, written to disk, and then a few
seconds later freed.  This leaves behind holes (free segments) where the
ZIL blocks used to be, which increases fragmentation, which negatively
impacts performance.

2. When under moderate load, ZIL allocations are of 128KB.  If the pool
is fairly fragmented, there may not be many free chunks of that size.
This causes ZFS to load more metaslabs to locate free segments of 128KB
or more.  The loading happens synchronously (from zil_commit()), and can
take around a second even if the metaslab's spacemap is cached in the
ARC.  All concurrent synchronous operations on this filesystem must wait
while the metaslab is loading.  This can cause a significant performance
impact.

3. If the pool is very fragmented, there may be zero free chunks of
128KB or more.  In this case, the ZIL falls back to txg_wait_synced(),
which has an enormous performance impact.

These problems can be eliminated by using a dedicated log device
("slog"), even one with the same performance characteristics as the
normal devices.

This change sets aside one metaslab from each top-level vdev that is
preferentially used for ZIL allocations (vdev_log_mg,
spa_embedded_log_class).  From an allocation perspective, this is
similar to having a dedicated log device, and it eliminates the
above-mentioned performance problems.

Log (ZIL) blocks can be allocated from the following locations.  Each
one is tried in order until the allocation succeeds:
1. dedicated log vdevs, aka "slog" (spa_log_class)
2. embedded slog metaslabs (spa_embedded_log_class)
3. other metaslabs in normal vdevs (spa_normal_class)

The space required for the embedded slog metaslabs is usually between
0.5% and 1.0% of the pool, and comes out of the existing 3.2% of "slop"
space that is not available for user data.

On an all-ssd system with 4TB storage, 87% fragmentation, 60% capacity,
and recordsize=8k, testing shows a ~50% performance increase on random
8k sync writes.  On even more fragmented systems (which hit problem #3
above and call txg_wait_synced()), the performance improvement can be
arbitrarily large (>100x).

Reviewed-by: Serapheim Dimitropoulos <serapheim@delphix.com>
Reviewed-by: George Wilson <gwilson@delphix.com>
Reviewed-by: Don Brady <don.brady@delphix.com>
Reviewed-by: Mark Maybee <mark.maybee@delphix.com>
Signed-off-by: Matthew Ahrens <mahrens@delphix.com>
Closes #11389
2021-01-21 15:12:54 -08:00
..
abd.c allow callers to allocate and provide the abd_t struct 2021-01-20 11:24:37 -08:00
aggsum.c Implement memory and CPU hotplug 2020-12-10 14:09:23 -08:00
arc.c allow callers to allocate and provide the abd_t struct 2021-01-20 11:24:37 -08:00
blkptr.c Add zstd support to zfs 2020-08-20 10:30:06 -07:00
bplist.c Fast Clone Deletion 2019-07-26 10:54:14 -07:00
bpobj.c Rename refcount.h to zfs_refcount.h 2020-07-29 16:35:33 -07:00
bptree.c Rename refcount.h to zfs_refcount.h 2020-07-29 16:35:33 -07:00
bqueue.c Implement Redacted Send/Receive 2019-06-19 09:48:12 -07:00
btree.c Fix typo in btree.c 2020-08-17 15:25:37 -07:00
dataset_kstats.c Fix panic on DilOS with kstat per dataset statistics 2019-09-03 12:12:31 -07:00
dbuf_stats.c Eliminate gratuitous bzeroing in dbuf_stats_hash_table_data 2020-09-30 13:24:38 -07:00
dbuf.c allow callers to allocate and provide the abd_t struct 2021-01-20 11:24:37 -08:00
ddt_zap.c Refactor dnode dirty context from dbuf_dirty 2020-02-26 16:09:17 -08:00
ddt.c Remove dead code 2020-06-18 12:21:18 -07:00
dmu_diff.c Mark write_record static 2019-12-03 09:51:44 -08:00
dmu_object.c Introduce CPU_SEQID_UNSTABLE 2020-11-02 11:51:12 -08:00
dmu_objset.c implicit conversion from 'boolean_t' to 'ds_hold_flags_t' 2020-12-27 16:31:02 -08:00
dmu_recv.c implicit conversion from 'boolean_t' to 'ds_hold_flags_t' 2020-12-27 16:31:02 -08:00
dmu_redact.c Fix dnode refcount tracking 2020-11-10 10:37:10 -08:00
dmu_send.c implicit conversion from 'boolean_t' to 'ds_hold_flags_t' 2020-12-27 16:31:02 -08:00
dmu_traverse.c zil_parse: make callback parameters const 2020-10-09 09:34:54 -07:00
dmu_tx.c Remove unused check from dmu_tx_count_write() 2020-12-21 20:17:13 -08:00
dmu_zfetch.c dmu_zfetch: don't leak unreferenced stream when zfetch is freed 2020-10-13 21:03:36 -07:00
dmu.c Extending FreeBSD UIO Struct 2021-01-20 21:27:30 -08:00
dnode_sync.c Improve zfs receive performance with lightweight write 2020-12-11 10:26:02 -08:00
dnode.c Improve zfs receive performance with lightweight write 2020-12-11 10:26:02 -08:00
dsl_bookmark.c Fix kernel panic induced by redacted send 2020-12-11 10:22:29 -08:00
dsl_crypt.c Fix raw sends on encrypted datasets when copying back snapshots 2020-12-04 14:34:29 -08:00
dsl_dataset.c Improve zfs receive performance with lightweight write 2020-12-11 10:26:02 -08:00
dsl_deadlist.c Fix i/o error handling of livelists and zap iteration 2020-08-05 10:22:09 -07:00
dsl_deleg.c Reduce loaded range tree memory usage 2019-10-09 10:36:03 -07:00
dsl_destroy.c Fix i/o error handling of livelists and zap iteration 2020-08-05 10:22:09 -07:00
dsl_dir.c Add 'zfs rename -u' to rename without remounting 2020-09-01 16:14:16 -07:00
dsl_pool.c dsl_pool: extend comment on DSL Pool Configuration Lock 2020-12-19 18:04:05 -08:00
dsl_prop.c Replace sprintf()->snprintf() and strcpy()->strlcpy() 2020-06-07 11:42:12 -07:00
dsl_scan.c Distributed Spare (dRAID) Feature 2020-11-13 13:51:51 -08:00
dsl_synctask.c nowait synctask must succeed 2020-09-04 10:29:39 -07:00
dsl_userhold.c Replace sprintf()->snprintf() and strcpy()->strlcpy() 2020-06-07 11:42:12 -07:00
edonr_zfs.c Add include files for prototypes 2020-06-18 12:21:25 -07:00
fm.c Avoid posting duplicate zpool events 2020-09-04 10:34:28 -07:00
gzip.c Add include files for prototypes 2020-06-18 12:21:25 -07:00
hkdf.c Encryption patch follow-up 2017-10-11 16:54:48 -04:00
lz4.c Prefix zfs internal endian checks with _ZFS 2020-07-28 13:02:49 -07:00
lzjb.c Add include files for prototypes 2020-06-18 12:21:25 -07:00
Makefile.in Distributed Spare (dRAID) Feature 2020-11-13 13:51:51 -08:00
metaslab.c Set aside a metaslab for ZIL blocks 2021-01-21 15:12:54 -08:00
mmp.c Distributed Spare (dRAID) Feature 2020-11-13 13:51:51 -08:00
multilist.c Implement memory and CPU hotplug 2020-12-10 14:09:23 -08:00
objlist.c Implement Redacted Send/Receive 2019-06-19 09:48:12 -07:00
pathname.c Replace ZFS on Linux references with OpenZFS 2020-10-08 20:10:13 -07:00
range_tree.c Fix incorrect deletion order in range_tree_add_impl gap case 2020-10-14 08:59:54 -07:00
refcount.c Rename refcount.h to zfs_refcount.h 2020-07-29 16:35:33 -07:00
rrwlock.c Rename refcount.h to zfs_refcount.h 2020-07-29 16:35:33 -07:00
sa.c Extending FreeBSD UIO Struct 2021-01-20 21:27:30 -08:00
sha256.c Add include files for prototypes 2020-06-18 12:21:25 -07:00
skein_zfs.c Add include files for prototypes 2020-06-18 12:21:25 -07:00
spa_boot.c Add include files for prototypes 2020-06-18 12:21:25 -07:00
spa_checkpoint.c Refactor dnode dirty context from dbuf_dirty 2020-02-26 16:09:17 -08:00
spa_config.c vdev_ashift should only be set once 2020-09-18 12:13:47 -07:00
spa_errlog.c Fix typos in module/zfs/ 2019-09-02 17:56:41 -07:00
spa_history.c record ioctl elapsed time in zpool history 2021-01-11 09:29:25 -08:00
spa_log_spacemap.c Make module tunables cross platform 2019-09-05 14:49:49 -07:00
spa_misc.c Set aside a metaslab for ZIL blocks 2021-01-21 15:12:54 -08:00
spa_stats.c FreeBSD: Add support for procfs_list 2020-09-23 16:43:51 -07:00
spa.c Set aside a metaslab for ZIL blocks 2021-01-21 15:12:54 -08:00
space_map.c Rename refcount.h to zfs_refcount.h 2020-07-29 16:35:33 -07:00
space_reftree.c Reduce loaded range tree memory usage 2019-10-09 10:36:03 -07:00
THIRDPARTYLICENSE.cityhash OpenZFS 8484 - Implement aggregate sum and use for arc counters 2018-06-06 09:35:59 -07:00
THIRDPARTYLICENSE.cityhash.descrip OpenZFS 8484 - Implement aggregate sum and use for arc counters 2018-06-06 09:35:59 -07:00
txg.c Implement memory and CPU hotplug 2020-12-10 14:09:23 -08:00
uberblock.c MMP interval and fail_intervals in uberblock 2019-03-21 12:47:57 -07:00
unique.c Reduce loaded range tree memory usage 2019-10-09 10:36:03 -07:00
vdev_cache.c Replace ASSERTV macro with compiler annotation 2019-12-05 12:37:00 -08:00
vdev_draid_rand.c Distributed Spare (dRAID) Feature 2020-11-13 13:51:51 -08:00
vdev_draid.c allow callers to allocate and provide the abd_t struct 2021-01-20 11:24:37 -08:00
vdev_indirect_births.c Fixes: #8934 Large kmem_alloc 2019-07-10 15:54:49 -07:00
vdev_indirect_mapping.c Replace ASSERTV macro with compiler annotation 2019-12-05 12:37:00 -08:00
vdev_indirect.c allow callers to allocate and provide the abd_t struct 2021-01-20 11:24:37 -08:00
vdev_initialize.c Distributed Spare (dRAID) Feature 2020-11-13 13:51:51 -08:00
vdev_label.c Distributed Spare (dRAID) Feature 2020-11-13 13:51:51 -08:00
vdev_mirror.c Distributed Spare (dRAID) Feature 2020-11-13 13:51:51 -08:00
vdev_missing.c Distributed Spare (dRAID) Feature 2020-11-13 13:51:51 -08:00
vdev_queue.c allow callers to allocate and provide the abd_t struct 2021-01-20 11:24:37 -08:00
vdev_raidz_math_aarch64_neon_common.h FreeBSD: fix the build with Clang 11 2020-08-17 15:40:17 -07:00
vdev_raidz_math_aarch64_neon.c Linux 5.0 compat: SIMD compatibility 2019-07-12 09:31:20 -07:00
vdev_raidz_math_aarch64_neonx2.c Linux 5.0 compat: SIMD compatibility 2019-07-12 09:31:20 -07:00
vdev_raidz_math_avx2.c FreeBSD: fix the build with Clang 11 2020-08-17 15:40:17 -07:00
vdev_raidz_math_avx512bw.c Refactor ccompile.h to not include system headers 2020-07-25 20:09:50 -07:00
vdev_raidz_math_avx512f.c FreeBSD: fix the build with Clang 11 2020-08-17 15:40:17 -07:00
vdev_raidz_math_impl.h Distributed Spare (dRAID) Feature 2020-11-13 13:51:51 -08:00
vdev_raidz_math_powerpc_altivec_common.h FreeBSD: fix the build with Clang 11 2020-08-17 15:40:17 -07:00
vdev_raidz_math_powerpc_altivec.c Prefix zfs internal endian checks with _ZFS 2020-07-28 13:02:49 -07:00
vdev_raidz_math_scalar.c Linux 5.3: Fix switch() fall though compiler errors 2019-08-21 09:29:23 -07:00
vdev_raidz_math_sse2.c FreeBSD: fix the build with Clang 11 2020-08-17 15:40:17 -07:00
vdev_raidz_math_ssse3.c Refactor ccompile.h to not include system headers 2020-07-25 20:09:50 -07:00
vdev_raidz_math.c Reduce fletcher4 and raidz benchmark times 2020-12-06 09:57:20 -08:00
vdev_raidz.c allow callers to allocate and provide the abd_t struct 2021-01-20 11:24:37 -08:00
vdev_rebuild.c Distributed Spare (dRAID) Feature 2020-11-13 13:51:51 -08:00
vdev_removal.c Set aside a metaslab for ZIL blocks 2021-01-21 15:12:54 -08:00
vdev_root.c Distributed Spare (dRAID) Feature 2020-11-13 13:51:51 -08:00
vdev_trim.c Distributed Spare (dRAID) Feature 2020-11-13 13:51:51 -08:00
vdev.c Set aside a metaslab for ZIL blocks 2021-01-21 15:12:54 -08:00
zap_leaf.c Refactor dnode dirty context from dbuf_dirty 2020-02-26 16:09:17 -08:00
zap_micro.c Rename refcount.h to zfs_refcount.h 2020-07-29 16:35:33 -07:00
zap.c Rename refcount.h to zfs_refcount.h 2020-07-29 16:35:33 -07:00
zcp_get.c Add include files for prototypes 2020-06-18 12:21:25 -07:00
zcp_global.c OpenZFS 8600 - ZFS channel programs - snapshot 2018-02-08 15:29:24 -08:00
zcp_iter.c Fix typos in module/zfs/ 2019-09-02 17:56:41 -07:00
zcp_set.c Support setting user properties in a channel program 2020-02-14 13:41:42 -08:00
zcp_synctask.c filesystem_limit/snapshot_limit is incorrectly enforced against root 2020-07-11 17:18:02 -07:00
zcp.c Channel program may spuriously fail with "memory limit exhausted" 2020-11-11 17:16:15 -08:00
zfeature.c Throw const on some strings 2020-10-02 17:44:10 -07:00
zfs_byteswap.c Mark functions as static 2020-06-18 12:20:38 -07:00
zfs_fm.c Distributed Spare (dRAID) Feature 2020-11-13 13:51:51 -08:00
zfs_fuid.c FreeBSD: Fix UNIX permissions checking 2020-08-18 09:57:07 -07:00
zfs_ioctl.c record ioctl elapsed time in zpool history 2021-01-11 09:29:25 -08:00
zfs_log.c Throw const on some strings 2020-10-02 17:44:10 -07:00
zfs_onexit.c Remove deduplicated send/receive code 2020-04-23 10:06:57 -07:00
zfs_quota.c File incorrectly zeroed when receiving incremental stream that toggles -L 2020-06-09 10:41:01 -07:00
zfs_ratelimit.c Change checksum & IO delay ratelimit values 2018-03-04 17:34:51 -08:00
zfs_replay.c Simplify FreeBSD's locking requirements in zfs_replay.c 2020-01-22 17:55:56 -08:00
zfs_rlock.c Add a "try" operation for range locks 2020-07-06 11:53:31 -07:00
zfs_sa.c Extending FreeBSD UIO Struct 2021-01-20 21:27:30 -08:00
zfs_vnops.c Extending FreeBSD UIO Struct 2021-01-20 21:27:30 -08:00
zil.c allow callers to allocate and provide the abd_t struct 2021-01-20 11:24:37 -08:00
zio_checksum.c Mark functions as static 2020-06-18 12:20:38 -07:00
zio_compress.c Avoid symbol collision with in-kernel zstdlib 2020-08-24 12:20:41 -07:00
zio_inject.c Distributed Spare (dRAID) Feature 2020-11-13 13:51:51 -08:00
zio.c Set aside a metaslab for ZIL blocks 2021-01-21 15:12:54 -08:00
zle.c Add include files for prototypes 2020-06-18 12:21:25 -07:00
zrlock.c Remove dead code 2020-06-18 12:21:18 -07:00
zthr.c Retain thread name when resuming a zthr 2020-09-03 20:09:52 -07:00
zvol.c Fix problems in zvol_set_volmode_impl 2020-11-17 09:50:52 -08:00