d/freebsd-nq
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
85ec5cbae2
freebsd-nq/include/sys
History
Matthew Ahrens 85ec5cbae2
Include scatter_chunk_waste in arc_size 
The ARC caches data in scatter ABD's, which are collections of pages,
which are typically 4K.  Therefore, the space used to cache each block
is rounded up to a multiple of 4K.  The ABD subsystem tracks this wasted
memory in the `scatter_chunk_waste` kstat.  However, the ARC's `size` is
not aware of the memory used by this round-up, it only accounts for the
size that it requested from the ABD subsystem.

Therefore, the ARC is effectively using more memory than it is aware of,
due to the `scatter_chunk_waste`.  This impacts observability, e.g.
`arcstat` will show that the ARC is using less memory than it
effectively is.  It also impacts how the ARC responds to memory
pressure.  As the amount of `scatter_chunk_waste` changes, it appears to
the ARC as memory pressure, so it needs to resize `arc_c`.

If the sector size (`1<<ashift`) is the same as the page size (or
larger), there won't be any waste.  If the (compressed) block size is
relatively large compared to the page size, the amount of
`scatter_chunk_waste` will be small, so the problematic effects are
minimal.

However, if using 512B sectors (`ashift=9`), and the (compressed) block
size is small (e.g. `compression=on` with the default `volblocksize=8k`
or a decreased `recordsize`), the amount of `scatter_chunk_waste` can be
very large.  On a production system, with `arc_size` at a constant 50%
of memory, `scatter_chunk_waste` has been been observed to be 10-30% of
memory.

This commit adds `scatter_chunk_waste` to `arc_size`, and adds a new
`waste` field to `arcstat`.  As a result, the ARC's memory usage is more
observable, and `arc_c` does not need to be adjusted as frequently.

Reviewed-by: Pavel Zakharov <pavel.zakharov@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: George Wilson <gwilson@delphix.com>
Reviewed-by: Ryan Moeller <ryan@iXsystems.com>
Signed-off-by: Matthew Ahrens <mahrens@delphix.com>
Closes #10701
2020-08-17 20:04:04 -07:00
..
crypto Avoid installing kernel headers on FreeBSD 2020-06-27 17:40:14 -07:00
fm Avoid installing kernel headers on FreeBSD 2020-06-27 17:40:14 -07:00
fs FreeBSD: fallback to /boot/ to look for zpool.cache 2020-08-17 14:43:47 -07:00
lua Avoid installing kernel headers on FreeBSD 2020-06-27 17:40:14 -07:00
sysevent Avoid installing kernel headers on FreeBSD 2020-06-27 17:40:14 -07:00
abd_impl.h Removing ZERO_PAGE abd_alloc_zero_scatter 2020-06-10 17:54:11 -07:00
abd.h Rename refcount.h to zfs_refcount.h 2020-07-29 16:35:33 -07:00
aggsum.h Reduce number of atomic_add() calls in aggsum 2020-02-06 13:21:06 -08:00
arc_impl.h Include scatter_chunk_waste in arc_size 2020-08-17 20:04:04 -07:00
arc.h Include scatter_chunk_waste in arc_size 2020-08-17 20:04:04 -07:00
avl_impl.h
avl.h Restore avl_update() calls and related functions 2020-06-03 09:49:32 -07:00
bitops.h Reduce loaded range tree memory usage 2019-10-09 10:36:03 -07:00
blkptr.h
bplist.h Fast Clone Deletion 2019-07-26 10:54:14 -07:00
bpobj.h Fast Clone Deletion 2019-07-26 10:54:14 -07:00
bptree.h
bqueue.h Implement Redacted Send/Receive 2019-06-19 09:48:12 -07:00
btree.h Fix typos 2020-06-09 21:24:09 -07:00
dataset_kstats.h port async unlinked drain from illumos-nexenta 2019-02-12 10:41:15 -08:00
dbuf.h Rename refcount.h to zfs_refcount.h 2020-07-29 16:35:33 -07:00
ddt.h Fix gcc10.1 truncation error 2020-06-13 11:02:00 -07:00
dmu_impl.h Prevent race condition in dnode_dest (#10101) 2020-03-12 10:25:56 -07:00
dmu_objset.h File incorrectly zeroed when receiving incremental stream that toggles -L 2020-06-09 10:41:01 -07:00
dmu_recv.h filesystem_limit/snapshot_limit is incorrectly enforced against root 2020-07-11 17:18:02 -07:00
dmu_redact.h Implement Redacted Send/Receive 2019-06-19 09:48:12 -07:00
dmu_send.h Add 'zfs send --saved' flag 2020-01-10 10:16:58 -08:00
dmu_traverse.h Implement Redacted Send/Receive 2019-06-19 09:48:12 -07:00
dmu_tx.h Rename refcount.h to zfs_refcount.h 2020-07-29 16:35:33 -07:00
dmu_zfetch.h Replace zf_rwlock with a mutex 2019-07-25 11:57:58 -07:00
dmu.h Annotate unused parameters on inline definitions as such 2020-07-23 17:41:48 -07:00
dnode.h Rename refcount.h to zfs_refcount.h 2020-07-29 16:35:33 -07:00
dsl_bookmark.h Rename refcount.h to zfs_refcount.h 2020-07-29 16:35:33 -07:00
dsl_crypt.h dmu_objset_from_ds must be called with dp_config_rwlock held 2020-03-12 10:55:02 -07:00
dsl_dataset.h Rename refcount.h to zfs_refcount.h 2020-07-29 16:35:33 -07:00
dsl_deadlist.h Add fast path for zfs_ioc_space_snaps() handling of empty_bpobj 2019-08-20 11:34:52 -07:00
dsl_deleg.h Remove code for zfs remap 2019-06-24 16:44:01 -07:00
dsl_destroy.h Fast Clone Deletion 2019-07-26 10:54:14 -07:00
dsl_dir.h Rename refcount.h to zfs_refcount.h 2020-07-29 16:35:33 -07:00
dsl_pool.h Eliminate Linux specific inode usage from common code 2019-12-11 11:53:57 -08:00
dsl_prop.h Support inheriting properties in channel programs 2020-01-22 17:03:17 -08:00
dsl_scan.h Add device rebuild feature 2020-07-03 11:05:50 -07:00
dsl_synctask.h OpenZFS 9425 - channel programs can be interrupted 2019-06-22 16:51:46 -07:00
dsl_userhold.h
edonr.h
efi_partition.h Fix typos in include/ 2019-08-30 09:53:15 -07:00
frame.h
hkdf.h
Makefile.am Rename refcount.h to zfs_refcount.h 2020-07-29 16:35:33 -07:00
metaslab_impl.h Use a struct to organize metaslab-group-allocator fields 2020-04-22 10:26:56 -07:00
metaslab.h Extend zdb to print inconsistencies in livelists and metaslabs 2020-07-14 17:51:05 -07:00
mmp.h Add zfs_multihost_interval tunable handler for FreeBSD 2020-06-23 13:32:42 -07:00
mntent.h Add FreeBSD required defines to mntent.h 2019-11-30 15:49:09 -08:00
mod.h Wrap Linux module macros 2019-11-01 10:41:03 -07:00
multilist.h Avoid extra taskq_dispatch() calls by DMU 2019-06-25 12:03:38 -07:00
note.h
nvpair_impl.h
nvpair.h
objlist.h Implement Redacted Send/Receive 2019-06-19 09:48:12 -07:00
pathname.h Disable unused pathname::pn_path* (unneeded in Linux) 2019-07-15 13:57:56 -07:00
qat.h QAT related bug fixes 2019-09-12 13:33:44 -07:00
range_tree.h Improve compatibility with C++ consumers 2020-06-06 12:54:04 -07:00
rrwlock.h Rename refcount.h to zfs_refcount.h 2020-07-29 16:35:33 -07:00
sa_impl.h Rename refcount.h to zfs_refcount.h 2020-07-29 16:35:33 -07:00
sa.h Fix typos in include/ 2019-08-30 09:53:15 -07:00
skein.h
spa_boot.h
spa_checkpoint.h
spa_checksum.h
spa_impl.h Rename refcount.h to zfs_refcount.h 2020-07-29 16:35:33 -07:00
spa_log_spacemap.h Log Spacemap Project 2019-07-16 10:11:49 -07:00
spa.h Prefix zfs internal endian checks with _ZFS 2020-07-28 13:02:49 -07:00
space_map.h Extend zdb to print inconsistencies in livelists and metaslabs 2020-07-14 17:51:05 -07:00
space_reftree.h Reduce loaded range tree memory usage 2019-10-09 10:36:03 -07:00
sysevent.h
txg_impl.h Fix typos in include/ 2019-08-30 09:53:15 -07:00
txg.h OpenZFS 9425 - channel programs can be interrupted 2019-06-22 16:51:46 -07:00
u8_textprep_data.h
u8_textprep.h
uberblock_impl.h MMP interval and fail_intervals in uberblock 2019-03-21 12:47:57 -07:00
uberblock.h
uio_impl.h
unique.h
uuid.h
vdev_disk.h Make struct vdev_disk_t be platform private 2020-06-16 11:43:33 -07:00
vdev_file.h Add zfs_file_* interface, remove vnodes 2019-11-21 09:32:57 -08:00
vdev_impl.h Add device rebuild feature 2020-07-03 11:05:50 -07:00
vdev_indirect_births.h
vdev_indirect_mapping.h
vdev_initialize.h Add TRIM support 2019-03-29 09:13:20 -07:00
vdev_raidz_impl.h Add prototypes 2020-06-18 12:21:32 -07:00
vdev_raidz.h Linux 5.0 compat: SIMD compatibility 2019-07-12 09:31:20 -07:00
vdev_rebuild.h Add device rebuild feature 2020-07-03 11:05:50 -07:00
vdev_removal.h panic in removal_remap test on 4K devices 2019-06-13 13:12:39 -07:00
vdev_trim.h Trim L2ARC 2020-06-09 10:15:08 -07:00
vdev.h Add device rebuild feature 2020-07-03 11:05:50 -07:00
xvattr.h
zap_impl.h
zap_leaf.h
zap.h fat zap should prefetch when iterating 2019-06-12 13:13:09 -07:00
zcp_global.h
zcp_iter.h
zcp_prop.h
zcp_set.h Support setting user properties in a channel program 2020-02-14 13:41:42 -08:00
zcp.h filesystem_limit/snapshot_limit is incorrectly enforced against root 2020-07-11 17:18:02 -07:00
zfeature.h
zfs_acl.h Return an error code from zfs_acl_chmod_setattr 2019-11-01 10:19:11 -07:00
zfs_context.h Remove KMC_KMEM and KMC_VMEM 2020-08-17 16:04:28 -07:00
zfs_debug.h Remove sdt.h 2019-10-25 13:38:37 -07:00
zfs_delay.h
zfs_file.h Re-share zfsdev_getminor and zfs_onexit_fd_hold 2020-02-28 14:50:32 -08:00
zfs_fuid.h Replace sprintf()->snprintf() and strcpy()->strlcpy() 2020-06-07 11:42:12 -07:00
zfs_ioctl_impl.h Restore support for in-kernel ZFS ioctls 2020-06-08 13:57:22 -07:00
zfs_ioctl.h drr_begin: can't forward declare untagged struct 2020-06-16 11:57:04 -07:00
zfs_onexit.h Remove deduplicated send/receive code 2020-04-23 10:06:57 -07:00
zfs_project.h Minor diff reduction with ZoF in include/sys 2019-11-27 11:11:03 -08:00
zfs_quota.h File incorrectly zeroed when receiving incremental stream that toggles -L 2020-06-09 10:41:01 -07:00
zfs_ratelimit.h
zfs_refcount.h Rename refcount.h to zfs_refcount.h 2020-07-29 16:35:33 -07:00
zfs_rlock.h Add a "try" operation for range locks 2020-07-06 11:53:31 -07:00
zfs_sa.h
zfs_stat.h
zfs_sysfs.h
zfs_znode.h Fix typos 2020-06-09 21:24:09 -07:00
zil_impl.h make zil max block size tunable 2019-06-10 11:48:42 -07:00
zil.h Add prototypes 2020-06-18 12:21:32 -07:00
zio_checksum.h
zio_compress.h lz4_decompress_abd declared but not defined 2019-06-13 13:14:34 -07:00
zio_crypt.h Rename refcount.h to zfs_refcount.h 2020-07-29 16:35:33 -07:00
zio_impl.h Fix typos in include/ 2019-08-30 09:53:15 -07:00
zio_priority.h Add device rebuild feature 2020-07-03 11:05:50 -07:00
zio.h Improve compatibility with C++ consumers 2020-06-06 12:54:04 -07:00
zrlock.h
zthr.h Introduce names for ZTHRs 2020-07-29 09:43:33 -07:00
zvol_impl.h Connect dataset_kstats for FreeBSD 2020-06-05 17:17:02 -07:00
zvol.h async zvol minor node creation interferes with receive 2020-02-03 09:33:14 -08:00