d/freebsd-dev
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
FreeBSD src
5,817 Commits 72 Branches 135 Tags 3.2 GiB
C 59.6%
C++ 25.5%
Roff 5.1%
Shell 2.9%
Assembly 2.7%
Other 3.6%
7261fc2e81
Go to file
Matthew Ahrens 7261fc2e81
Improve zfs receive performance by batching writes 
For each WRITE record in the stream, `zfs receive` creates a DMU
transaction (`dmu_tx_create()`) and writes this block's data into the
object.  If per-block overheads (as opposed to per-byte overheads)
dominate performance (as is often the case with small recordsize), the
per-dmu-transaction overheads can be significant.  For example, in some
workloads the `receieve_writer` thread is 100% on CPU, and more than
half of its CPU time is in these per-tx routines (e.g.
dmu_tx_hold_write, dmu_tx_assign, dmu_tx_commit).

To improve performance of `zfs receive`, this commit batches WRITE
records which are to nearby offsets of the same object, and uses one DMU
transaction to write them all.  By default the batch size is 1MB, which
for recordsize=8K reduces the number of DMU transactions by 128x for
full send streams (incrementals will depend on how "clumpy" the changed
blocks are).

This commit improves the performance of `dd if=stream | zfs recv`
from 78,800 blocks/sec to 98,100 blocks/sec (25% improvement).

Reviewed-by: Paul Dagnelie <pcd@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Matthew Ahrens <mahrens@delphix.com>
Closes #10099
2020-03-16 11:51:56 -07:00
.github Add an .editorconfig; document git whitespace settings 2020-01-27 13:32:52 -08:00
cmd Add trim support to zpool wait 2020-03-04 15:07:11 -08:00
config Fix CONFIG_MODULES=no Linux kernel config 2020-02-28 09:23:48 -08:00
contrib Fix zfs-functions packaging bug 2020-03-10 09:53:20 -07:00
etc Fix zfs-functions packaging bug 2020-03-10 09:53:20 -07:00
include dmu_objset_from_ds must be called with dp_config_rwlock held 2020-03-12 10:55:02 -07:00
lib Change default to overlay=on 2020-03-06 09:28:19 -08:00
man Improve zfs receive performance by batching writes 2020-03-16 11:51:56 -07:00
module Improve zfs receive performance by batching writes 2020-03-16 11:51:56 -07:00
rpm Change http://zfsonlinux.org links to https://zfsonlinux.org 2020-01-13 16:43:59 -08:00
scripts zloop.sh should call ZDB with pool name 2020-03-11 10:02:23 -07:00
tests ZTS: Update flaky tests in zts-report 2020-03-13 09:29:10 -07:00
udev Create symbolic links in /dev/disk/by-vdev for nvme disk devices 2019-12-17 17:50:20 -08:00
.editorconfig Add an .editorconfig; document git whitespace settings 2020-01-27 13:32:52 -08:00
.gitignore Adapt gitignore for modules 2019-12-02 13:23:47 -08:00
.gitmodules Add zimport.sh compatibility test script 2014-02-21 12:10:31 -08:00
.travis.yml Add .travis.yml 2017-11-13 09:18:18 -08:00
AUTHORS Update build system and packaging 2018-05-29 16:00:33 -07:00
autogen.sh Cause autogen.sh to fail if autoreconf fails 2018-07-06 09:27:37 -07:00
CODE_OF_CONDUCT.md Add CODE_OF_CONDUCT.md 2019-04-30 10:58:45 -07:00
configure.ac Fix zfs-functions packaging bug 2020-03-10 09:53:20 -07:00
copy-builtin bash scripts: use /usr/bin/env for bash shebangs 2020-02-10 13:13:46 -08:00
COPYRIGHT ICP: Improve AES-GCM performance 2020-02-10 12:59:50 -08:00
LICENSE Update build system and packaging 2018-05-29 16:00:33 -07:00
Makefile.am Perform KABI checks in parallel 2019-10-01 12:50:34 -07:00
META Update maximum kernel version to 5.4 2019-12-23 14:24:36 -08:00
NEWS Add NEWS file 2018-09-18 12:03:47 -07:00
NOTICE Update build system and packaging 2018-05-29 16:00:33 -07:00
README.md Update README for OpenZFS 2020-02-25 11:43:20 -08:00
TEST Remove CI builder customization from TEST 2020-03-16 10:46:03 -07:00
zfs.release.in Move zfs.release generation to configure step 2012-07-12 12:22:51 -07:00

README.md

img

OpenZFS is an advanced file system and volume manager which was originally developed for Solaris and is now maintained by the OpenZFS community. This repository contains the code for running OpenZFS on Linux and FreeBSD.

codecov coverity

Official Resources

  • Wiki - for using and developing this repo
  • ZoL Site - Linux release info & links
  • Mailing lists
  • OpenZFS site - for conference videos and info on other platforms (illumos, OSX, Windows, etc)

Installation

Full documentation for installing OpenZFS on your favorite Linux distribution can be found at the ZoL Site.

FreeBSD support is a work in progress. See the PR.

Contribute & Develop

We have a separate document with contribution guidelines.

We have a Code of Conduct.

Release

OpenZFS is released under a CDDL license. For more details see the NOTICE, LICENSE and COPYRIGHT files; UCRL-CODE-235197

Supported Kernels

  • The META file contains the officially recognized supported Linux kernel versions.