freebsd-nq

freebsd with flexible iflib nic queues

Go to file

Andriy Gapon c13f1d82c8 MFV r323535: 8585 improve batching done in zil_commit() FreeBSD notes: - this MFV reverts FreeBSD commit r314549 to make the merge easier - at present our emulation of cv_timedwait_hires is rather poor, so I elected to use cv_timedwait_sbt directly Please see the differential revision for details. Unfortunately, I did not get any positive reviews, so there could be bugs in the FreeBSD-specific piece of the merge. Hence, the long MFC timeout. illumos/illumos-gate@1271e4b10d `1271e4b10d` https://www.illumos.org/issues/8585 The current implementation of zil_commit() can introduce significant latency, beyond what is inherent due to the latency of the underlying storage. The additional latency comes from two main problems: 1. When there's outstanding ZIL blocks being written (i.e. there's already a "writer thread" in progress), then any new calls to zil_commit() will block waiting for the currently oustanding ZIL blocks to complete. The blocks written for each "writer thread" is coined a "batch", and there can only ever be a single "batch" being written at a time. When a batch is being written, any new ZIL transactions will have to wait for the next batch to be written, which won't occur until the current batch finishes. As a result, the underlying storage may not be used as efficiently as possible. While "new" threads enter zil_commit() and are blocked waiting for the next batch, it's possible that the underlying storage isn't fully utilized by the current batch of ZIL blocks. In that case, it'd be better to allow these new threads to generate (and issue) a new ZIL block, such that it could be serviced by the underlying storage concurrently with the other ZIL blocks that are being serviced. 2. Any call to zil_commit() must wait for all ZIL blocks in its "batch" to complete, prior to zil_commit() returning. The size of any given batch is proportional to the number of ZIL transaction in the queue at the time that the batch starts processing the queue; which doesn't occur until the previous batch completes. Thus, if there's a lot of transactions in the queue, the batch could be composed of many ZIL blocks, and each call to zil_commit() will have to wait for all of these writes to complete (even if the thread calling zil_commit() only cared about one of the transactions in the batch). Reviewed by: Brad Lewis <brad.lewis@delphix.com> Reviewed by: Matt Ahrens <mahrens@delphix.com> Reviewed by: George Wilson <george.wilson@delphix.com> Approved by: Dan McDonald <danmcd@joyent.com> Author: Prakash Surya <prakash.surya@delphix.com> MFC after: 1 month Differential Revision: https://reviews.freebsd.org/D12355		2017-09-26 11:04:08 +00:00
bin	Ammend bin/cat/cat.c so the output is the same aside	2017-09-21 14:14:49 +00:00
cddl	MFV r323535: 8585 improve batching done in zil_commit()	2017-09-26 11:04:08 +00:00
contrib	Fix the return value from _Unwind_Backtrace() on arm.	2017-09-25 23:50:10 +00:00
crypto	Upgrade to OpenSSH 7.5p1.	2017-08-04 12:57:24 +00:00
etc	Do not actually install uneeded alias for man	2017-09-26 05:46:10 +00:00
gnu	Remove redundant source and object files.	2017-09-09 13:18:32 +00:00
include	An MMC/SD/SDIO stack using CAM	2017-07-09 16:57:24 +00:00
kerberos5	Replace the include path using CURDIR with KRB5DIR. This is consistent	2017-08-21 12:42:05 +00:00
lib	capsicum_helpers: Add SEEK to default stdio rights set	2017-09-25 19:33:32 +00:00
libexec	Deorbit catman. The tradeoff of disk for performance has long since tipped	2017-09-13 16:35:16 +00:00
release	Revert r323812 from release/tools/arm.subr, which has broken the	2017-09-22 14:34:27 +00:00
rescue	DIRDEPS_BUILD: Update dependencies.	2017-05-09 01:48:23 +00:00
sbin	Continuing efforts to provide hardening of FFS, this change adds a	2017-09-22 12:45:15 +00:00
secure	Convert traditional ${MK_TESTS} conditional idiom for including test	2017-08-02 08:35:51 +00:00
share	Add the ability to report and set debug flags as text strings instead of	2017-09-24 13:14:50 +00:00
sys	MFV r323535: 8585 improve batching done in zil_commit()	2017-09-26 11:04:08 +00:00
targets	Deorbit catman. The tradeoff of disk for performance has long since tipped	2017-09-13 16:35:16 +00:00
tests	crypto(9): Use a more specific error code when a capable driver is not found	2017-09-26 01:31:49 +00:00
tools	Fix packages with interactive post install scripts.	2017-09-24 14:22:36 +00:00
usr.bin	Add myself to the calendar.freebsd	2017-09-24 14:36:01 +00:00
usr.sbin	mountd: Replace malloc+strcpy to strdup	2017-09-26 09:18:18 +00:00
.arcconfig	callsign isn't required anymore	2016-09-29 06:19:45 +00:00
.arclint
COPYRIGHT	Bump copyright year.	2016-12-31 12:41:42 +00:00
LOCKS	Explicitly require Security Officer's approval for kernel PRNG bits.	2013-09-17 14:19:05 +00:00
MAINTAINERS	Add asomers as a pjdfstest co-maintainer per the project status change	2017-06-28 09:25:15 +00:00
Makefile	Quote ${MAKE} when passing in env in case it contains spaces.	2017-08-16 17:54:24 +00:00
Makefile.inc1	Do not print error when running make delete-old on system	2017-09-26 05:33:15 +00:00
Makefile.libcompat	Honor NO_RTLD for rtld-elf, similar to what's done in libexec/Makefile, with	2017-08-17 20:15:20 +00:00
ObsoleteFiles.inc	Remove empty lines for consistency with other entries	2017-09-26 05:47:33 +00:00
README	Correct the URL to instructions for updated system sources.	2017-05-15 15:18:36 +00:00
README.md	Remove hard line breaks from README.md	2017-03-01 15:39:58 +00:00
UPDATING	Fix language used in the r322923.	2017-09-17 10:28:45 +00:00

README.md

FreeBSD Source:

This is the top level of the FreeBSD source directory. This file was last revised on: FreeBSD

For copyright information, please see the file COPYRIGHT in this directory (additional copyright information also exists for some sources in this tree - please see the specific source directories for more information).

The Makefile in this directory supports a number of targets for building components (or all) of the FreeBSD source tree. See build(7) and http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/makeworld.html for more information, including setting make(1) variables.

The buildkernel and installkernel targets build and install the kernel and the modules (see below). Please see the top of the Makefile in this directory for more information on the standard build targets and compile-time flags.

Building a kernel is a somewhat more involved process. See build(7), config(8), and http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/kernelconfig.html for more information.

Note: If you want to build and install the kernel with the buildkernel and installkernel targets, you might need to build world before. More information is available in the handbook.

The kernel configuration files reside in the sys/<arch>/conf sub-directory. GENERIC is the default configuration used in release builds. NOTES contains entries and documentation for all possible devices, not just those commonly used.

Source Roadmap:

bin				System/user commands.

cddl			Various commands and libraries under the Common Development  
				and Distribution License.

contrib			Packages contributed by 3rd parties.

crypto			Cryptography stuff (see crypto/README).

etc				Template files for /etc.

gnu				Various commands and libraries under the GNU Public License.  
				Please see gnu/COPYING* for more information.

include			System include files.

kerberos5		Kerberos5 (Heimdal) package.

lib				System libraries.

libexec			System daemons.

release			Release building Makefile & associated tools.

rescue			Build system for statically linked /rescue utilities.

sbin			System commands.

secure			Cryptographic libraries and commands.

share			Shared resources.

sys				Kernel sources.

tests			Regression tests which can be run by Kyua.  See tests/README
				for additional information.

tools			Utilities for regression testing and miscellaneous tasks.

usr.bin			User commands.

usr.sbin		System administration commands.

For information on synchronizing your source tree with one or more of the FreeBSD Project's development branches, please see:

http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/synching.html