freebsd with flexible iflib nic queues
Go to file
Konstantin Belousov 698f05ab95 Mitigate several problems with the softdep_request_cleanup() on busy
host.

Problems start appearing when there are several threads all doing
operations on a UFS volume and the SU workqueue needs a cleanup.  It is
possible that each thread calling softdep_request_cleanup() owns the
lock for some dirty vnode (e.g. all of them are executing mkdir(2),
mknod(2), creat(2) etc) and all vnodes which must be flushed are locked
by corresponding thread. Then, we get all the threads simultaneously
entering softdep_request_cleanup().

There are two problems:
- Several threads execute MNT_VNODE_FOREACH_ALL() loops in parallel.  Due
  to the locking, they quickly start executing 'in phase' with the speed
  of the slowest thread.
- Since each thread already owns the lock for a dirty vnode, other threads
  non-blocking attempt to lock the vnode owned by other thread fail,
  and loops executing without making the progress.
Retry logic does not allow the situation to recover.  The result is
a livelock.

Fix these problems by making the following changes:
- Allow only one thread to enter MNT_VNODE_FOREACH_ALL() loop per mp.
  A new flag FLUSH_RC_ACTIVE guards the loop.
- If there were failed locking attempts during the loop, abort retry
  even if there are still work items on the mp work list.  An
  assumption is that the items will be cleaned when other thread
  either fsyncs its vnode, or unlock and allow yet another thread to
  make the progress.

It is possible now that some calls would get undeserved ENOSPC from
ffs_alloc(), because the cleanup is not aggressive enough. But I do
not see how can we reliably clean up workitems if calling
softdep_request_cleanup() while still owning the vnode lock. I thought
about scheme where ffs_alloc() returns ERESTART and saves the retry
counter somewhere in struct thread, to return to the top level, unlock
the vnode and retry.  But IMO the very rare (and unproven) spurious
ENOSPC is not worth the complications.

Reported and tested by:	pho
Style and comments by:	mckusick
Reviewed by:	mckusick
Sponsored by:	The FreeBSD Foundation
MFC after:	2 weeks
2017-06-03 16:18:50 +00:00
bin Fix integer overflow detection in dd 2017-05-31 16:07:32 +00:00
cddl MFV r316922: 5380 receive of a send -p stream doesn't need to try renaming snapshots 2017-05-24 22:30:21 +00:00
contrib tsan: set noexec stack on aarch64 2017-06-03 13:13:57 +00:00
crypto Merge OpenSSL 1.0.2l. 2017-05-25 20:52:16 +00:00
etc crashinfo: add "batch" mode and use it during boot 2017-06-01 21:23:04 +00:00
gnu Add 11.1 to the mdoc(7) mdoc.local.in. 2017-05-22 14:21:07 +00:00
include Use __BSD_VISIBLE test instead checking for absense of _POSIX_SOURCE. 2017-05-24 09:25:13 +00:00
kerberos5 Disconnect heimdal version of qsort.c from build because we are already 2017-05-26 06:09:11 +00:00
lib Decode the 'who' argument passed to getrusage(). 2017-06-03 14:22:15 +00:00
libexec Document direct execution mode for rtld. 2017-05-29 13:38:26 +00:00
release Turn on support for the Amazon "Elastic Network Adapter" in EC2 AMIs. 2017-05-25 19:02:54 +00:00
rescue DIRDEPS_BUILD: Update dependencies. 2017-05-09 01:48:23 +00:00
sbin ipfw.8: Note that the ipfw_nat kernel module must be loaded or that the 2017-06-01 09:14:49 +00:00
secure Merge OpenSSL 1.0.2l. 2017-05-25 20:52:16 +00:00
share Consistently use lowercase hex numbers in ascii(7). 2017-06-02 15:32:11 +00:00
sys Mitigate several problems with the softdep_request_cleanup() on busy 2017-06-03 16:18:50 +00:00
targets Enable llvm-objdump by default per r310775 2017-05-16 03:12:51 +00:00
tests tests/sys/opencrypto/runtests: apply minor polish to test script 2017-06-01 19:58:40 +00:00
tools Fix some new errors and a warning in cryptotest. 2017-06-01 21:07:32 +00:00
usr.bin Decode the 'who' argument passed to getrusage(). 2017-06-03 14:22:15 +00:00
usr.sbin makefs: rename variable for NetBSD diff reduction 2017-06-03 14:28:19 +00:00
.arcconfig callsign isn't required anymore 2016-09-29 06:19:45 +00:00
.arclint
COPYRIGHT Bump copyright year. 2016-12-31 12:41:42 +00:00
LOCKS
MAINTAINERS Due to time constraints remove myself as a vmm(4) maintainer. 2017-04-15 22:42:23 +00:00
Makefile Import OpenSSL 1.0.2l. 2017-05-25 19:38:38 +00:00
Makefile.inc1 native-xtools: Connect lld. 2017-06-02 00:57:59 +00:00
Makefile.libcompat Support skipping 'make obj' tree-walk. 2017-05-09 20:21:38 +00:00
ObsoleteFiles.inc Remove stale cap_rights_get(2) manpage. 2017-06-02 03:53:34 +00:00
README Import OpenSSL 1.0.2l. 2017-05-25 19:38:38 +00:00
README.md Remove hard line breaks from README.md 2017-03-01 15:39:58 +00:00
UPDATING Revert r319457. 2017-06-01 20:29:48 +00:00

FreeBSD Source:

This is the top level of the FreeBSD source directory. This file was last revised on: FreeBSD

For copyright information, please see the file COPYRIGHT in this directory (additional copyright information also exists for some sources in this tree - please see the specific source directories for more information).

The Makefile in this directory supports a number of targets for building components (or all) of the FreeBSD source tree. See build(7) and http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/makeworld.html for more information, including setting make(1) variables.

The buildkernel and installkernel targets build and install the kernel and the modules (see below). Please see the top of the Makefile in this directory for more information on the standard build targets and compile-time flags.

Building a kernel is a somewhat more involved process. See build(7), config(8), and http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/kernelconfig.html for more information.

Note: If you want to build and install the kernel with the buildkernel and installkernel targets, you might need to build world before. More information is available in the handbook.

The kernel configuration files reside in the sys/<arch>/conf sub-directory. GENERIC is the default configuration used in release builds. NOTES contains entries and documentation for all possible devices, not just those commonly used.

Source Roadmap:

bin				System/user commands.

cddl			Various commands and libraries under the Common Development  
				and Distribution License.

contrib			Packages contributed by 3rd parties.

crypto			Cryptography stuff (see crypto/README).

etc				Template files for /etc.

gnu				Various commands and libraries under the GNU Public License.  
				Please see gnu/COPYING* for more information.

include			System include files.

kerberos5		Kerberos5 (Heimdal) package.

lib				System libraries.

libexec			System daemons.

release			Release building Makefile & associated tools.

rescue			Build system for statically linked /rescue utilities.

sbin			System commands.

secure			Cryptographic libraries and commands.

share			Shared resources.

sys				Kernel sources.

tests			Regression tests which can be run by Kyua.  See tests/README
				for additional information.

tools			Utilities for regression testing and miscellaneous tasks.

usr.bin			User commands.

usr.sbin		System administration commands.

For information on synchronizing your source tree with one or more of the FreeBSD Project's development branches, please see:

http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/synching.html