Using a benchmark which creates 2 million files in one TXG, I observe
that the thread running spa_sync() is on CPU almost the entire time we
are syncing, and therefore can be a performance bottleneck. About 50% of
the time in spa_sync() is in dmu_objset_do_userquota_updates().
The problem is that dmu_objset_do_userquota_updates() calls
zap_increment_int(DMU_USERUSED_OBJECT) once for every file that was
modified (or created). In this benchmark, all the files are owned by the
same user/group, so all 2 million calls to zap_increment_int() are
modifying the same entry in the zap. The same issue exists for the
DMU_GROUPUSED_OBJECT.
We should keep an in-memory map from user to space delta while we are
syncing, and when we finish, iterate over the in-memory map and modify
the ZAP once per entry. This reduces the number of calls to
zap_increment_int() from "number of objects modified" to "number of
owners/groups of modified files".
This reduced the time spent in spa_sync() in the file create benchmark
by ~33%, from 11 seconds to 7 seconds.
Closes#107
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Steve Gonczi <steve.gonczi@delphix.com>
Reviewed by: Ned Bass <bass6@llnl.gov>
Reviewed by: Jinshan Xiong <jinshan.xiong@intel.com>
Author: Matthew Ahrens <mahrens@delphix.com>
openzfs/openzfs@5fc46359c5
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Yuri Pankov <yuri.pankov@nexenta.com>
Reviewed by: Andrew Stormont <andyjstormont@gmail.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author: Toomas Soome <tsoome@me.com>
openzfs/openzfs@c8811bd3e2
Submitted by: Hongjiang Zhang <honzhan microsoft com>
Reported by: Lili Deng <v-lide microsoft com>
MFC after: 3 days
Sponsored by: Microsoft
Differential Revision: https://reviews.freebsd.org/D8238
RPI3 kernel config builds kernel compatible with latest upstream device
tree and firmware: https://github.com/raspberrypi/firmware/tree/master/boot
As of today it's 597c662a613df1144a6bc43e5f4505d83bd748ca
Default console is PL01x, so pi3-disable-bt dt overlay should be configured
in config.txt and stock U-Boot should be patched to use proper serial port.
Yet unsupported: SMP, VCHIQ, RNG driver. RNG requires some work due to
upstream device tree incompatibility.
Multiple people contributed to this work over time: db@, loos@, manu@
Cases other than MK_* (e.g. ${MACHINE_CPUARCH} == "i386") have been left
as is.
Reviewed by: imp
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D8246
- Revert BUS_SPACE_PHYSADDR back to rman_get_start. BUS_SPACE_PHYSADDR was
introduced in 2013 as temporary wrapper until proper solution appears.
It's ARM only and since we need this file for ARM64 build and no proper
API has been introduced - just revert the change and make sure it's
going to appear when people grep for BUS_SPACE_PHYSADDR in sources.
- Fix printf format for size_t variables
dev_net.c code.
The NETIF_OPEN_CLOSE_ONCE flag was added in r201932 to prevent that behaviour
on some architectures (sparc64 and powerpc64) the default was left to always
open and close the device for each open and close of a file by the loader
because it was necessary for u-boot on arm.
Since it has been added, the flag was turned on for every arches including the
u-boot loader for arm.
This also fixes netbooting on RPi3 (tested by gonzo@)
For the loader.efi it greatly speeds up netbooting
Reviewed by: emaste, gonzo, tsoome
Approved by: gonzo
MFC after: 1 month
Sponsored by: Gandi.net
Differential Revision: https://reviews.freebsd.org/D8230
Ignore the ECN bits on 'tos' and 'set-tos' and allow to use
DCSP names instead of having to embed their TOS equivalents
as plain numbers.
Obtained from: OpenBSD
Sponsored by: OPNsense
Differential Revision: https://reviews.freebsd.org/D8165
compiler-rt's complex division support routines contain calls to
compiler builtins such as `__builtin_scalbnl`. Unfortunately Clang
turns these back into a call to `scalbnl`.
For now link libm's C version of the required support routines.
Reviewed by: ed
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D8190
They are not yet connected to the build, but I am adding them to allow
for easier testing, ports exp-runs, etc.
Reviewed by: ed
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D8188
Back in 2015 when I reimplemented these functions to use an AVL tree, I
was annoyed by the weakness of the typing of these functions. Both tree
nodes and keys are represented by 'void *', meaning that things like the
documentation for these functions are an absolute train wreck.
To make things worse, users of these functions need to cast the return
value of tfind()/tsearch() from 'void *' to 'type_of_key **' in order to
access the key. Technically speaking such casts violate aliasing rules.
I've observed actual breakages as a result of this by enabling features
like LTO.
I've filed a bug report at the Austin Group. Looking at the way the bug
got resolved, they made a pretty good step in the right direction. A new
type 'posix_tnode' has been added to correspond to tree nodes. It is
still defined as 'void' for source-level compatibility, but in the very
far future it could be replaced by a proper structure type containing a
key pointer.
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D8205
buildkernel failed with GCC 5.3 with
error: comparison of constant '852736' with boolean expression is always true
Sponsored by: The FreeBSD Foundation
Linux's copy of the Cavium SDK does not have these non-ASCII characters
and this reduces noise in diffs when comparing the two.
Sponsored by: The FreeBSD Foundation
Suppose that we have an exclusively busy page, and a thread which can
accept shared-busy page. In this case, typical code waiting for the
page xbusy state to pass is
again:
VM_OBJECT_WLOCK(object);
...
if (vm_page_xbusied(m)) {
vm_page_lock(m);
VM_OBJECT_WUNLOCK(object); <---1
vm_page_busy_sleep(p, "vmopax");
goto again;
}
Suppose that the xbusy state owner locked the object, unbusied the
page and unlocked the object after we are at the line [1], but before we
executed the load of the busy_lock word in vm_page_busy_sleep(). If it
happens that there is still no waiters recorded for the busy state,
the xbusy owner did not acquired the page lock, so it proceeded.
More, suppose that some other thread happen to share-busy the page
after xbusy state was relinquished but before the m->busy_lock is read
in vm_page_busy_sleep(). Again, that thread only needs vm_object lock
to proceed. Then, vm_page_busy_sleep() reads busy_lock value equal to
the VPB_SHARERS_WORD(1).
In this case, all tests in vm_page_busy_sleep(9) pass and we are going
to sleep, despite the page being share-busied.
Update check for m->busy_lock == VPB_UNBUSIED in vm_page_busy_sleep(9)
to also accept shared-busy state if we only wait for the xbusy state to
pass.
Merge sequential if()s with the same 'then' clause in
vm_page_busy_sleep().
Note that the current code does not share-busy pages from parallel
threads, the only way to have more that one sbusy owner is right now
is to recurse.
Reported and tested by: pho (previous version)
Reviewed by: alc, markj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D8196
- Add inttypes.h #include for PRId64 macro
- Use FreeBSD's copy of getfh(2), which doesn't include a `fh_size` parameter.
Use sizeof(fhandle_t) instead as the size of fhp is always fixed as
fhandle_t, unlike NetBSD's copy of fhp, which is void*.
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
Previously the driver used more low level operations like iicbus_start
and iicbus_write. The problem is that those operations are not
implemented by iicbus(4) and the calls were effectively routed to
a driver to which the bus is attached.
But not all of the controllers implement such low level operations
while all of the drivers are expected to have iicbus_transfer.
While there fix incorrect implementation of iicsmb_bwrite and iicsmb_bread.
The former should send a byte count before the actual bytes, while the
latter should first receive the byte count and then receive the bytes.
I have tested only these commands:
- quick (r/w)
- send byte
- receive byte
- read byte
- write byte
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D8170
Unfortunately removing files with uchg set always succeeds with root on
FreeBSD. Unfortunately running the test as an unprivileged user isn't doable
because mounting tmpfs requires root
PR: 212861
Sponsored by: Dell EMC Isilon
with no creative content. Include "lost" changes from git:
o Use /dev/efi instead of /dev/efidev
o Remove redundant NULL checks.
Submitted by: kib@, dim@, zbb@, emaste@
Currently zfsbe ensures that subordinate filesystems are mounted at the
right mount points.
The script assumes that the subordinate filesystems of a boot environment
have their canmount property set to noauto, so that they are not
automatically mounted on boot. Whereas the root filesystem is mounted
by the kernel, there was nothing to mount its subordinates.
rc.d/zfsbe fills that gap.
Discussed with: allanjude, will
MFC after: 3 weeks
Differential Revision: https://reviews.freebsd.org/D7797
The runtime kernel loader, linker_load_file, unloads kernel files that
failed to load all of their modules. For consistency, treat preloaded
(loader.conf loaded) kernel files in the same way.
Reviewed by: kib
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D8200
because they can use sysrc in conjunction with ssh and xargs to perform
en-masse changes in a large distribution with lots of jails spread over
many hosts on a LAN/WAN.
Provide a mechanism for disabling the warning eschewed by /etc/rc.d/jail
in said situation. If jail_confwarn="NO" is in rc.conf(5) (default "YES")
skip the warning that per-jail configurations are obsolete and that the
user should migrate to jail.conf(5).
Reviewed by: jelischer
MFC after: 3 days
Sponsored by: FIS Global, Inc.
Differential Revision: https://reviews.freebsd.org/D7465
to ieee80211_add_rx_params() + drop last (ieee80211_rx_stats) parameter
Note: there is an additional check for ieee80211_get_rx_params()
return value (which does not exist in the original diff).
Reviewed by: adrian
Differential Revision: https://reviews.freebsd.org/D8207
