Revamp sbuf_put_byte() to sbuf_put_bytes() in the obvious fashion and
fixup callers.
Add a thin shim around sbuf_put_bytes() with the old ABI to avoid ugly
changes to some callers.
Reviewed by: jhb, markj
Obtained from: Dan Sledz
Sponsored by: EMC / Isilon Storage Division
Differential Revision: https://reviews.freebsd.org/D3717
are updated lockess, different CPUs write its own view of timecounter
state. The critical section is done for safety, callers of
tc_cpu_ticks() are supposed to already enter critical section, or to
own a spinlock.
The change fixes sporadical reports of too high values reported for
the (W)CPU on platforms that do not provide cpu ticker and use
tc_cpu_ticks(), in particular, arm*.
Diagnosed and reviewed by: jhb
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
This fixes atf-c.h not properly being installed to /usr/include/ (in
the stagedir) via its override of 'INCSDIR_atf-c.h= ${INCLUDEDIR}'.
This fixes building things that depend on atf.
Staging seems to ignore OWN/GRP/MODE settings and needs further exploration.
Sponsored by: EMC / Isilon Storage Division
A change to a property on a dataset must be propagated to its descendants
in case that property is inherited. For datasets whose information is
not currently loaded into memory (e.g. a snapshot that isn't currently
mounted), there is nothing to do; the property change will take effect
the next time that dataset is loaded. To handle updates to datasets that
are in-core, ZFS registers a callback entry for each property of each
loaded dataset with the dsl directory that holds that dataset. There
is a dsl directory associated with each live dataset that references
both the live dataset and any snapshots of the live dataset. A property
change is effected by doing a traversal of the tree of dsl directories
for a pool, starting at the directory sourcing the change, and invoking
these callbacks.
The current implementation both registers and de-registers properties
individually for each loaded dataset. While registration for a property is
O(1) (insert into a list), de-registration is O(n) (search list and then
remove). The 'n' for de-registration, however, is not limited to the size
(number of snapshots + 1) of the dsl directory. The eviction portion
of the life cycle for the in core state of datasets is asynchronous,
which allows multiple copies of the dataset information to be in-core
at once. Only one of these copies is active at any time with the rest
going through tear down processing, but all copies contribute to the
cost of performing a dsl_prop_unregister().
One way to create multiple, in-flight copies of dataset information
is by performing "zfs list" operations from multiple threads
concurrently. In-core dataset information is loaded on demand and then
evicted when reference counts drops to zero. For datasets that are not
mounted, there is no persistent reference count to keep them resident.
So, a list operation will load them, compute the information required to
do the list operation, and then evict them. When performing this operation
from multiple threads it is possible that some of the in-core dataset
information will be reused, but also possible to lose the race and load
the dataset again, even while the same information is being torn down.
Compounding the performance issue further is a change made for illumos
issue 5056 which made dataset eviction single threaded. In environments
using automation to manage ZFS datasets, it is now possible to create
enough of a backlog of dataset evictions to consume excessive amounts
of kernel memory and to bog down the system.
The fix employed here is to make property de-registration O(1). With this
change in place, it is hoped that a single thread is more than sufficient
to handle eviction processing. If it isn't, the problem can be solved
by increasing the number of threads devoted to the eviction taskq.
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_dataset.c
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_dir.c:
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_prop.c:
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/dsl_dataset.h:
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/dsl_dir.h:
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/dsl_prop.h:
Associate dsl property callback records with both the
dsl directory and the dsl dataset that is registering the
callback. Both connections are protected by the dsl directory's
"dd_lock".
When linking callbacks into a dsl directory, group them by
the property type. This helps reduce the space penalty for the
double association (the property name pointer is stored once
per dsl_dir instead of in each record) and reduces the number of
strcmp() calls required to do callback processing when updating
a single property. Property types are stored in a linked list
since currently ZFS registers a maximum of 10 property types
for each dataset.
Note that the property buckets/records associated with a dsl
directory are created on demand, but only freed when the dsl
directory is freed. Given the static nature of property types
and their small number, there is no benefit to freeing the few
bytes of memory used to represent the property record earlier.
When a property record becomes empty, the dsl directory is either
going to become unreferenced a little later in this thread of
execution, or there is a high chance that another dataset is
going to be loaded that would recreate the bucket anyway.
Replace dsl_prop_unregister() with dsl_prop_unregister_all().
All callers of dsl_prop_unregister() are trying to remove
all property registrations for a given dsl dataset anyway. By
changing the API, we can avoid doing any lookups of callbacks
by property type and just traverse the list of all callbacks
for the dataset and free each one.
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_objset.c:
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vfsops.c:
Replace use of dsl_prop_unregister() with the new
dsl_prop_unregister_all() API.
illumos/illumos-gate@03bad06fbb
Author: Justin Gibbs <gibbs@scsiguy.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Prakash Surya <prakash.surya@delphix.com>
Approved by: Dan McDonald <danmcd@omniti.com>
Illumos issue:
6171 dsl_prop_unregister() slows down dataset eviction
https://www.illumos.org/issues/6171
MFC after: 2 weeks
bsd.obj.mk handles the needs fine. When an objdir exists it will
just rm -Rf the objdir. When it does not exist though it will
call 'clean' and 'cleandepend', which properly recurse in bsd.progs.mk.
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
Refer to the usb_quirk(4) manual page for more details on how to use
this new feature.
Submitted by: Maxime Soule <btik-fbsd@scoubidou.com>
PR: 203249
MFC after: 2 weeks
This fixes the following errors:
make: don't know how to make bsd.README. Stop
make: don't know how to make auto.obj.mk. Stop
This is easily seen in sys/dev/*.
The new behavior is now the expected output:
make: no target to make.
This would happen as MAKESYSPATH (.../share/mk) is auto added to the -I list.
Any directory where make is ran in the src tree that has no local Makefile
would then try executing the target in share/mk/Makefile, which by default
was to build the first entry in FILES. Of course, because bsd.README and
auto.obj.mk are not in the current directory the error is shown.
This check only works for bmake, but I will still MFC it with an extra
'!defined(.PARSEDIR) ||' guard for stable/10.
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
* Don't free the mbuf in the tx path - it uses the transmit path now,
so the caller frees the mbuf.
* Don't decrement the node ref upon error - that's up to the caller to
do as well.
Tested:
* Intel 5300 3x3 wifi, station mode
Noticed by: <s3erios@gmail.com>
This avoids needing a large boot partition / file system in order to
accommodate multiple kernels, and provides consistency with userland
debug. This also simplifies the process of moving kernel debug files
to a separate package and installing them on demand.
In addition, change kernel debug file extension to .debug, to match
userland debug files.
When using the supported kernel installation method the
/usr/lib/debug/boot/kernel directory will be renamed (to kernel.old)
as is done with /boot/kernel.
Developers wishing to maintain the historical behavior of installing
debug files in /boot/kernel/ can set KERN_DEBUGDIR="" in src.conf(5).
Reviewed by: bdrewery, brooks, imp, markj
Relnotes: yes
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D1006
This mostly fixes an interaction with bsd.test.mk with PROGS and SCRIPTS.
This was most notable with 'make clean' and 'make install', which r281055
and r272055 attempted to address but were inadequate.
It also addresses similar issues in bsd.progs.mk when not using bsd.test.mk.
This also fixes cases of NOT running commands in the parent when using
bsd.progs.mk:
- 'make clean' was not run for the main process for Makefiles which had both
FILES and SUBDIR but no PROGS or SCRIPTS. This usually was just a
leftover Kyuafile.auto. One such example is usr.bin/bmake/tests/sysmk/t1/2.
- 'make obj' was not running in the current directory with bsd.test.mk due
to early inclusion of bsd.subdir.mk. This was not really a problem due to
the SUBDIRS using 'mkdir -p' for their objdirs.
There were subtle bugs causing this wrong behavior:
1. bsd.progs.mk needs to set SCRIPTS to empty when recursing to avoid
the sub-makes from installing, cleaning or building the SCRIPTS;
only the parent make should be doing this. r281055 effectively did
the same but wasn't enough.
2. CLEANFILES may contain (especially from *.test.mk) files which only
the parent should clean, such as from FILES and SCRIPTS. To resolve
sub-makes also cleaning these, reset CLEANFILES and CLEANDIRS in the
children before including bsd.prog.mk. A tempting alternative would be
to only handle CLEANFILES in the parent but then the child bsd.prog.mk
CLEANFILES of per-PROGS wouldn't be setup.
3. bsd.subdir.mk was included too soon in bsd.test.mk. It needs to be
included after bsd.prog.mk as the SCRIPTS logic is short-circuitted if
'install:' is already defined (which bsd.subdir.mk does). There is
actually no need to include bsd.subdir.mk from bsd.test.mk as bsd.prog.mk
and bsd.obj.mk will do so in the proper order. The description in r257095
covers this for FILES and was fixed differently, though changing the
handling of target(install) in bsd.prog.mk may make sense after more
research.
4. bsd.progs.mk had extra logic to handle recursing SCRIPTS if PROGS was
empty, which isn't its business to be doing. SCRIPTS is handled fine
by bsd.prog.mk. This mostly reverts and reworks the fix in r259209 and
partially reverts r272055.
5. bsd.progs.mk has no need to depend 'all:' on SCRIPTS and FILES. These
are handled by bsd.prog.mk/bsd.files.mk fine. This also partially reverts
r272055.
6. bsd.progs.mk was not drop-in safe for bsd.prog.mk. Move the PROGS
check from r273186 to allow it to be used safely.
Specific tested cases:
SCRIPTS:no PROGS:no FILES:yes SUBDIR:yes
usr.bin/bmake/tests/sysmk/t1/2
SCRIPTS:yes PROGS:no FILES:yes SUBDIR:no
usr.bin/bmake/tests/sysmk/t1/2/1
SCRIPTS:yes PROGS:yes FILES:yes SUBDIR:yes
lib/libthr/tests
SCRIPTS:yes PROGS:no FILES:yes SUBDIR:no
usr.bin/yacc/tests
libexec/atf/atf-sh/tests
A full buildworld/installworld/clean comparison with mtree was also done.
The only relevant difference was the new fixed behavior of removing
Kyuafile.auto from the objdir in 'clean'.
Converting SCRIPTS to be a special case FILES group will make this less
fragile and is being explored.
One known remaining issue is 'cleandepend' removing the tags files for
every recursive call.
Note that the 'make clean' command runs for the CURDIR last, which can make
it appear to run multiple times when cleaning in tests/, but each command is
for a SUBDIR returning up the chain. This is purely bsd.subdir.mk behavior.
PR: 191055
PR: 191955
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
In the first build the TOOLSDIR does not exit yet which causes CC to default
to the sys.mk version. Once a TOOLSDIR is created during the build though,
this logic was changing CC to ${TOOLSDIR}/usr/bin/cc even though that file
did not exist. Thus CC went from 'cc' to '/usr/bin/cc' which forced a
rebuild of everything while using the same compiler. Check that TOOLSDIR is
not empty to avoid this. If there is actually a TOOLSDIR cc then it will be
used and properly rebuild.
Sponsored by: EMC / Isilon Storage Division
Filemon(4) will record paths as they are seen, not as fully resolved. make(1)
will take the .MAKE.META.IGNORE_PATHS values and resolve them. This creates
a discrepancy if CCACHE_DIR is a symlink. Fix this by ensuring it is
resolved for its actual usage.
Submitted by: sjg
