This speeds up buildworld by 16% on my system and buildkernel by 35%.
Rather than calling mkdep(1), which is just a wrapper around 'cc -E',
use the modern -MD -MT -MF flags to gather and generate dependencies during
compilation. This flag was introduced in GCC "a long time ago", in GCC 3.0,
and is also supported by Clang. (It appears that ICC also supports this but I
do not have access to test it). This avoids running the preprocessor *twice*
for every build, in both 'make depend' and 'make all'. This is especially
noticeable when using ccache since it does not cache preprocessor results from
mkdep(1) / 'cc -E', but still speeds up compilation with the -MD flags.
For 'make depend' a tree-walk is still done to ensure that all DPSRCS
are generated when expected, and that beforedepend/afterdepend and
_EXTRADEPEND are all still respected. In time this may change but for now
I've been conservative. The time for a tree-walk with -j combined with
SUBDIR_PARALLEL is not significant. For example, it takes about 9 seconds
with -j15 to walk all of src/ for 'make depend' now on my system.
A .depend file is still generated with the various rules that apply to
the final target, or custom rules. Otherwise there are now
per-built-object-file .depend files, such as .depend.filename.o. These
are included directly by make rather than populating .depend with a loop
and .depend lines, which only added overhead to the now almost-NOP 'make
depend' phase.
Before this I experimented with having mkdep(1) called in parallel per-file.
While this improved the kernel and lib/libc 'make depend' phase, it resulted
in slower build times overall.
The -M flags are removed from CFLAGS when linking since they have no effect.
Enabling this by default, for src or out-of-src, can be done once more testing
has been done, such as a ports exp-run, and with more compilers.
The system I used for testing was:
WITNESS
Build options: -j20 WITH_LLDB=yes WITH_DEBUG_FILES=yes WITH_FAST_DEPEND=yes
DISK: ZFS 3-way mirror with very slow disks using SSD l2arc/log.
The arc was fully populated with src tree files.
RAM: 76GiB
CPU: Intel(R) Xeon(R) CPU L5520 @2.27GHz
2 package(s) x 4 core(s) x 2 SMT threads = hw.ncpu=16
buildworld:
x buildworld-before
+ buildworld-fastdep
+-------------------------------------------------------------------------------+
|+ |
|+ |
|+ xx x|
| |_MA___||
|A |
+-------------------------------------------------------------------------------+
N Min Max Median Avg Stddev
x 3 3744.13 3794.31 3752.25 3763.5633 26.935139
+ 3 3153.34 3155.16 3154.2 3154.2333 0.91045776
Difference at 95.0% confidence
-609.33 +/- 43.1943
-16.1902% +/- 1.1477%
(Student's t, pooled s = 19.0569)
buildkernel:
x buildkernel-before
+ buildkernel-fastdep
+-------------------------------------------------------------------------------+
|+ x |
|++ xx|
| A||
|A| |
+-------------------------------------------------------------------------------+
N Min Max Median Avg Stddev
x 3 571.57 573.94 571.79 572.43333 1.3094401
+ 3 369.12 370.57 369.3 369.66333 0.79033748
Difference at 95.0% confidence
-202.77 +/- 2.45131
-35.4225% +/- 0.428227%
(Student's t, pooled s = 1.0815)
Sponsored by: EMC / Isilon Storage Division
MFC after: 3 weeks
Relnotes: yes
The command was checking local/remote system uptime, so rename the script to
match its function and to avoid confusion
The controlling variable in /etc/periodic.conf has been renamed from
daily_status_rwho_enable to daily_status_uptime_enable.
MFC after: 3 days
Reported by: Peter Jeremy <peter@rulingia.com>
Relnotes: yes
Sponsored by: EMC / Isilon Storage Division
MK_USB != no
Add the manpages to OptionalObsoleteFiles.inc
As a side-effect, this also fixes installworld with MK_USB == no
X-MFC with: r290128
Sponsored by: EMC / Isilon Storage Division
Other sidenotes:
- Remove unused variables with main(..)
- Convert errx/exit with -1 to errx/exit with 1
- Fix a bogus test in try_directory_open
(expected_errno == expected_errno -> errno == expected_errno) [*]
- Fix some warnings related to discarded qualifiers
- Remove a bogus else-statement at the end of check_mmap_exec(..) in the
successful case. mmap(2), POSIX, Linux, etc all don't state what the
behavior is when mixing O_WRONLY + PROT_EXEC, so assume success for now to
get the test program to pass again.
PR: 201286 [*]
MFC after: 1 week
Submitted by: David Binderman <dcb314@hotmail.com>
Sponsored by: EMC / Isilon Storage Division
Allows DMA from/to arbitrary KVA or physical address. /dev/ioat_test
must be enabled by root and is only R/W root, so this is approximately
as dangerous as /dev/mem and /dev/kmem.
Sponsored by: EMC / Isilon Storage Division
listen, and connect. The listen program is a simple server that
accepts and closes sockets, until a fixed limit, then sets the listen
queue to 0 and counts how many remaining connections it processes.
The connect program repeatedly opens connections and closes them
serving as the driver for the listen program.
Sponsored by: Limelight Networks
The IOAT hardware supports writing a 64-bit pattern to some destination
buffer. The same limitations on buffer length apply as for copy
operations. Throughput is a bit higher (probably because fill does not
have to spend bandwidth reading from a source in memory).
Support for testing Block Fill has been added to ioatcontrol(8) and the
ioat_test device. ioatcontrol(8) accepts the '-f' flag, which tests
Block Fill. (If the flag is omitted, the tool tests copy by default.)
The '-V' flag, in conjunction with '-f', verifies that buffers are
filled in the expected pattern.
Tested on: Broadwell DE (Xeon D-1500)
Sponsored by: EMC / Isilon Storage Division
The test logic now preallocates memory before running the test.
The buffer size is now configurable. Post-copy verification is
configurable. The number of copies to chain into one transaction (one
interrupt) is configurable.
A 'duration' mode is added, which repeats the test until the duration
has elapsed, reporting the B/s and transactions completed.
ioatcontrol.8 has been updated to document the new arguments.
Initial limits (on this particular Broadwell-DE) (and when the
interrupts are working) seem to be: 256 interrupts/sec or ~6 GB/s,
whichever limit is more restrictive.
Unfortunately, it seems the interrupt-reset handling on Broadwell isn't
working as intended. That will be fixed in a later commit.
Sponsored by: EMC / Isilon Storage Division
Extend OptionalObsoleteFiles.inc to delete all lib32 files when MK_LIB32 is
set to no on a system that previously had lib32 libraries installed.
Also, to prevent "make delete-old-dirs" from always deleting lib32 directories
after an installworld, move the lib32 subtree to its own mtree file that only
gets applied when MK_LIB32=yes.
Test: Ran "make delete-old" and "make delete-old-libs" on a system that never
had MK_LIB32 enabled, and on a system where MK_LIB32 was enabled and later
disabled. Did this both on amd64 and powerpc64.
Test: Ran "make tinderbox" without errors.
Reviewed by: emaste
Differential Revision: https://reviews.freebsd.org/D3923
- Make the requirements more complete for the testcases
- Detect prerequisites so the tests won't fail (zfs.ko is loaded, zpool(1)
is available, ACL support is enabled with UFS, etc).
- Work with temporary files/directories/mountpoints that work with atf/kyua
- Limit the testcases to work on temporary filesystems to reduce tainting the
test host
MFC after: 2 weeks
Reviewed by: trasz (earlier version)
Differential Revision: https://reviews.freebsd.org/D3810
test suite as tests/sys/posixshm
Some other highlights:
- Convert the testcases over to ATF
- Don't use hardcoded paths to /tmp (which violate the ATF/kyua samdbox); use
mkstemp to generate temporary paths for non-SHM_ANON shm objects.
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
Compare the fields that the AMD [1] and Intel [2] specs say will be
set once fnstenv returns.
Not all amd64 capable processors zero out the env.__x87.__other field
(example: AMD Opteron 6308). The AMD64/x64 specs aren't explicit on what the
env.__x87.__other field will contain after fnstenv is executed, so the values
in env.__x87.__other could be filled with arbitrary data depending on how the
CPU-specific implementation of fnstenv.
1. http://support.amd.com/TechDocs/26569_APM_v5.pdf
2. http://www.intel.com/Assets/en_US/PDF/manual/253666.pdf
Discussed with: kib, Anton Rang <anton.rang@isilon.com>
Reviewed by: Daniel O'Connor <darius@dons.net.au> (earlier patch; pre-generalization)
MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division
Reported by: Bill Morchin <wmorchin@isilon.com>
Also note that these env-only vars can be specified on the command line.
This fixes the dependent options that are env-only (such as WITH_META_MODE
and WITH_AUTO_OBJ) to properly display their dependencies.
Sponsored by: EMC / Isilon Storage Division
