Tools removed from the list are now provided by the ELF Tool Chain
project and are not controlled by the WITHOUT_BINUTILS_BOOTSTRAP knob.
Sponsored by: The FreeBSD Foundation
Although POSIX literally permits failing with [EINVAL] if IPC_CREAT and
IPC_EXCL were both passed, the semaphore set already exists and has fewer
semaphores than nsems, this does not allow an application to retry safely:
if the [EINVAL] is actually because of the semmsl limit, an infinite loop
would result.
PR: 206927
the object directory among all builds where it makes sense. When
building with NANO_CPUTYPE, separate that out to its own object
directory. Put disk files in their own directories.
This should make having multiple variants of the same architecture
saner.
o Make sure we create bsd label for MBR scheme (though we don't
really need it for the efi case, and boot1 can't read it). Add
notes about why we have to do this, at least for BIOS.
o Make the BIOS / UEFI with gpt config work.
o Remove now-moribund packaging stuff
Add support for being able to boot off both UEFI and BIOS firmware,
ala the memstick trick.
Add support for writing to GPT volumes.
Move away from using bsd labels at all for these embedded stuff.
Minor tweaks to README.
spelled ${NANO_SLICE_ROOT}a and ${NANO_SLICE_ALTROOT}a respectively,
and that's the default value. This will allow nanobsd on systems
without a bsd label. That's rarely needed these days, even in an MBR
world. The default will shift to this in the future, but remain an
option.
the second set (increment the original ports by 10)
This avoids issues where the first listening socket might not be torn
down by the time it makes it to the second set of testcases.
The sockets should likely only be setup once, but this keeps in the
spirit of the original testcases, so this will be easier to backport
to ^/stable/9
MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division
It always fails when trying to send through the sendit(9) private KPI in the
kernel due to a size mismatch between the msghdr and data being sent [*], which
suspiciously seems like it's related to sizeof pointers instead of scalars, or
something of that ilk
MFC after: 1 week
PR: 206543, 206544 [*]
Sponsored by: EMC / Isilon Storage Division
check_scm_creds_sockcred after initial != NULL checks have been done for
debugging purposes
- Use more terse names for bintime (bt), cmesgcred (cmcred),
sockcred (sc), and timeval (tv) [*]
- Add some debug messages to better understand some of the flow of the test
program
MFC after: 1 week
Requested by: bde [*]
Use of the word "terse" (^.^) corrected by: jhb, rpokala [*]
Sponsored by: EMC / Isilon Storage Division
This bug could be reproduced easily by calling sem_open() with O_CREAT |
O_EXCL on a semaphore that is already open in the process. The struct
sem_nameinfo would be freed while still in sem_list and later calls to
sem_open() or sem_close() could access freed memory.
PR: 206396
MFC after: 5 days
This is needed to be able to run check-links.sh against a "sysrooted"
binary while ensuring that the ldd(1) call done on the host uses the
host libc. It is not possible to set LD_LIBRARY_PATH before calling
check-links.sh as then the "sysrooted" libc would be incorrectly used.
A LD_PRELOAD=libc.so is used to ldd(1) as it needs to use the host libc
to run. ldd(1) is a simple wrapper around execve(2) and dlopen(2) with
env LD_TRACE_LOADED_OBJECTS set. Due to the dlopen(2) restriction on
shared library tracing ldd(1) is still required for this lookup.
Sponsored by: EMC / Isilon Storage Division
- Get rid of unused argc/argv variables in main
- Exit on failure with a return code of 1 instead of -1 with err/errx as a
return code of -1 is implementation dependent
- Bump WARNS to 6
MFC after: 5 days
Sponsored by: EMC / Isilon Storage Division
reconnect.c:
- Convert the K&R prototype of main to an ANSI prototype to mute a
warning from gcc 4.2.1
- Close s_sock2 after finishing off the last test to plug a leak and
mute a warning from gcc 5.0 about a -Wunused-but-set variable
sendfile.c:
- Fix a -Wunused-but-set warning with gcc 5.0 with pagesize in main(..)
MFC after: 5 days
Sponsored by: EMC / Isilon Storage Division
- `SOCK_RAW` is the implied supported type parameter for socket(2) per route(4)
- localsw in `sys/kern/uipc_usrreq.c` doesn't have an entry for `SOCK_RAW`, so
the prototype is invalid (this isn't explicitly documented anywhere I could
find)
MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division
This will help ensure that scripts/parsers don't get confused when the message
is printed out
MFC after: 3 days
Sponsored by: EMC / Isilon Storage Division
tools/regression/geom_{concat,eli,gate,mirror,nop,raid3,shsec,stripe,uzip}
in to the FreeBSD test suite as
tests/sys/geom/class/{concat,eli,gate,mirror,nop,raid3,shsec,stripe,uzip}
The tools/regression/geom and tools/regression/geom_part testcases are being
left alone because both test sets are both currently broken.
The majority of this work was done on ^/user/ngie/more-tests2 . The differences
are as follows:
- tests/sys/geom/class/Makefile.inc is not present; it was
inlined into the class's Makefiles for explicitness.
- The testcases officially require root via kyua
- The geom_gate(4) tests don't use the pidfile changes proposed in
https://reviews.freebsd.org/D4836 .
MFC after: 1 month
Sponsored by: EMC / Isilon Storage Division
- Add a conf.sh file for executing common functions with geom_gate
- Use attach_md for attaching md(4) devices
- Don't hardcode /tmp for temporary files, which violates the kyua sandbox
- Add/increase sleeps to try and improve synchronization
- Add debug output for when checksums fail
test-1.t:
- Use pkill for killing ggated
MFC after: 3 weeks
Sponsored by: EMC / Isilon Storage Division
It is built in libgcc_s.so and libgcc_eh.a to simplify transition.
It is enabled by default on arm64 (where we previously had no other
unwinder) and may be enabled for testing on other platforms by setting
WITH_LLVM_LIBUNWIND in src.conf(5).
Also add compiler-rt's __gcc_personality_v0 implementation for use with
the LLVM unwinder.
Relnotes: Yes
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D4787
(geom_test_cleanup, etc) down so the testcases don't emit noise when
bailing
- Conform to the TAP protocol better when dealing with classes that can't
be loaded and with temporary files that can't be allocated for tracking
md(4) devices.
MFC after: 2 weeks
X-MFC with: r293028, r293029, r293048
Sponsored by: EMC / Isilon Storage Division
- Delete test-2.sh as it was an incomplete testcase, and the contents were
basically a subset of test-1.sh
- Add a conf.sh file for executing common functions with geom_uzip
- Use attach_md for attaching md(4) devices
- Don't hardcode /tmp for temporary files, which violates the kyua sandbox
MFC after: 3 weeks
Sponsored by: EMC / Isilon Storage Division
exit so things are cleaned up properly
- Use attach_md for attaching md(4) devices
- Don't hardcode /tmp for temporary files, which violates the kyua sandbox
MFC after: 3 weeks
Sponsored by: EMC / Isilon Storage Division
exit so things are cleaned up properly
- Use attach_md for attaching md(4) devices
- Don't hardcode /tmp for temporary files, which violates the kyua sandbox
MFC after: 3 weeks
Sponsored by: EMC / Isilon Storage Division
exit so things are cleaned up properly
- Use attach_md for attaching md(4) devices
- Don't hardcode /tmp for temporary files, which violates the kyua sandbox
MFC after: 3 weeks
Sponsored by: EMC / Isilon Storage Division
-- Use linear probing to find the first unique md(4) device, unlike the other
code which uses attach_md, as geli(8) allocates the md(4) devices itself
- Don't hardcode /tmp for temporary files, which violates the kyua sandbox
MFC after: 3 weeks
Sponsored by: EMC / Isilon Storage Division
- Add a geom_concat specific cleanup function and trap on that function at
exit so things are cleaned up properly
- Don't hardcode /tmp for temporary files, which violates the kyua sandbox
MFC after: 3 weeks
Sponsored by: EMC / Isilon Storage Division
only. You need to remount / rw and export TMPDIR=/pkg/tmp. pkg will
then work. It's slow though: 15 minutes to pkg install git on an RPi 2
with a decently fast SD card. Since this is for testing, we set
DEFAULT_ALWAYS_YES and ASSUME_ALWAYS_YES to YES.
- Implement a gmirror_test_cleanup function, which in turn calls
geom_test_cleanup to clean up all md(4) providers allocated in the test
run.
- Remove duplicate logic in test scripts for removing md(4) providers.
- Don't create files in /tmp (outside the kyua sandbox); use the current
directory instead
MFC after: 3 weeks
Sponsored by: EMC / Isilon Storage Division
for all geom classes, e.g. geom_uzip(4)
- These tests require root. Skip all of the tests if they're run as non-root
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
suite as tests/sys/kern/unix_passfd_test
- Convert testcases to ATF
- Fix an alignment issues
- Mark rights_creds_payload(..) as an expected failure (see PR # 181741)
Based [in part] on the following Differential Revision:
https://reviews.freebsd.org/D689
MFC after: 1 week
Submitted by: markj
Sponsored by: EMC / Isilon Storage Division
- Delete some spurious whitespace
- Use calloc instead of malloc in the last test to ensure that
sendspace is properly zero'ed out
Differential Revision: https://reviews.freebsd.org/D689 (part of a larger diff)
MFC after: 1 week
Reviewed by: asomers, ngie
Submitted by: markj
Sponsored by: EMC / Isilon Storage Division
bugfix-only release, with no new features.
Please note that from 3.5.0 onwards, clang and llvm require C++11
support to build; see UPDATING for more information.
tools/regression/mac/mac_portacl into the FreeBSD test suite as
tests/sys/mac/bsdextended and tests/sys/mac/portacl, respectively
MFC after: 1 month
Sponsored by: EMC / Isilon Storage Division
- A trap(1) call has been added to the test scripts to better
ensure that the tests do a better job at trying to restore the
test host state at the end of the tests (if the test was
interrupted before it would leave the system in an odd state,
potentially making the test results for subsequent runs
non-deterministic).
- Add root user checks
- Fix nc(1) usage:
-- -o is deprecated
-- Using `-w 10` will make the call timeout after 10 seconds so it
doesn't block indefinitely
- Use local variables
- Be more terse in the error messages
- Parameterize out "127.0.0.1"
MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division
- Use nitems(x) instead of handrolled sizeof(x) / sizeof(*x) macro
- Do not mark count != 0 case with bsde_get_rule_count as a failure; this
generates false positives on systems with ugidfw rules set on it
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
Use temporary filesystems / memory disks instead of a hardcoded path
which doesn't exist on test systems
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
Also, don't compile the ldexpl(3) testcases on platforms that don't support
the libcall (technically only x86 right now). This makes this test buildable on
arm*/mips*/powerpc*
PR: 205449 [*]
MFC after: 1 week
Tested on: stable/10 (amd64/i386), head (amd64/i386)
Sponsored by: EMC / Isilon Storage Division
It completely fails all assertions on i386 on both stable/9 and stable/10
PR: 205448
MFC after: 1 week
X-MFC to: stable/10
Sponsored by: EMC / Isilon Storage Division
- Use a separate variable for tracking the testcase count instead
of hardcoding the offset for the testcases
MFC after: 3 days
Sponsored by: EMC / Isilon Storage Division
Also, don't compile the exp2l(3) testcases on platforms that don't support the
libcall (technically only x86 right now). This makes this test buildable on
arm*/mips*/powerpc*
Tested on: stable/10 (amd64/i386), head (amd64/i386)
PR: 205446 [*]
MFC after: 1 week
X-MFC to: stable/10
Sponsored by: EMC / Isilon Storage Division
into the FreeBSD test suite
There's no functional change with these testcases; they're purposely
being left in TAP format for the time being
Other testcases which crash on amd64/i386 as-is have not been
integrated yet (they need to be retested on a later version of
CURRENT, as I haven't used i386 in some time)
MFC after: 3 weeks
Sponsored by: EMC / Isilon Storage Division
lib/libc/tests/nss
- Convert the testcases to ATF
- Do some style(9) cleanups:
-- Sort headers
-- Apply indentation fixes
-- Remove superfluous parentheses
- Explicitly print out debug printfs for use with `kyua {debug,report}`; for
items that were overly noisy, they've been put behind #ifdef DEBUG
conditionals
- Fix some format strings
MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division
It may only be used with WITH_AUTO_OBJ, which the WITH_DIRDEPS_BUILD does. We
could support this in the normal build as well if we forced creating the directory
and setting .OBJDIR.
Sponsored by: EMC / Isilon Storage Division
The new flag, -c <period>, sets the interrupt coalescing period in
microseconds through the new ioat(4) API ioat_set_interrupt_coalesce().
Also add a -z flag to zero ioat statistics before tests, to make it easy
to measure results.
Sponsored by: EMC / Isilon Storage Division
doesn't have lots of ../../foo in it.
o Tweak the powerpc64 variant a bit. This gets us closer to working
with qemu-system-poewrpc64, but we aren't quite there yet.
make. Add support for generating powerpc64 qemu images. We
can generate them, but there's something wrong booting them.
This also simplifies the user config files a bit, and removes
bits no longer true.
nanobsd. implement support for NanoBSD touching a file (and possibly
recording that fact) as well as replacing a directory with a symlink.
Also specify the default uname and gname for files and use that as a
/set command at the top of the generated METALOG file.
it and then chmod back. There's no chmod -push / chmod -pop so hard
code 444 as the right permissions here.
Also, fix more stray detritus that crept in (out?) while re-arranging
the deck chairs.
Build with ../nanobsd.sh -c rpi.cfg, for example.
This can be done as a normal user.
This is a work in progress. It relies on the new nopriv
build stuff committed to nanobsd, but isn't complete yet.
Currently, one must copy files into the DOS partition
in the image. Also, ownership isn't preserved because
this doesn't use the new mtree-dedup.awk yet, but rather
some crazy mtree stuff. The image building bits will
move up into nanobsd when they are ready.
Also includes very preliminary support for building qemu
images for all platforms that we can for qemu. It is
missing aarch64, and we put the image on s2 instead of
s1 and mkimg can't mark s2 as active, so there's some
issues. Oh, and I didn't do it for arm.
Take a look, kick the tires, expect problems.
(and soon augmented by nanobsd), performs the actions documented in
the script, and then spits out a new mtree file suitable for feeding
to makefs.
Discussed on: arch@
appropriate. First step in supporting a build w/o root. More to
follow as actions by customization scripts are not (yet) recorded in
the metalog, and duplicate entries in it aren't removed.
being in the environment. Also filter out the new SRC_ENV_CONF as
well. If you really need these set, set them in your config file,
not in the build environment used to launch nanobsd.
Pointed out by: bdrewery@
o Move SRCCONF and __MAKE_CONF into the environment to cope with
file paths with spaces in them better.
o Move the rest of the variable setting command line args into
__MAKE_CONF files.
o Trace the commands that we're using to build so they appear at the
top of the log.
o Be more consistent about quoting paths for cd and similar commands
to better cope with paths with spaces in them, though some more
work is likely needed.
o Add some comments about all this.
o Minor formatting tweaks in a couple places
Sponsored by: Netflix, Inc
The indent_wrapper tool only accepts full context diffs and works by
identifying the surrounding C-block touched by a diff and passing only
that to indent for styling. In the end a diff is produced or an
external tool like meld can be invoked, to show the styling
differences.
tests/sys/kern/pipe
- Fix style(9) bugs
- Fix compiler warnings
- Use `nitems(x)` instead of `sizeof(x) / sizeof(*x)` pattern
The testcases will be converted over to ATF eventually, but for now will be
integrated in as plain C tests
MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division
This allows META_FILES option to be renamed META_MODE.
Also add META_COOKIE_TOUCH for use in targets that can benefit
from a cookie when in meta mode.
Differential Revision: https://reviews.freebsd.org/D4153
Reviewed by: bdrewery
need s1 to be a FAT partition, s2 to be the config partition and s3
and s4 to be the ping-pong upgrade partitions.
NANO_SLICE_ROOT defaults to s1
NANO_SLICE_ALTROOT defaults to s2
NANO_SLICE_CFG defaults to s3
NANO_SLICE_DATA defaults to s4
All can be overridden in the config file. Some basic sanity checking
is in place, but is no substitute for being careful.
directory symlink when the target directory does not exist. This will
cause an error instead of a broken setup.
Sponsored by: EMC / Isilon Storage Division
- Move all section 5 bluetooth manpages under MK_BLUETOOTH != no
MFC after: 3 days
PR: 193260
Reported by: Philippe Michel <philippe.michel7@sfr.fr>
Sponsored by: EMC / Isilon Storage Division
as lib/libc/tests/gen
The code in test-fnmatch that was used for generating:
- bin/sh/tests/builtins/case2.0
- bin/sh/tests/builtins/case3.0
has been left undisturbed. The target `make sh-tests` has been moved over
from tools/regression/lib/libc/gen/Makefile to
lib/libc/tests/gen/Makefile and made into a PHONY target
case2.0 and case3.0 test input generation isn't being done automatically.
This needs additional discussion.
MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division
as lib/libc/tests/net
Also, fix eui64_aton_test:test_str(..). The test was comparing the result
of eui64_aton to a pointer of the expected result.
MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division
as lib/libc/tests/stdlib
- Make the code a bit more style(9) compliant
- Convert a sizeof(x)/sizeof(x[0]) to nitems
MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division
ccache is mostly beneficial for frequent builds where -DNO_CLEAN is not
used to achieve a safe pseudo-incremental build. This is explained in
more detail upstream [1] [2]. It incurs about a 20%-28% hit to populate the
cache, but with a full cache saves 30-50% in build times. When combined with
the WITH_FAST_DEPEND feature it saves up to 65% since ccache does cache the
resulting dependency file, which it does not do when using mkdep(1)/'CC
-E'. Stats are provided at the end of this message.
This removes the need to modify /etc/make.conf with the CC:= and CXX:=
lines which conflicted with external compiler support [3] (causing the
bootstrap compiler to not be built which lead to obscure failures [4]),
incorrectly invoked ccache in various stages, required CCACHE_CPP2 to avoid
Clang errors with parenthesis, and did not work with META_MODE.
The option name was picked to match the existing option in ports. This
feature is available for both in-src and out-of-src builds that use
/usr/share/mk.
Linking, assembly compiles, and pre-processing avoid using ccache since it is
only overhead. ccache does nothing special in these modes, although there is
no harm in calling it for them.
CCACHE_COMPILERCHECK is set to 'content' when using the in-tree bootstrap
compiler to hash the content of the compiler binary to determine if it
should be a cache miss. For external compilers the 'mtime' option is used
as it is more efficient and likely to be correct. Future work may optimize the
'content' check using the same checks as whether a bootstrap compiler is needed
to be built.
The CCACHE_CPP2 pessimization is currently default in our devel/ccache
port due to Clang requiring it. Clang's -Wparentheses-equality,
-Wtautological-compare, and -Wself-assign warnings do not mix well with
compiling already-pre-processed code that may have expanded macros that
trigger the warnings. GCC has so far not had this issue so it is allowed to
disable the CCACHE_CPP2 default in our port.
Sharing a cache between multiple checkouts, or systems, is explained in
the ccache manual. Sharing a cache over NFS would likely not be worth
it, but syncing cache directories between systems may be useful for an
organization. There is also a memcached backend available [5]. Due to using
an object directory outside of the source directory though you will need to
ensure that both are in the same prefix and all users use the same layout. A
possible working layout is as follows:
Source: /some/prefix/src1
Source: /some/prefix/src2
Source: /some/prefix/src3
Objdir: /some/prefix/obj
Environment: CCACHE_BASEDIR='${SRCTOP:H}' MAKEOBJDIRPREFIX='${SRCTOP:H}/obj'
This will use src*/../obj as the MAKEOBJDIRPREFIX and tells ccache to replace
all absolute paths to be relative. Using something like this is required due
to -I and -o flags containing both SRC and OBJDIR absolute paths that ccache
adds into its hash for the object without CCACHE_BASEDIR.
distcc can be hooked into by setting CCACHE_PREFIX=/usr/local/bin/distcc.
I have not personally tested this and assume it will not mix well with
using the bootstrap compiler.
The cache from buildworld can be reused in a subdir by first running
'make buildenv' (from r290424).
Note that the cache is currently different depending on whether -j is
used or not due to ccache enabling -fdiagnostics-color automatically if
stderr is a TTY, which bmake only does if not using -j.
The system I used for testing was:
WITNESS
Build options: -j20 WITH_LLDB=yes WITH_DEBUG_FILES=yes WITH_CCACHE_BUILD=yes
DISK: ZFS 3-way mirror with very slow disks using SSD l2arc/log.
The arc was fully populated with src tree files and ccache objects.
RAM: 76GiB
CPU: Intel(R) Xeon(R) CPU L5520 @2.27GHz
2 package(s) x 4 core(s) x 2 SMT threads = hw.ncpu=16
The WITH_FAST_DEPEND feature was used for comparison here as well to show
the dramatic time savings with a full cache.
buildworld:
x buildworld-before
+ buildworld-ccache-empty
* buildworld-ccache-full
% buildworld-ccache-full-fastdep
# buildworld-fastdep
+-------------------------------------------------------------------------------+
|% * # +|
|% * # +|
|% * # xxx +|
| |A |
| A|
| A |
|A |
| A |
+-------------------------------------------------------------------------------+
N Min Max Median Avg Stddev
x 3 3744.13 3794.31 3752.25 3763.5633 26.935139
+ 3 4519 4525.04 4520.73 4521.59 3.1104823
Difference at 95.0% confidence
758.027 +/- 43.4565
20.1412% +/- 1.15466%
(Student's t, pooled s = 19.1726)
* 3 1823.08 1827.2 1825.62 1825.3 2.0785572
Difference at 95.0% confidence
-1938.26 +/- 43.298
-51.5007% +/- 1.15045%
(Student's t, pooled s = 19.1026)
% 3 1266.96 1279.37 1270.47 1272.2667 6.3971113
Difference at 95.0% confidence
-2491.3 +/- 44.3704
-66.1952% +/- 1.17895%
(Student's t, pooled s = 19.5758)
# 3 3153.34 3155.16 3154.2 3154.2333 0.91045776
Difference at 95.0% confidence
-609.33 +/- 43.1943
-16.1902% +/- 1.1477%
(Student's t, pooled s = 19.0569)
buildkernel:
x buildkernel-before
+ buildkernel-ccache-empty
* buildkernel-ccache-empty-fastdep
% buildkernel-ccache-full
# buildkernel-ccache-full-fastdep
@ buildkernel-fastdep
+-------------------------------------------------------------------------------+
|# @ % * |
|# @ % * x + |
|# @ % * xx ++|
| MA |
| MA|
| A |
| A |
|A |
| A |
+-------------------------------------------------------------------------------+
N Min Max Median Avg Stddev
x 3 571.57 573.94 571.79 572.43333 1.3094401
+ 3 727.97 731.91 728.06 729.31333 2.2492295
Difference at 95.0% confidence
156.88 +/- 4.17129
27.4058% +/- 0.728695%
(Student's t, pooled s = 1.84034)
* 3 527.1 528.29 528.08 527.82333 0.63516402
Difference at 95.0% confidence
-44.61 +/- 2.33254
-7.79305% +/- 0.407478%
(Student's t, pooled s = 1.02909)
% 3 400.4 401.05 400.62 400.69 0.3306055
Difference at 95.0% confidence
-171.743 +/- 2.16453
-30.0023% +/- 0.378128%
(Student's t, pooled s = 0.954969)
# 3 201.94 203.34 202.28 202.52 0.73020545
Difference at 95.0% confidence
-369.913 +/- 2.40293
-64.6212% +/- 0.419774%
(Student's t, pooled s = 1.06015)
@ 3 369.12 370.57 369.3 369.66333 0.79033748
Difference at 95.0% confidence
-202.77 +/- 2.45131
-35.4225% +/- 0.428227%
(Student's t, pooled s = 1.0815)
[1] https://ccache.samba.org/performance.html
[2] http://www.mail-archive.com/ccache@lists.samba.org/msg00576.html
[3] https://reviews.freebsd.org/D3484
[5] https://github.com/jrosdahl/ccache/pull/30
PR: 182944 [4]
MFC after: 3 weeks
Sponsored by: EMC / Isilon Storage Division
Relnotes: yes
This speeds up buildworld by 16% on my system and buildkernel by 35%.
Rather than calling mkdep(1), which is just a wrapper around 'cc -E',
use the modern -MD -MT -MF flags to gather and generate dependencies during
compilation. This flag was introduced in GCC "a long time ago", in GCC 3.0,
and is also supported by Clang. (It appears that ICC also supports this but I
do not have access to test it). This avoids running the preprocessor *twice*
for every build, in both 'make depend' and 'make all'. This is especially
noticeable when using ccache since it does not cache preprocessor results from
mkdep(1) / 'cc -E', but still speeds up compilation with the -MD flags.
For 'make depend' a tree-walk is still done to ensure that all DPSRCS
are generated when expected, and that beforedepend/afterdepend and
_EXTRADEPEND are all still respected. In time this may change but for now
I've been conservative. The time for a tree-walk with -j combined with
SUBDIR_PARALLEL is not significant. For example, it takes about 9 seconds
with -j15 to walk all of src/ for 'make depend' now on my system.
A .depend file is still generated with the various rules that apply to
the final target, or custom rules. Otherwise there are now
per-built-object-file .depend files, such as .depend.filename.o. These
are included directly by make rather than populating .depend with a loop
and .depend lines, which only added overhead to the now almost-NOP 'make
depend' phase.
Before this I experimented with having mkdep(1) called in parallel per-file.
While this improved the kernel and lib/libc 'make depend' phase, it resulted
in slower build times overall.
The -M flags are removed from CFLAGS when linking since they have no effect.
Enabling this by default, for src or out-of-src, can be done once more testing
has been done, such as a ports exp-run, and with more compilers.
The system I used for testing was:
WITNESS
Build options: -j20 WITH_LLDB=yes WITH_DEBUG_FILES=yes WITH_FAST_DEPEND=yes
DISK: ZFS 3-way mirror with very slow disks using SSD l2arc/log.
The arc was fully populated with src tree files.
RAM: 76GiB
CPU: Intel(R) Xeon(R) CPU L5520 @2.27GHz
2 package(s) x 4 core(s) x 2 SMT threads = hw.ncpu=16
buildworld:
x buildworld-before
+ buildworld-fastdep
+-------------------------------------------------------------------------------+
|+ |
|+ |
|+ xx x|
| |_MA___||
|A |
+-------------------------------------------------------------------------------+
N Min Max Median Avg Stddev
x 3 3744.13 3794.31 3752.25 3763.5633 26.935139
+ 3 3153.34 3155.16 3154.2 3154.2333 0.91045776
Difference at 95.0% confidence
-609.33 +/- 43.1943
-16.1902% +/- 1.1477%
(Student's t, pooled s = 19.0569)
buildkernel:
x buildkernel-before
+ buildkernel-fastdep
+-------------------------------------------------------------------------------+
|+ x |
|++ xx|
| A||
|A| |
+-------------------------------------------------------------------------------+
N Min Max Median Avg Stddev
x 3 571.57 573.94 571.79 572.43333 1.3094401
+ 3 369.12 370.57 369.3 369.66333 0.79033748
Difference at 95.0% confidence
-202.77 +/- 2.45131
-35.4225% +/- 0.428227%
(Student's t, pooled s = 1.0815)
Sponsored by: EMC / Isilon Storage Division
MFC after: 3 weeks
Relnotes: yes
The command was checking local/remote system uptime, so rename the script to
match its function and to avoid confusion
The controlling variable in /etc/periodic.conf has been renamed from
daily_status_rwho_enable to daily_status_uptime_enable.
MFC after: 3 days
Reported by: Peter Jeremy <peter@rulingia.com>
Relnotes: yes
Sponsored by: EMC / Isilon Storage Division
MK_USB != no
Add the manpages to OptionalObsoleteFiles.inc
As a side-effect, this also fixes installworld with MK_USB == no
X-MFC with: r290128
Sponsored by: EMC / Isilon Storage Division
Other sidenotes:
- Remove unused variables with main(..)
- Convert errx/exit with -1 to errx/exit with 1
- Fix a bogus test in try_directory_open
(expected_errno == expected_errno -> errno == expected_errno) [*]
- Fix some warnings related to discarded qualifiers
- Remove a bogus else-statement at the end of check_mmap_exec(..) in the
successful case. mmap(2), POSIX, Linux, etc all don't state what the
behavior is when mixing O_WRONLY + PROT_EXEC, so assume success for now to
get the test program to pass again.
PR: 201286 [*]
MFC after: 1 week
Submitted by: David Binderman <dcb314@hotmail.com>
Sponsored by: EMC / Isilon Storage Division
Allows DMA from/to arbitrary KVA or physical address. /dev/ioat_test
must be enabled by root and is only R/W root, so this is approximately
as dangerous as /dev/mem and /dev/kmem.
Sponsored by: EMC / Isilon Storage Division
listen, and connect. The listen program is a simple server that
accepts and closes sockets, until a fixed limit, then sets the listen
queue to 0 and counts how many remaining connections it processes.
The connect program repeatedly opens connections and closes them
serving as the driver for the listen program.
Sponsored by: Limelight Networks
The IOAT hardware supports writing a 64-bit pattern to some destination
buffer. The same limitations on buffer length apply as for copy
operations. Throughput is a bit higher (probably because fill does not
have to spend bandwidth reading from a source in memory).
Support for testing Block Fill has been added to ioatcontrol(8) and the
ioat_test device. ioatcontrol(8) accepts the '-f' flag, which tests
Block Fill. (If the flag is omitted, the tool tests copy by default.)
The '-V' flag, in conjunction with '-f', verifies that buffers are
filled in the expected pattern.
Tested on: Broadwell DE (Xeon D-1500)
Sponsored by: EMC / Isilon Storage Division
