Commit Graph

166 Commits

Author SHA1 Message Date
Bryan Drewery
68add21344 FAST_DEPEND: Don't waste time generating an empty .depend file.
The .depend file will still be generated if _EXTRADEPEND is used.  The target
is kept with a dependency on DPSRCS though so that 'make depend' will generate
all files.

Sponsored by:	EMC / Isilon Storage Division
2016-02-24 17:19:09 +00:00
Bryan Drewery
d7beff082b FAST_DEPEND: Rework how guessed dependencies are handled.
Rather than depend on .depend not existing, check the actual
.depend.OBJ file that will be used for that object.  If it doesn't
exist then use the guessed dependencies.

FAST_DEPEND may never have a .depend file.  Not having one means all of the
previous logic would over-depend all object files on all headers which is not
what we wanted.  It also means that if a .depend is generated before a build
is done for _EXTRADEPEND (such as for PROG or LIB) then all of these
dependencies would not be used since the .depend wasn't generated from mkdep
and the real .depend.* files are not generated until the build.

Sponsored by:	EMC / Isilon Storage Division
2016-02-24 17:19:05 +00:00
Bryan Drewery
3d437ee23f Follow-up r295667 with fixes for SRCS defined.
cleandepend should always remove CLEANDEPEND* if they are not empty,
but bsd.dep.mk should not add the tags entries unless SRCS is defined
as it did before.  The .depend file itself it still always removed
to avoid accidentally keeping a stale one around as done in r295666.

Sponsored by:	EMC / Isilon Storage Division
2016-02-24 17:18:55 +00:00
Bryan Drewery
9fb02a70d0 Avoid reading .depend.* in simple cases where not needed.
This will speed up some tree-walks with FAST_DEPEND which otherwise
would include length(SRCS) .depend files.

This also uses a trick suggested by sjg@ to still read them in when
specifying _V_READ_DEPEND=1 in the env/make args.

Sponsored by:	EMC / Isilon Storage Division
2016-02-19 00:41:24 +00:00
Bryan Drewery
ae038fc336 Add CLEANDEPEND[FILES|DIRS] and simplify the tags cleanup to use it.
Sponsored by:	EMC / Isilon Storage Division
2016-02-16 19:11:17 +00:00
Bryan Drewery
b601b230d8 Always remove .depend.* regardless of WITH_FAST_DEPEND in case of changing
which is used.

Sponsored by:	 EMC / Isilon Storage Division
2016-02-16 19:02:04 +00:00
Bryan Drewery
3b1e60a40a Remove temporary hack from r294370 for SSH upgrades.
MFC after:	1 week
Sponsored by:	EMC / Isilon Storage Division
2016-02-16 19:00:22 +00:00
Bryan Drewery
2cf60b3ce2 Fix -include .depend hack from r294370 for headers not in .PATH.
This hack will be removed in a few weeks.  It is here to fix incremental
builds of SSH between r291941 and r294370.

Reported by:	jmallett
MFC after:	1 day
Sponsored by:	EMC / Isilon Storage Division
2016-01-28 18:57:47 +00:00
Bryan Drewery
5444d429b9 Revert yacc dependency back to pre-r241298.
Several attempts to fix this logic was done after r241298, which were
all reverted, yet this change was not.

The .h file does not depend on the .c file, so do not impose such a
dependency on it.  They are generated by the same command but do not
depend on each other.  Restore the .ORDER which should handle parallel build
issues.  This fixes an actual bug where the .h file is not recreated
when missing [1].  For example:
  cd lib/libc
  make cleanobj
  make nsparser.h
  rm nsparser.h
  make nsparser.h # will not rebuild nsparser.h

I have been trying to track down a build problem where nsparser.h is
missing when nslexer.o is built.  It is possible this is related.

Reported by:	bde [1]
		https://lists.freebsd.org/pipermail/svn-src-all/2012-October/059481.html
		https://lists.freebsd.org/pipermail/svn-src-all/2012-October/060038.html
MFC after:	3 weeks
Sponsored by:	EMC / Isilon Storage Division
2016-01-27 01:33:26 +00:00
Bryan Drewery
369737d7f0 Fix DIRDEPS_BUILD after r294752.
DIRDEPS_BUILD does not yet support PROGS having their own dependency
file.

Overriding .MAKE.DEPENDFILE here causes major problems with the meta
mode logic since it creates the Makefile.depend as '.depend' resulting
in infinite loops in make due to dirdeps.mk including .depend endlessly.

X-MFC-With:	r294752
MFC after:	1 week
Sponsored by:	EMC / Isilon Storage Division
2016-01-27 01:24:05 +00:00
Bryan Drewery
fc2c2cd9e3 Fix PROGS not reading .depend files after r284288 by making DEPENDFILE work.
We have had this user-modifable DEPENDFILE variable forever that does nothing
relevant for the user since fmake always used '.depend'.  Bmake
introduced the .MAKE.DEPENDFILE variable that can be modified to change
the name of '.depend'.

Prior to r284288, bsd.progs.mk was setting .MAKE.DEPENDFILE to allow
working incremental builds.  This was modified most likely to not
conflict with the META MODE handling of .MAKE.DEPENDFILE as it has a lot
more special logic for that variable.

MFC after:	1 week
Sponsored by:	EMC / Isilon Storage Division
2016-01-25 22:29:44 +00:00
Bryan Drewery
cfa5f2be13 Remove unneeded beforebuild hacks for dtrace probes.
This still allows skipping 'make depend'.  Headers in SRCS generate
properly.

MFC after:	1 week
Sponsored by:	EMC / Isilon Storage Division
2016-01-25 22:29:41 +00:00
Bryan Drewery
a298c2797c Fix incremental build of dtrace probes.
Currently dtrace(1) -Go does not properly rebuild the target if it
exists.  It results in missing symbols.

  dtrace -C -x nolibs -G -o usdt.o -s /root/git/freebsd/cddl/contrib/opensolaris/cmd/dtrace/test/tst/common/json/usdt.d tst.usdt.o
  dtrace: target object (usdt.o) already exists. Please remove the target
  dtrace: object and rebuild all the source objects if you wish to run the DTrace
  dtrace: linking process again
  cc -O2 -pipe -O0 -g -I/root/git/freebsd/cddl/usr.sbin/dtrace/tests/common/json -std=gnu99 -fstack-protector-strong -Qunused-arguments  -o tst.usdt.exe.full tst.usdt.o usdt.o
  tst.usdt.o: In function `main':
  /root/git/freebsd/cddl/contrib/opensolaris/cmd/dtrace/test/tst/common/json/tst.usdt.c:56: undefined reference to `__dtrace_bunyan_fake___log__debug'
  /root/git/freebsd/cddl/contrib/opensolaris/cmd/dtrace/test/tst/common/json/tst.usdt.c:60: undefined reference to `__dtrace_bunyan_fake___log__debug'
  cc: error: linker command failed with exit code 1 (use -v to see invocation)
  *** [tst.usdt.exe.full] Error code 1

This is a consequence of r212358.

MFC after:	1 week
Sponsored by:	EMC / Isilon Storage Division
2016-01-25 22:29:32 +00:00
Bryan Drewery
473bcb0163 mkdep: Fix -include not being added for .depend tracking.
This fixes incremental build of OpenSSH after the recent upgrade.

For example, in secure/lib/libssh, -include ssh_namespace.h is used on
all files.  This is not tracked in the .depend file though due to
MKDEP_CFLAGS not including it.  The ssh example was broken in r291941
when not using FAST_DEPEND due to the .depend bug.  FAST_DEPEND was not
affected by this because it generates dependencies at compile time and
thus sees the -include.

This ugly make syntax could be simpler for bmake by using :tW but
fmake-compatible syntax is used since this needs to be MFC'd all the way
to stable/9.

Also add a temporary hack to workaround existing checkouts building
incrementally with a .depend file not having these headers.

MFC after:	1 week
Sponsored by:	EMC / Isilon Storage Division
2016-01-20 01:40:18 +00:00
Bryan Drewery
d3b887fd0d FAST_DEPEND: Still use if filemon is not used.
If filemon is used then there is no need to generate dependency files during
compilation as the .meta files will achieve the same result.

This is a temporary solution until FAST_DEPEND is default.  Once that is
default there will be an option to disable dependency generation entirely
as it is only useful if an incremental build is planned, thus META_MODE+filemon
can enable that option to short-circuit all FAST_DEPEND-related logic.

Sponsored by:	EMC / Isilon Storage Division
2016-01-19 22:41:58 +00:00
Bryan Drewery
0b6ba3f22a Define .MAKE.MODE to normal to avoid the need for :U later.
Sponsored by:	EMC / Isilon Storage Division
2016-01-19 22:41:44 +00:00
Bryan Drewery
9f286e8295 FAST_DEPEND: Rework optimization for r290524.
The .MAKEFLAGS check inside of the .for loop is extremely slow for some
reason.  Just moving it out of the loop trimmed -V lookup time from 11
seconds to 1 second in the kernel obj directory.

Sponsored by:	EMC / Isilon Storage Division
2016-01-15 22:08:51 +00:00
Bryan Drewery
10f696a52d FAST_DEPEND: Only pass -MF if we care about the object being compiled.
This will save time generating dependency files that we didn't expect
due to cases where SRCS!=OBJS or for building custom targetted objects
in Makefiles that do not end up in the DEPENDOBJS list.

This uses a bmake trick to modify CFLAGS based on ${.TARGET}.  A
.PARSEDIR check is done for the sake of MFC safety.

MFC after:	2 weeks
Sponsored by:	EMC / Isilon Storage Division
2015-12-07 16:08:22 +00:00
Bryan Drewery
b9964333e0 FAST_DEPEND: Move handling code below yacc/lex/dtrace code that modified SRCS.
This fixes some of those newly added SRCS not having their depend files
included.

MFC after:	2 weeks
Sponsored by:	EMC / Isilon Storage Division
2015-12-07 16:08:19 +00:00
Bryan Drewery
69afbf0275 FAST_DEPEND: Only try to use dependencies from C/C++ SRCS as mkdep did.
Rather than try to guess at all of the OBJS variables just use SRCS
using the same patterns that mkdep does.  This also fixes a mistake
where dependencies were being generated with FAST_DEPEND when they were
not for mkdep.  This happens when OBJS!=SRCS as is the case in
gnu/lib/csu where SRCS has 1 file and OBJS has several other files that
does not even contain the 1 SRCS file.  Generally in these cases the
OBJS have custom dependencies defined in their Makefile.  If we generate
dependencies for those and then load a .depend file, then .IMPSRC may
contain duplicate sources and lead to errors such as:
  cc: error: cannot specify -o when generating multiple output files

MFC after:	2 weeks
Sponsored by:	EMC / Isilon Storage Division
2015-12-07 16:08:16 +00:00
Bryan Drewery
6feede08e1 FAST_DEPEND: Actually use -MP from DEPEND_MP, completing r291554.
X-MFC-With:	r291554
MFC after:	1 week
Pointyhat to:	bdrewery
Sponsored by:	EMC / Isilon Storage Division
2015-12-01 15:50:35 +00:00
Bryan Drewery
b6725ecdb2 FAST_DEPEND: Workaround hacks in the wild that have '..' or '/' in OBJS.
This is mostly working around the converts/iconv port having '../ces/file.o'
in its OBJS list which resulted in '.depend../ces/file.o'.  Now it will have
'.depend.._ces_file.o'.

Other implementations have :T which would result in '.depend.file.o' here, but
that could lead to collisions.

X-MFC-With:	r291554
MFC after:	1 week
Sponsored by:	EMC / Isilon Storage Division
2015-12-01 03:09:54 +00:00
Bryan Drewery
593e465913 FAST_DEPEND: Allow not using -MP by setting DEPEND_MP blank.
-MP creates empty targets for all dependency files, which can be useful when a
dependency is deleted from the file system.  This would otherwise cause an
error for "don't know how to build FOO" since the .depend file is included
with the dependency registered.

This is mostly a workaround for the misc/dahdi-kmod port using '::' for one of
its dependencies, while -MP uses just ':'.  This results in an 'Inconsistent
operator for' error.

X-MFC-With:	r290433
MFC after:	1 week
Sponsored by:	EMC / Isilon Storage Division
2015-12-01 03:00:10 +00:00
Simon J. Gerraty
948f327ee4 Rename META_MODE option to DIRDEPS_BUILD
This allows META_FILES option to be renamed META_MODE.
Also add META_COOKIE_TOUCH for use in targets that can benefit
from a cookie when in meta mode.

Differential Revision:	https://reviews.freebsd.org/D4153
Reviewed by:	bdrewery
2015-11-14 03:24:48 +00:00
Bryan Drewery
b03ca5c3ec FAST_DEPEND: Fix building of wrong source files in some cases.
Similar to the original reason for these dependency hints to be added,
in r124637, the missing-dependency file case can lead to building of the
wrong source.

A clear example of this is in gnu/lib/libstdc++ where the .PATH contains
both contrib/gcc and contrib/libstdc++/src.

  contrib/gcc has a debug.c.
  contrib/libstdc++/src has a debug.cc.

  When building for the objects of debug.o, debug.So, and debug.po, it is
  ambiguous for which src file to use due to the suffix transformation
  rules, even though the proper one is listed first in .PATH.

  This was normally avoided due to these dependency hints for the initial
  build, and then mkdep would add an explicit 'debug.o: debug.cc'
  dependency into the .depend file.  WITH_FAST_DEPEND does not generate
  the .depend file with these, but puts them into .depend.debug.o instead.

Rather than extending the exists() check to each object's .depend.*
file, just enable the hint when when using WITH_FAST_DEPEND.  It fixes
the problem and seems to be safe enough to use since it is mapping SRCS
back to OBJS, rather than letting make make assumptions from OBJS to
SRCS.

A similar check mapping objects to headers is present in some mk files
but was not extended here for FAST_DEPEND since it has not yet been
found to be a problem.

X-MFC-With:	r290433
MFC after:	3 weeks
Sponsored by:	EMC / Isilon Storage Division
2015-11-09 23:37:04 +00:00
Bryan Drewery
9160419c7a Add built-in ccache build support via WITH_CCACHE_BUILD option.
ccache is mostly beneficial for frequent builds where -DNO_CLEAN is not
used to achieve a safe pseudo-incremental build.  This is explained in
more detail upstream [1] [2].  It incurs about a 20%-28% hit to populate the
cache, but with a full cache saves 30-50% in build times.  When combined with
the WITH_FAST_DEPEND feature it saves up to 65% since ccache does cache the
resulting dependency file, which it does not do when using mkdep(1)/'CC
-E'.  Stats are provided at the end of this message.

This removes the need to modify /etc/make.conf with the CC:= and CXX:=
lines which conflicted with external compiler support [3] (causing the
bootstrap compiler to not be built which lead to obscure failures [4]),
incorrectly invoked ccache in various stages, required CCACHE_CPP2 to avoid
Clang errors with parenthesis, and did not work with META_MODE.

The option name was picked to match the existing option in ports.  This
feature is available for both in-src and out-of-src builds that use
/usr/share/mk.

Linking, assembly compiles, and pre-processing avoid using ccache since it is
only overhead.  ccache does nothing special in these modes, although there is
no harm in calling it for them.

CCACHE_COMPILERCHECK is set to 'content' when using the in-tree bootstrap
compiler to hash the content of the compiler binary to determine if it
should be a cache miss.  For external compilers the 'mtime' option is used
as it is more efficient and likely to be correct.  Future work may optimize the
'content' check using the same checks as whether a bootstrap compiler is needed
to be built.

The CCACHE_CPP2 pessimization is currently default in our devel/ccache
port due to Clang requiring it.  Clang's -Wparentheses-equality,
-Wtautological-compare, and -Wself-assign warnings do not mix well with
compiling already-pre-processed code that may have expanded macros that
trigger the warnings.  GCC has so far not had this issue so it is allowed to
disable the CCACHE_CPP2 default in our port.

Sharing a cache between multiple checkouts, or systems, is explained in
the ccache manual.  Sharing a cache over NFS would likely not be worth
it, but syncing cache directories between systems may be useful for an
organization.  There is also a memcached backend available [5].  Due to using
an object directory outside of the source directory though you will need to
ensure that both are in the same prefix and all users use the same layout.  A
possible working layout is as follows:
  Source: /some/prefix/src1
  Source: /some/prefix/src2
  Source: /some/prefix/src3
  Objdir: /some/prefix/obj
  Environment: CCACHE_BASEDIR='${SRCTOP:H}' MAKEOBJDIRPREFIX='${SRCTOP:H}/obj'
This will use src*/../obj as the MAKEOBJDIRPREFIX and tells ccache to replace
all absolute paths to be relative.  Using something like this is required due
to -I and -o flags containing both SRC and OBJDIR absolute paths that ccache
adds into its hash for the object without CCACHE_BASEDIR.

distcc can be hooked into by setting CCACHE_PREFIX=/usr/local/bin/distcc.
I have not personally tested this and assume it will not mix well with
using the bootstrap compiler.

The cache from buildworld can be reused in a subdir by first running
'make buildenv' (from r290424).

Note that the cache is currently different depending on whether -j is
used or not due to ccache enabling -fdiagnostics-color automatically if
stderr is a TTY, which bmake only does if not using -j.

The system I used for testing was:
  WITNESS
  Build options: -j20 WITH_LLDB=yes WITH_DEBUG_FILES=yes WITH_CCACHE_BUILD=yes
  DISK: ZFS 3-way mirror with very slow disks using SSD l2arc/log.
        The arc was fully populated with src tree files and ccache objects.
  RAM: 76GiB
  CPU: Intel(R) Xeon(R) CPU L5520 @2.27GHz
       2 package(s) x 4 core(s) x 2 SMT threads = hw.ncpu=16

The WITH_FAST_DEPEND feature was used for comparison here as well to show
the dramatic time savings with a full cache.

buildworld:
  x buildworld-before
  + buildworld-ccache-empty
  * buildworld-ccache-full
  % buildworld-ccache-full-fastdep
  # buildworld-fastdep
  +-------------------------------------------------------------------------------+
  |%            *                               #                                +|
  |%            *                               #                                +|
  |%            *                               #             xxx                +|
  |                                                           |A                  |
  |                                                                              A|
  |             A                                                                 |
  |A                                                                              |
  |                                             A                                 |
  +-------------------------------------------------------------------------------+
      N           Min           Max        Median           Avg        Stddev
  x   3       3744.13       3794.31       3752.25     3763.5633     26.935139
  +   3          4519       4525.04       4520.73       4521.59     3.1104823
  Difference at 95.0% confidence
          758.027 +/- 43.4565
          20.1412% +/- 1.15466%
          (Student's t, pooled s = 19.1726)
  *   3       1823.08        1827.2       1825.62        1825.3     2.0785572
  Difference at 95.0% confidence
          -1938.26 +/- 43.298
          -51.5007% +/- 1.15045%
          (Student's t, pooled s = 19.1026)
  %   3       1266.96       1279.37       1270.47     1272.2667     6.3971113
  Difference at 95.0% confidence
          -2491.3 +/- 44.3704
          -66.1952% +/- 1.17895%
          (Student's t, pooled s = 19.5758)
  #   3       3153.34       3155.16        3154.2     3154.2333    0.91045776
  Difference at 95.0% confidence
          -609.33 +/- 43.1943
          -16.1902% +/- 1.1477%
          (Student's t, pooled s = 19.0569)

buildkernel:
  x buildkernel-before
  + buildkernel-ccache-empty
  * buildkernel-ccache-empty-fastdep
  % buildkernel-ccache-full
  # buildkernel-ccache-full-fastdep
  @ buildkernel-fastdep
  +-------------------------------------------------------------------------------+
  |#                        @   %                  *                              |
  |#                        @   %                  *     x                      + |
  |#                        @   %                  *     xx                     ++|
  |                                                      MA                       |
  |                                                                             MA|
  |                                                A                              |
  |                             A                                                 |
  |A                                                                              |
  |                         A                                                     |
  +-------------------------------------------------------------------------------+
      N           Min           Max        Median           Avg        Stddev
  x   3        571.57        573.94        571.79     572.43333     1.3094401
  +   3        727.97        731.91        728.06     729.31333     2.2492295
  Difference at 95.0% confidence
          156.88 +/- 4.17129
          27.4058% +/- 0.728695%
          (Student's t, pooled s = 1.84034)
  *   3         527.1        528.29        528.08     527.82333    0.63516402
  Difference at 95.0% confidence
          -44.61 +/- 2.33254
          -7.79305% +/- 0.407478%
          (Student's t, pooled s = 1.02909)
  %   3         400.4        401.05        400.62        400.69     0.3306055
  Difference at 95.0% confidence
          -171.743 +/- 2.16453
          -30.0023% +/- 0.378128%
          (Student's t, pooled s = 0.954969)
  #   3        201.94        203.34        202.28        202.52    0.73020545
  Difference at 95.0% confidence
          -369.913 +/- 2.40293
          -64.6212% +/- 0.419774%
          (Student's t, pooled s = 1.06015)
  @   3        369.12        370.57         369.3     369.66333    0.79033748
  Difference at 95.0% confidence
          -202.77 +/- 2.45131
          -35.4225% +/- 0.428227%
          (Student's t, pooled s = 1.0815)

[1] https://ccache.samba.org/performance.html
[2] http://www.mail-archive.com/ccache@lists.samba.org/msg00576.html
[3] https://reviews.freebsd.org/D3484
[5] https://github.com/jrosdahl/ccache/pull/30

PR:		182944 [4]
MFC after:	3 weeks
Sponsored by:	EMC / Isilon Storage Division
Relnotes:	yes
2015-11-08 00:50:18 +00:00
Bryan Drewery
dbced32af5 FAST_DEPEND: Don't enable when .MAKE.MODE=meta.
This is because the .meta files generated from filemon already contain a
list of all files read to generate the object.

X-MFC-With:	r290433
MFC after:	3 weeks
Sponsored by:	EMC / Isilon Storage Division
2015-11-08 00:50:13 +00:00
Bryan Drewery
3cccdc9d58 FAST_DEPEND: Don't include depend files when using 'make -V'.
This is especially noticeable in the kernel obj directory since it
includes so many files.

X-MFC-With:	r290433
MFC after:	3 weeks
Sponsored by:	EMC / Isilon Storage Division
2015-11-08 00:50:09 +00:00
Bryan Drewery
cf1eeb33be Add a FAST_DEPEND option, off by default, which speeds up the build significantly.
This speeds up buildworld by 16% on my system and buildkernel by 35%.

Rather than calling mkdep(1), which is just a wrapper around 'cc -E',
use the modern -MD -MT -MF flags to gather and generate dependencies during
compilation.  This flag was introduced in GCC "a long time ago", in GCC 3.0,
and is also supported by Clang.  (It appears that ICC also supports this but I
do not have access to test it).  This avoids running the preprocessor *twice*
for every build, in both 'make depend' and 'make all'.  This is especially
noticeable when using ccache since it does not cache preprocessor results from
mkdep(1) / 'cc -E', but still speeds up compilation with the -MD flags.

For 'make depend' a tree-walk is still done to ensure that all DPSRCS
are generated when expected, and that beforedepend/afterdepend and
_EXTRADEPEND are all still respected.  In time this may change but for now
I've been conservative.  The time for a tree-walk with -j combined with
SUBDIR_PARALLEL is not significant.  For example, it takes about 9 seconds
with -j15 to walk all of src/ for 'make depend' now on my system.

A .depend file is still generated with the various rules that apply to
the final target, or custom rules.  Otherwise there are now
per-built-object-file .depend files, such as .depend.filename.o.  These
are included directly by make rather than populating .depend with a loop
and .depend lines, which only added overhead to the now almost-NOP 'make
depend' phase.

Before this I experimented with having mkdep(1) called in parallel per-file.
While this improved the kernel and lib/libc 'make depend' phase, it resulted
in slower build times overall.

The -M flags are removed from CFLAGS when linking since they have no effect.

Enabling this by default, for src or out-of-src, can be done once more testing
has been done, such as a ports exp-run, and with more compilers.

The system I used for testing was:
  WITNESS
  Build options: -j20 WITH_LLDB=yes WITH_DEBUG_FILES=yes WITH_FAST_DEPEND=yes
  DISK: ZFS 3-way mirror with very slow disks using SSD l2arc/log.
        The arc was fully populated with src tree files.
  RAM: 76GiB
  CPU: Intel(R) Xeon(R) CPU L5520 @2.27GHz
       2 package(s) x 4 core(s) x 2 SMT threads = hw.ncpu=16

buildworld:
  x buildworld-before
  + buildworld-fastdep
  +-------------------------------------------------------------------------------+
  |+                                                                              |
  |+                                                                              |
  |+                                                                       xx    x|
  |                                                                       |_MA___||
  |A                                                                              |
  +-------------------------------------------------------------------------------+
      N           Min           Max        Median           Avg        Stddev
  x   3       3744.13       3794.31       3752.25     3763.5633     26.935139
  +   3       3153.34       3155.16        3154.2     3154.2333    0.91045776
  Difference at 95.0% confidence
          -609.33 +/- 43.1943
          -16.1902% +/- 1.1477%
          (Student's t, pooled s = 19.0569)

buildkernel:
  x buildkernel-before
  + buildkernel-fastdep
  +-------------------------------------------------------------------------------+
  |+                                                                            x |
  |++                                                                           xx|
  |                                                                             A||
  |A|                                                                             |
  +-------------------------------------------------------------------------------+
      N           Min           Max        Median           Avg        Stddev
  x   3        571.57        573.94        571.79     572.43333     1.3094401
  +   3        369.12        370.57         369.3     369.66333    0.79033748
  Difference at 95.0% confidence
          -202.77 +/- 2.45131
          -35.4225% +/- 0.428227%
          (Student's t, pooled s = 1.0815)

Sponsored by:	EMC / Isilon Storage Division
MFC after:	3 weeks
Relnotes:	yes
2015-11-06 04:45:29 +00:00
Simon J. Gerraty
f5374544f1 Building on fmake is no longer possible so removed tests for bmake. 2015-06-10 18:14:38 +00:00
Simon J. Gerraty
b17ff922d4 Merge from head 2015-05-26 21:52:57 +00:00
Mark Johnston
0c76c8aa8c Add a DTRACEFLAGS variable, which can be used to pass additional variables
to dtrace(1) invocations during a build. This change includes -C in the
default flags, which has dtrace(1) run input scripts through the
preprocessor. While here, sort the definitions of CP and CPP in sys.mk.

Differential Revision:	https://reviews.freebsd.org/D2204
Reviewed by:		imp, rpaulo (previous revision)
2015-04-08 02:43:05 +00:00
Mark Johnston
03a5f9f0fd Remove drti.o's dependency on libelf. This makes it possible to add DTrace
probes to userland programs and libraries without also needing to link
libelf.

dtrace -G places the __SUNW_dof symbol at the beginning of the DOF (DTrace
probe and provider metdata) section in the generated object file; drti.o
now just uses this symbol to locate the section. A complication occurs
when multiple dtrace-generated object files are linked together, since the
__SUNW_dof symbol defined in each file is global. This is handled by
using objcopy(1) to convert __SUNW_dof to a local symbol once drti.o has
been linked with the generated object file. Upstream, this is done using a
linker feature not present in GNU ld.

Differential Revision:	https://reviews.freebsd.org/D1757
Reviewed by:	rpaulo
MFC after:	1 month
Relnotes:	yes
2015-02-18 03:54:54 +00:00
Simon J. Gerraty
9268022b74 Merge from head@274682 2014-11-19 01:07:58 +00:00
Baptiste Daroussin
8447c44fe7 The dependencies are computed with CC even if sources are C++, when building
when building with an external gcc, we want to be able to pass the path to
the libc++ headers so dependencies are correctly computed for C++ source files.
Add a DEPFLAGS for that purpose

Reviewed by:	imp
2014-10-21 19:56:45 +00:00
Mark Johnston
e9083b2dc5 Fix a few small bugs in the DTrace USDT rules:
* anchor search strings appropriately,
* use .ALLSRC to pass the full path to the D script to dtrace(1),
* don't insert the auto-generated header into SRCS - it doesn't
  accomplish anything, and we end up having to remove it from OBJS anyway.

Reviewed by:		rpaulo
Differential Revision:	https://reviews.freebsd.org/D978
MFC after:		3 weeks
Sponsored by:		EMC / Isilon Storage Division
2014-10-21 04:30:00 +00:00
Mark Johnston
721dae3a52 Correct the way that libelf is linked when USDT is used. 2014-10-01 04:28:39 +00:00
Enji Cooper
41ae64f374 Remove many false positives with make checkdpadd
- Reduce DPADD and LDADD in checkdpadd to -l<foo>
- Skip over -Wl,[es]*-group because -Wl,--end-group and
  -Wl,--start-group might be required to properly link objects (see
  usr.bin/clang/lldb as an example)

This caveat has been present for a while with some components of
the build. However, these false positives were made more more apparent
after r269648.

Phabric: D635
Reviewed by: jmmv (an earlier version)
PR: 192730
MFC after: 2 weeks
2014-09-10 07:55:51 +00:00
Mark Johnston
5439a71566 Define _DTRACE_VERSION in sdt.h rather than setting it manually. This is
similar to what illumos does, and makes it easier to enable USDT probes
in third-party software that doesn't make use of the system makefiles.
2014-09-02 23:43:06 +00:00
Simon J. Gerraty
ee7b0571c2 Merge head from 7/28 2014-08-19 06:50:54 +00:00
Rui Paulo
9c851217b6 Make sure the DTrace header files are built before depend and before
the build starts.

This adds a new variable DHDRS that contains a list of all DTrace
header files.  Then, we use the beforedepend hook to make sure the
heaeder files are built.

Introduce a beforebuild dependency (from projects/bmake) based on
feedback from Simon J. Gerraty.  This lets us generate the header
files without running make depend.

Reviewed by:	sjg, imp
MFC after:	3 days
2014-08-13 01:27:51 +00:00
Rui Paulo
17b227229c Revert r269839.
This doesn't work like I expected and the if clause is never matched.
It turns out that the problem with the older make is unrelated.

MFC after:	3 days
2014-08-11 22:43:44 +00:00
Rui Paulo
8a152f97e7 Add .d files to the reverse matching pattern in bsd.dep.mk to account
for source file dependencies.

This follows the .y and .l behaviour.

MFC after:	3 days
2014-08-11 21:42:06 +00:00
Rui Paulo
6b2edf61bd Fix another typoe in the definition of DTrace SOBJS.
MFC after:	3 days
2014-08-11 21:14:08 +00:00
Rui Paulo
04c241ea2d bsd.dep.mk: use the empty() function with the check for .d files.
Seems to work better with the old make.

MFC after:	3 days
2014-08-11 20:44:57 +00:00
Rui Paulo
23e9784542 Fix an error in the dependency of the DTrace object file.
MFC after:	3 days
2014-08-11 20:41:14 +00:00
Rui Paulo
9b646f2533 Fix a few problems with the USDT probes:
* Include OBJDIR to make sure the generated file is found;
* Simplify the definition of OBJS;
* Add targets for shared objects and for profiled objects.

MFC after:	3 days
2014-08-10 06:43:40 +00:00
Dag-Erling Smørgrav
48b0feca16 Older versions of flex (like the one used in FreeBSD <= 9) expect the
output file name to follow the -o flag immediately, with no intervening
whitespace.

Reported by:	ian@
MFC after:	1 week
2014-07-20 14:49:24 +00:00
Dag-Erling Smørgrav
8f812fec9f Use -o (output to file) instead of -t (print to stdout) and a redirect.
MFC after:	1 week
2014-07-19 14:26:04 +00:00
Rui Paulo
d8b631554e Optimisation: replace ${_D:R} with just ${_D}.
MFC after:	3 weeks
2014-07-12 01:06:29 +00:00
Rui Paulo
ff0f36e773 Automatically build and link DTrace USDT probes.
This completely replaces bsd.dtrace.mk which never worked reliably.

MFC after:	3 weeks
2014-07-12 00:54:34 +00:00
Rui Paulo
9f65e34ff8 bsd.dep.mk: add support for generating DTrace header files from .d files.
MFC after:	3 weeks
2014-07-05 20:08:35 +00:00
Simon J. Gerraty
23090366f7 Sync from head 2012-11-04 02:52:03 +00:00
Marcel Moolenaar
9e0a344174 Revert r241752. The claim that it's more correct while at the same time
causing a build breakage is interesting to say the least.
2012-10-20 22:44:22 +00:00
Marcel Moolenaar
9120686391 Improve upon the previous commit to fix the yacc rule.
1.  Have the resulting C file depend on the resulting H
    file as it should be. Touch the C file to make sure
    the C file is newer than the H file to keep make
    happy.
2.  Apply the same fix to the other instance of .ORDER,
    missed in the previous commit.
2012-10-19 20:13:08 +00:00
Marcel Moolenaar
0815243c39 Add support for bmake. This includes:
1.  Don't do upgrade_checks when using bmake. As long as we have WITH_BMAKE,
    there's a bootstrap complication in ths respect. Avoid it. Make the
    necessary changes to have upgrade_checks work wth bmake anyway.
2.  Remove the use of -E. It's not needed in our build because we use ?= for
    the respective variables, which means that we'll take the environment
    value (if any) anyway.
3.  Properly declare phony targets as phony as bmake is a lot smarter (and
    thus agressive) about build avoidance.
4.  Make sure CLEANFILES is complete and use it on .NOPATH. bmake is a lot
    smarter about build avoidance and should not find files we generate in
    the source tree. We should not have files in the repository we want to
    generate, but this is an easier way to cross this hurdle.
5.  Have behavior under bmake the same as it is under make with respect to
    halting when sub-commands fail. Add "set -e" to compound commands so
    that bmake is informed when sub-commands fail.
6.  Make sure crunchgen uses the same make as the rest of the build. This
    is important when the make utility isn't called make (but bmake for
    example).
7.  While here, add support for using MAKEOBJDIR to set the object tree
    location. It's the second alternative bmake looks for when determining
    the actual object directory (= .OBJDIR).

Submitted by:	Simon Gerraty <sjg@juniper.net>
Submitted by:	John Van Horne <jvanhorne@juniper.net>
2012-10-06 20:01:05 +00:00
Dimitry Andric
2d12214697 Make sure bsd.dep.mk does not filter out -stdlib=xxx from CXXFLAGS,
since this determines parts of the C++ include path.

MFC after:	1 week
2012-08-23 17:03:33 +00:00
Marcel Moolenaar
7750ad47a9 Sync FreeBSD's bmake branch with Juniper's internal bmake branch.
Requested by: Simon Gerraty <sjg@juniper.net>
2012-08-22 19:25:57 +00:00
David Schultz
f4b36cb777 Also pass the -ansi flag through to mkdep (requested by bde), and fix
a cut-and-paste-o (noticed by rdivacky).
2011-10-11 05:17:26 +00:00
David Schultz
085f20ec39 Pass -std= flags in CFLAGS to mkdep. Without this, preprocessor tests
for particular compiler features might be evaluated differently by
mkdep than they would be by cc.
2011-10-10 15:39:29 +00:00
Dimitry Andric
58ff0f42ba Remove support for the Intel C Compiler from the build infrastructure.
This support has not worked for several years, and is not likely to work
again, unless Intel decides to release a native FreeBSD version of their
compiler. ;)
2011-04-19 18:09:21 +00:00
Dimitry Andric
6d67f69487 Remove libobjc and other Objective-C related components, as these are
extremely outdated, and not used by anything in the base system.

Silence from:	current@
2011-04-17 21:03:23 +00:00
Rui Paulo
187278cadc For every instance of '.if ${CC} == "foo"' or '.if ${CC} != "foo"' in
Makefiles or *.mk files, use ${CC:T:Mfoo} instead, so only the basename
of the compiler command (excluding any arguments) is considered.

This allows you to use, for example, CC="/nondefault/path/clang -xxx",
and still have the various tests in bsd.*.mk identify your compiler as
clang correctly.

ICC if cases were also changed.

Submitted by:	Dimitry Andric <dimitry at andric.com>
2010-08-17 20:39:28 +00:00
Rui Paulo
efdcc3d8d8 Make sure any -U CFLAGS are not filtered out when calling mkdep, as
gnu/usr.bin/cc/libiberty first defines the PREFIX macro, and then
undefines it again, so mkdep with clang complains.

Submitted by:	Dimitry Andric <dimitry at andric.com>
Reviewed by:	ru
2010-08-17 13:02:08 +00:00
Ruslan Ermilov
213a44479c Allow the CTAGS to be set to something other than "gtags" or "ctags",
but assume it supports a ctags(1)-compatible syntax.

PR:		46676
Submitted by:	Lyndon Nerenberg
MFC after:	3 days
2010-01-18 15:58:02 +00:00
Ruslan Ermilov
2f82e3b16d If CTAGS is not set or set to something other than "ctags" or "gtags",
"cleandepend" was not removing the .depend file; fixed.

PR:		126747
MFC after:	3 days
2010-01-18 15:41:55 +00:00
Dag-Erling Smørgrav
5fb6bed92c Revert CLEANDEPFILES commit per ru@'s request; it does not really solve
the problem.  The correct fix will follow.
2008-02-05 08:52:36 +00:00
Dag-Erling Smørgrav
0d7cc1d0eb Normally, when a header file is removed from the build (as i4b headers
were recently), a simple 'make cleandepend; make depend' is sufficient
to keep the tree buildable after a cvs update when doing incremental
builds.

However, kdump and truss use a script which searches for header files
that define ioctls, and generates C code that includes them.  This
script will usually not need updating when a header file is removed,
so the normal dependency mechanism will not realize that it needs to
be re-run.  One is therefore left with code that references dead files
but will only be removed by a full 'make clean', which defeats the
purpose of incremental builds.

To work around this, modify the cleandepend target in bsd.dep.mk to
also remove any files listed in a new variable named CLEANDEPFILES,
and modify kdump's and truss's Makefiles accordingly.

MFC after:	2 weeks
2008-02-03 11:34:56 +00:00
Kirill Ponomarev
d966048e7e Remove trailing spaces. 2005-01-06 11:12:43 +00:00
Ruslan Ermilov
6260f1d8ee NOTAGS -> NO_TAGS 2004-12-21 09:43:25 +00:00
Tom Rhodes
06d6e4fcfe This are the build infrastructure changes to allow to use the
Intel C/C++ compiler (lang/icc) to build the kernel.

The icc CPUTYPE CFLAGS use icc v7 syntax, icc v8 moans about them, but
doesn't abort. They also produce CPU specific code (new instructions
of the CPU, not only CPU specific scheduling), so if you get coredumps
with signal 4 (SIGILL, illegal instruction) you've used the wrong
CPUTYPE.

Incarnations of this patch survive gcc compiles and my make universe.
I use it on my desktop.

To use it update share/mk, add
	/usr/local/intel/compiler70/ia32/bin	(icc v7, works)
or
	/usr/local/intel_cc_80/bin		(icc v8, doesn't work)
to your PATH, make sure you have a new kernel compile directory
(e.g. MYKERNEL_icc) and run
	CFLAGS="-O2 -ip" CC=icc make depend
	CFLAGS="-O2 -ip" CC=icc make
in it.

Don't compile with -ipo, the build infrastructure uses ld directly to
link the kernel and the modules, but -ipo needs the link step to be
performed with Intel's linker.

Problems with icc v8:
 - panic: npx0 cannot be emulated on an SMP system
 - UP: first start of /bin/sh results in a FP exception

Parts of this commit contains suggestions or submissions from
Marius Strobl <marius@alchemy.franken.de>.

Reviewed by:	silence on -arch
Submitted by:	netchild
2004-03-12 21:36:12 +00:00
Ruslan Ermilov
65d2bdc62c Fixed bogus ${FOO:Mbar} tests where the actual intent is to check
if the result set is empty.  While here, replaced non-bogus empty
string comparisons with equivalent empty() checks.
2004-01-27 23:22:15 +00:00
Ruslan Ermilov
6b14fdfa52 Guess better the source for object files in case .depend file is
missing and there are multiple choices using multiple inference
(suffix transformation) rules.

This is known to fix compilation of s_log1p.o in lib/msun on i386,
as otherwise it attempted to use s_log1p.S as the source (which is
marked broken) instead of legal s_log1p.c which is in CFLAGS.  The
normal case where .depend file exists is not affected.

Reviewed by:	bde
2004-01-17 18:51:55 +00:00
Ruslan Ermilov
11dc929821 Introduce DPSRCS, that holds the list of source files which are
needed for generating dependencies.  SRCS are always part of it,
and normally only they.

This can be useful in some random cases where it's necessary to
have something in .depend that isn't part of SRCS.  This will be
used to replace a hack in lib/libpam/libpam/Makefile.
2004-01-13 17:37:45 +00:00
Ruslan Ermilov
484933a925 - Replaced several := ops with their += equivalents; the old
form became unnecessary with the bsd.prog.mk,v 1.69 change.

- Eliminated duplicate y.tab.h in SRCS.

Reviewed by:	bde

- Complementary to the said bsd.prog.mk change, use the fact
  that inner .for loops are not real loops but a tricky form
  of a local macro for the outer loop's variable, and switch
  to using faster variable modifiers to replace extensions.
2004-01-12 15:29:47 +00:00
Bruce Evans
2e3c617921 Use make(1) instead of a shell script to implement the checkdpadd target.
This is simpler, and is easy to do now that make(1) supports substituting
regexps.  Fixed missing '$' anchor in the regexp.  Use less cryptic names
for temporary variables.

Submitted by:	ru (early version)
Reviewed by:	ru
2003-07-03 11:43:57 +00:00
Ruslan Ermilov
6319fd15c9 The .s files do not have to be preprocessed with cpp(1). 2003-07-02 16:43:07 +00:00
Ruslan Ermilov
a51613b6da There's no reason to keep separate AINC knob anymore.
The only real use of it (lib/libc/Makefile) has been
fixed, and if necessary, the contents of AINC should
be added to CFLAGS.

Explained by:	bde
2003-07-01 15:15:45 +00:00
Ruslan Ermilov
ccc4bab104 Added the new variable CTAGS which, if set to "ctags", reverts
to creating the tags file using ctags(1).  Defaults to "gtags".
Made GTAGSFLAGS and HTAGSFLAGS overrideable, added CTAGSFLAGS.
Folded bsd.prog.mk version of `tags' into bsd.dep.mk.

PR:		bin/42852
2002-10-17 13:48:13 +00:00
Peter Wemm
66422f5b7a Initiate deorbit burn for the i386-only a.out related support. Moves are
under way to move the remnants of the a.out toolchain to ports.  As the
comment in src/Makefile said, this stuff is deprecated and one should not
expect this to remain beyond 4.0-REL.  It has already lasted WAY beyond
that.

Notable exceptions:
gcc - I have not touched the a.out generation stuff there.
ldd/ldconfig - still have some code to interface with a.out rtld.
old as/ld/etc - I have not removed these yet, pending their move to ports.
some includes - necessary for ldd/ldconfig for now.

Tested on: i386 (extensively), alpha
2002-09-17 01:49:00 +00:00
Ruslan Ermilov
6e02167b57 Moved checkdpadd target to where it logically belongs. 2002-07-03 12:44:06 +00:00
Bruce Evans
732757a74c Fixed missing quotes in the default for MKDEPCMD in the CC != cc case.
"CC='cc -Dfoo' make depend" was very broken.
2002-05-07 00:15:53 +00:00
Ruslan Ermilov
b25a566d47 Merge bsd.obj.mk's version of the _SUBDIR target with bsd.subdir.mk.
Ensure all standard targets honor SUBDIR.  Now `make obj' descends into
SUBDIRs even if NOOBJ is set (some descendants may still need an object
directory, but we do not have such precedents).  Now `make install' in
non-bsd.subdir.mk makefiles runs `afterinstall' target _after_ `install'
in SUBDIRs, like we do in bsd.subdir.mk.  Nothing depended on the wrong
order anyway.

Fixed `distribute' targets (except for the bsd.subdir.mk version) so that
they do not depend on _SUBDIR; `distribute' calls `install' which already
depends on _SUBDIR.

De-standardize `maninstall', otherwise manpages would be installed twice.
(To be revised later.)
2002-04-23 09:03:56 +00:00
Ruslan Ermilov
5ffdf3618e Prohibit the direct inclusion of bsd.dep.mk. 2002-04-22 10:04:41 +00:00
Ruslan Ermilov
70e82473a4 Doh, previous revision made .depend to be almost always out of date.
Spotted by:	bde

Fortunately, we have a powerful make(1).  Apply some black magic to
make it DTRT.  (Better viewed as diff to revision 1.30.)
2002-04-17 05:42:18 +00:00
Ruslan Ermilov
0cfacc8666 Don't spawn extra make(1) process to execute _EXTRADEPEND. 2002-04-16 12:27:07 +00:00
Ruslan Ermilov
e16529d335 Only pass our idea of `CC' down to mkdep(1) if it's non-default. 2002-04-15 15:23:45 +00:00
Bruce Evans
afabc4dfc2 Removed bogus env(1) commands. make(1) uses a real shell, so
"env name=value ... cmd ..." was just a pessimized way of doing
"name=value ... cmd ...".  Note that make(1) can't optimize
either of these to an exec of env(1) or "cmd" even if the second
"..." is simple, since it can't tell that the shell metacharacter
in "name=value" is actually handled by env(1).
2002-03-19 06:11:43 +00:00
David E. O'Brien
9722145f6f Pass our idea of `CC' down to mkdep.
Tested on:	sparc64
2002-03-15 18:43:59 +00:00
Peter Wemm
7f3dea244c $Id$ -> $FreeBSD$ 1999-08-28 00:22:10 +00:00
Hidetoshi Shimokawa
f9ed7e775f Clean new index file(GPATH) of global.
Submitted by:	Shigio Yamaguchi <shigio@wafu.netgate.net>
1999-01-18 07:48:41 +00:00
Dmitrij Tejblum
c7cbe79e18 Recognize ".cpp" as a C++ suffix.
PR:		3476
1998-06-05 18:38:55 +00:00
Bruce Evans
5d9599c863 Oops, forgot references in previous commit.
Submitted by:	Brian Cully <shmit@erols.com>
PR:		6178
1998-05-15 09:34:48 +00:00
Bruce Evans
70a6e3314e Support Objective C almost as well as C++.
Notes:
- We no longer use -fgnu-runtime in bsd.lib.mk, since it is the default
  and bsd.lib.mk is the wrong place to override it.
- Gnu C doesn't have a special compiler driver for Objective C like it
  does for C++.  The defaults are suitable for Gnu C.  Use `OBJCLIBS='
  in /etc/make.conf for POC.
1998-05-15 09:30:13 +00:00
Bruce Evans
2ec8b6deea Oops, the lex and yacc sources must be added to SRCS before ${SRCS}
is used in the dependency list for ${DEPENDFILE}.  `make depend' was
broken for a few days.  `make world' only uses `make depend' when
NOCLEAN is defined, so only a few people noticed the bug.

Submitted by:	mostly by jmg
1998-05-11 15:37:13 +00:00
Bruce Evans
29901fcc76 Moved lex and yacc source handling from bsd.prog.mk to bsd.dep.mk
so that it applies to libraries (and lkms) as well as programs.
1998-05-08 06:31:05 +00:00
Bruce Evans
5a9eee48b6 Only include bsd.dep.mk in `mk'files that handle C sources. Abuse
bsd.obj.mk instead of bsd.dep.mk for defining the _SUBDIR target
and a default tags target.  Abuse bsd.obj.mk for defining default
cleandepend and depend targets.
1998-05-06 18:44:04 +00:00
Eivind Eklund
1ed9b1c368 Support new version of global.
Submitted by:	Shigio Yamaguchi <shigio@wafu.netgate.net>
Tested by:	make buildworld
1998-03-23 14:58:30 +00:00
Bruce Evans
a813856e54 Pass -nostdinc to mkdep for assembler and C files (it was already
passed for C++ files).

Don't pass -nostdlib to mkdep for C++ files.

Don't use temporary variables to obfuscate the main rule.
1998-03-07 13:57:37 +00:00
Bruce Evans
b7052cd110 Fixed cleaning of tags files. GSYMS was forgotten. 1998-02-20 14:32:30 +00:00