This reworks r289254 and removes ALL_SUBDIR_TARGETS.
Because there is an include guard in this file there is no need for
LOCAL_ or ?= on SUBDIR_TARGETS or STANDALONE_SUBDIR_TARGETS. These can
just be set via src.conf. By the time bsd.subdir.mk is included it will
just append the values to the existing value and work fine. This allows
a consistent way to append to these variables without introducing a
LOCAL_ var for STANDALONE_SUBDIR_TARGETS or renaming the historical
SUBDIR_TARGETS.
Sponsored by: EMC / Isilon Storage Division
My changes in r291635 broke 'make install*' for DIRDEPS_BUILD but also
revealed that some other targets were not guaranteed to be created if
there was a SUBDIR defined. One example is 'installfiles' was never
defined if SUBDIR was not empty.
Sponsored by: EMC / Isilon Storage Division
The problem was that 'afterinstall' was not coming after SUBDIRs were
installed which was the expectation at least in sys/modules for kldxref.
Reported by: np
Pointyhat to: bdrewery
Sponsored by: EMC / Isilon Storage Division
This is to fix 'make all' causing it to recurse on both 'all' and 'buildconfig'
due to 'buildconfig' being in ALL_SUBDIR_TARGETS and being a dependency of
'all'.
This now adds all of the '*includes', '*files' targets as subdir targets,
allowing them to recurse.
This also removes the need for some 'realinstall' hacks in bsd.subdir.mk since
it no longer recurses; only 'install' will recurse and call the proper
'beforeinstall', 'realinstall', and 'afterinstall' in each sub-directory.
This fixes 'make includes' and 'make files' to not be a rerolled ${MAKE}
sub-shell but to rather just recurse on 'inclues' and 'files'. This avoids
various issues such as the one fixed in r289462. As such revert Makefile.inc1
back to using 'includes' which avoids an extra tree walk and parallelizes
the includes phases better.
Makefile.inc1 includes a guard so that 'make all' will not use SUBDIR_PARALLEL,
added in r289438. This is so users do not get a probably broken build if they
run 'make all' from the top-level. Before the change in this commit, the
workaround for 'make everything' was 'par-all' which would depend on 'all' and
cause a proper parallel recursion. Now that will not work so a new
_PARALLEL_SUBUDIR_OK is used to allow it.
This is still part of an effort to combine bsd.(files|incs|confs).mk and move
some of its logic out of bsd.subdir.mk, as attempted in r289282 and reverted in
r289331. This commit fixes the problems found there which was mostly double
recursing during 'includes' which would recurse on itself and 'buildincludes'
and 'installincludes', all in parallel. The logic is still in bsd.subdir.mk
for now.
I've been cautious about this commit but have experienced no breakage on the
tree except for the 'par-all' case which was already a hack. If something foo
is depending on something bar that should recurse, it is very likely that the
foo target is being recursed on already meaning that bar will still effectively
recurse once sub-directories call foo.
Discussed on: arch@
MFC after: never
Sponsored by: EMC / Isilon Storage Division
This is to fix 'make all' causing it to recurse on both 'all' and 'buildconfig'
due to 'buildconfig' being in ALL_SUBDIR_TARGETS and being a dependency of
'all'.
This now adds all of the '*includes', '*files' targets as subdir targets,
allowing them to recurse.
This also removes the need for some 'realinstall' hacks in bsd.subdir.mk since
it no longer recurses; only 'install' will recurse and call the proper
'beforeinstall', 'realinstall', and 'afterinstall' in each sub-directory.
This fixes 'make includes' and 'make files' to not be a rerolled ${MAKE}
sub-shell but to rather just recurse on 'inclues' and 'files'. This avoids
various issues such as the one fixed in r289462. As such revert Makefile.inc1
back to using 'includes' which avoids an extra tree walk and parallelizes
the includes phases better.
Makefile.inc1 includes a guard so that 'make all' will not use SUBDIR_PARALLEL,
added in r289438. This is so users do not get a probably broken build if they
run 'make all' from the top-level. Before the change in this commit, the
workaround for 'make everything' was 'par-all' which would depend on 'all' and
cause a proper parallel recursion. Now that will not work so a new
_PARALLEL_SUBUDIR_OK is used to allow it.
This is still part of an effort to combine bsd.(files|incs|confs).mk and move
some of its logic out of bsd.subdir.mk, as attempted in r289282 and reverted in
r289331. This commit fixes the problems found there which was mostly double
recursing during 'includes' which would recurse on itself and 'buildincludes'
and 'installincludes', all in parallel. The logic is still in bsd.subdir.mk
for now.
I've been cautious about this commit but have experienced no breakage on the
tree except for the 'par-all' case which was already a hack. If something foo
is depending on something bar that should recurse, it is very likely that the
foo target is being recursed on already meaning that bar will still effectively
recurse once sub-directories call foo.
Discussed on: arch@
MFC after: never
Sponsored by: EMC / Isilon Storage Division
This allows META_FILES option to be renamed META_MODE.
Also add META_COOKIE_TOUCH for use in targets that can benefit
from a cookie when in meta mode.
Differential Revision: https://reviews.freebsd.org/D4153
Reviewed by: bdrewery
Using .USEBEFORE had the unintended side-effect of changing the directory for
the real target ran in the current directory. For example this meant that
the 'make clean' would run in one of the SUBDIR.
Sponsored by: EMC / Isilon Storage Division
Pointyhat to: bdrewery
- Rather than allow 'make clean*' to ignore dependencies, make a static
list of targets in STANDALONE_SUBDIR_TARGETS that are known to be safe.
This allows a user to override them if needed and avoids adding this feature
to user-defined targets that are in ${SUBDIR_TARGETS}. [1]
- This now also allows to force SUBDIR_PARALLEL when calling these
targets, since no dependencies are needed.
Reported by: ian [1]
Sponsored by: EMC / Isilon Storage Division
MFC after: 3 weeks
X-MFC-With: r289778
This reverts r266473 as the need for it, working around .MAKE and '+'
issues, is no longer needed after r289460. This avoids extra log
output in -j builds of '-- _sub.TARGET --' that are redundant with the
'-- TARGET --' and '-- TARGET_subdir_DIR --' entries already showing.
r266473 also made a subtle change in the ordering of _SUBDIR handling. Before
the change, SUBDIRS were recursed into after building the TARGET due to the
.USE of _SUBDIR *appending* the commands onto the TARGET. After the change
though the indirection caused TARGET to depend on _sub.TARGET which had the
_SUBDIR handling in it. This TARGET would run after recursing. However, the
SUBDIR_PARALLEL handling from r263778 has this ordering as well. Since
this has so far not been a problem, for now make this behavior for
non-SUBDIR_PARALLEL use of _SUBDIR explicit by using .USEBEFORE.
Further research may change this back to .USE as well as the
SUBDIR_PARALLEL handling and bsd.progs.mk recursing.
Sponsored by: EMC / Isilon Storage Division
The 'config' target isn't really needed right now so just remove it to avoid
any clashes with config(8) building. It's also likely misspelled and should
be 'configs' if we decide to add it back. This was just a convenience
target recently added.
Sponsored by: EMC / Isilon Storage Division
more typical ALL_SUBDIR_TARGETS entries and target hooks in bsd.incs.mk,
bsd.files.mk and bsd.confs.mk.
This allows the targets to be NOPs if unneeded and still work with the
shortcut 'make includes' to build and then install in a parallel-safe manner.
Sort and re-indent the ALL_SUBDIR_TARGETS with the new entries.
Sponsored by: EMC / Isilon Storage Division
Also improve documentation.
The SUBDIR_TARGETS variable should really be named LOCAL_SUBDIR_TARGETS, but
renaming it may be a surprise for downstream vendors who use this variable.
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
Before this, the target was unknown. Now it will recurse on subdirs and run
the target in the current directory. It is required to recurse as there
may be subdirs that have objs in their directory or in the object directory,
so it is not enough to just delete the objdir of the subdir parent.
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
When bmake was initially imported at r241298 shell commands were no longer
ran with 'set -e' as they were before. This was fixed in r254980 so they
again always use 'set -e'.
Sponsored by: EMC / Isilon Storage Division
remove duplicates. We cannot sort SUBDIR because many Makefiles
have .WAIT in the list which is strongly ordering. Rather than
try to second guess when to sort and when to not sort depending
on .WAIT being in the list, just remove duplicates.
during SUBDIR_PARALLEL builds. This augments the coarse .WAIT mechanism,
which is still useful if you've got a situation such as "almost everything
depends on A and B".
Because the parallel subdir mechanism uses non-obvious mangling of
target names, which should probably remain a private detail of the
implementation, it's not easy to do things like "libfoo: libbar", so
instead the new mechanism lets you set a variable that lists dependencies:
SUBDIR_DEPEND_libfoo= libgroodah libpouet
Note that while I'm using libraries as an example here, it really has
nothing to do with the generated library files. This is really saying
"build in directory libfoo after building in the libgroodah and libpouet
directories."
This updates lib/Makefile with dependency information based on the old
almost-accurate comment block and by combing through lib/* makefiles
looking for LDADD dependencies to other libraries within lib/*.
Reviewed by: Jia-Shiun Li <jiashiun@gmail.com>
Force all the contents of /usr/tests to go into a separate distribution
file so that users of binary releases can easily choose to not install it.
To make this possible, we need two fixes:
- bsd.subdir.mk needs to properly honor NO_SUBDIR in all cases so that we
do not recurse into 'tests' subdirectories when we needn't. Otherwise,
we end up with some Kyuafiles in base.txz.
- etc/Makefile needs to skip installing tests in its 'distribute' target
so that a Kyuafile doesn't leak into base.txz.
Approved by: gjb
this allows make -n to do tree walks as expected without
doing anything else (as intended).
Use prefix _sub. to help avoid conflict with any real target.
Reviewed by: imp
Targets thus marked are supposed to run even with -n.
As such they should not do anything except run the sub-make.
Use an intermediate target _* to associate with _SUBDIR and which
depends on installincludes etc so that we get the correct behavior with -n.
Reviewed by: marcel
all the SUBDIR entries in parallel, instead of serially. Apply this
option to a selected number of Makefiles, which can greatly speed up the
build on multi-core machines, when using make -j.
This can be extended to more Makefiles later on, whenever they are
verified to work correctly with parallel building.
I tested this on a 24-core machine, with make -j48 buildworld (N = 6):
before stddev after stddev
======= ====== ======= ======
real time 1741.1 16.5 959.8 2.7
user time 12468.7 16.4 14393.0 16.8
sys time 1825.0 54.8 2110.6 22.8
(user+sys)/real 8.2 17.1
E.g. the build was approximately 45% faster in real time. On machines
with less cores, or with lower -j settings, the speedup will not be as
impressive. But at least you can now almost max out a machine with
buildworld!
Submitted by: jilles
MFC after: 2 weeks
1. Don't do upgrade_checks when using bmake. As long as we have WITH_BMAKE,
there's a bootstrap complication in ths respect. Avoid it. Make the
necessary changes to have upgrade_checks work wth bmake anyway.
2. Remove the use of -E. It's not needed in our build because we use ?= for
the respective variables, which means that we'll take the environment
value (if any) anyway.
3. Properly declare phony targets as phony as bmake is a lot smarter (and
thus agressive) about build avoidance.
4. Make sure CLEANFILES is complete and use it on .NOPATH. bmake is a lot
smarter about build avoidance and should not find files we generate in
the source tree. We should not have files in the repository we want to
generate, but this is an easier way to cross this hurdle.
5. Have behavior under bmake the same as it is under make with respect to
halting when sub-commands fail. Add "set -e" to compound commands so
that bmake is informed when sub-commands fail.
6. Make sure crunchgen uses the same make as the rest of the build. This
is important when the make utility isn't called make (but bmake for
example).
7. While here, add support for using MAKEOBJDIR to set the object tree
location. It's the second alternative bmake looks for when determining
the actual object directory (= .OBJDIR).
Submitted by: Simon Gerraty <sjg@juniper.net>
Submitted by: John Van Horne <jvanhorne@juniper.net>
cleanilinks wasn't listed in <bsd.subdir.mk>. Instead of adding it to
/sys/modules/Makefile, we'd better just add it to <bsd.subdir.mk>
directly, so we don't need to change files like /sys/modules/sound/Makefile
as well. This means you can finally clean up all those dangling symlinks
created by individual module compilation at once.
to `:', so that it stays compatible with a stale dependency recorded in
.depend when the type of "foo" changes from file to directory or back.
Compensate for the loss of the "If no sources are specified, the target
is always re-created" feature by marking these targets with the .PHONY
attribute. While here, fix a bug in the target's script (nobody uses
these targets apparently).
additional targets that will cause descending into subdirs.
Example:
cd /sys/modules; make load SUBDIR_TARGETS=load
(But don't try it with your pet.)
Submitted by: Alexey Klimov
PR: 47601
the US Senate, Canadian Parliament and Australian Senate, it was
causing some confusion. After some consultation with Mark Murray,
change this to 'without objection' since often times a plain-speaking
term is preferable to a regionally used term.
Also, clarify that this procedure is to be used when for more mundane
matters that need a sanity check, but don't need the whole, ponderous
voting proceedure that more difficult issues require. Core members
that read email in any given 48 hour period are trusted enough to know
the difference and to provide the sanity check as necessary.
Reviewed by: markm
given -n. For POLA reasons this behaviour is switched on only when
at least two -n flags are given to make. One -n flag keeps the old behaviour
of showing the shell command that would recurse into the sub-directories.
Discussed with: ru
