d/freebsd-skq
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Rework the world subdir build targets to use the standard SUBDIR_PARALLEL mechanism.

Browse Source 
Back in r30113, the 'par-*' targets were added to parallelize portions of
the build in a very similar fashion as the SUBDIR_PARALLEL feature used in
r263778.  Calling a target without 'par-' (for 'parallel') resulted in the
standard bsd.subdir.mk handling without parallelization.  Given we have
SUBDIR_PARALLEL now there is no reason to duplicate the handling here.

In build logs this will result in the ${dir}.${target}__D targets now showing
as the normal ${target}_subdir_${dir} targets.

I audited all of the uses of Makefile.inc1 and Makefile's targets that use
bsd.subdir.mk and found that all but 'all' and 'install' were fine to use
as always parallel.
  - For 'install' (from installworld -j) the ordering of lib/ and libexec/
    before the rest of the system (described in r289433), and etc/ being last
    (described in r289435), is all that matters.  So now a .WAIT is added in
    the proper places when invoking any 'install*' target.  A parallel
    installworld does work and took 46% of the time a non-parallel
    install would take on my system with -j15 to ZFS.
  - For 'all' I left the default handling for this to not run in parallel.  A
    'par-all' target is still used by the 'everything' stage of buildworld
    to continue building in parallel as it already has been.  This works
    because most of the dependencies are handled by the early bootstrap
    phases as well as 'libraries' and 'includes' phases.  This lets
    all of the SUBDIR build in parallel fine, such as bin/ and lib/.  This
    will not work if the user invokes 'all' though as we have dependencies
    spread all over the system with no way to depend between them (except
    for the dirdeps feature in the META_MODE build).  Calling 'make all'
    from the top-level is still useful at least when using SUBDIR_OVERRIDE.

MFC after:	2 weeks
Sponsored by:	EMC / Isilon Storage Division
This commit is contained in:
Bryan Drewery 2015-10-17 03:51:50 +00:00
parent 23ffbc1011
commit fb84a99e0f
1 changed files with 25 additions and 22 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

47
Makefile.inc1
View File

@ -63,6 +63,10 @@ SRCDIR?= ${.CURDIR}
SUBDIR= ${SUBDIR_OVERRIDE}
.else
SUBDIR= lib libexec
.if make(install*)
# Ensure libraries are installed before progressing.
SUBDIR+=.WAIT
.endif
SUBDIR+=bin
.if ${MK_CDDL} != "no"
SUBDIR+=cddl
@ -115,6 +119,9 @@ SUBDIR+= ${_DIR}
# by calling 'makedb' in share/man. This is only relevant for
# install/distribute so they build the whatis file after every manpage is
# installed.
.if make(install*)
SUBDIR+=.WAIT
.endif
SUBDIR+=etc
.if defined(NOCLEAN)
@ -589,9 +596,9 @@ _cleanobj:
@echo "--------------------------------------------------------------"
@echo ">>> stage 2.1: cleaning up the object tree"
@echo "--------------------------------------------------------------"
${_+_}cd ${.CURDIR}; ${WMAKE} ${CLEANDIR:S/^/par-/}
${_+_}cd ${.CURDIR}; ${WMAKE} ${CLEANDIR}
.if defined(LIB32TMP)
${_+_}cd ${.CURDIR}; ${LIB32WMAKE} -f Makefile.inc1 ${CLEANDIR:S/^/par-/}
${_+_}cd ${.CURDIR}; ${LIB32WMAKE} -f Makefile.inc1 ${CLEANDIR}
.endif
.endif
_obj:
@ -599,7 +606,7 @@ _obj:
@echo "--------------------------------------------------------------"
@echo ">>> stage 2.2: rebuilding the object tree"
@echo "--------------------------------------------------------------"
${_+_}cd ${.CURDIR}; ${WMAKE} par-obj
${_+_}cd ${.CURDIR}; ${WMAKE} obj
_build-tools:
@echo
@echo "--------------------------------------------------------------"
@ -618,7 +625,7 @@ _includes:
@echo "--------------------------------------------------------------"
@echo ">>> stage 4.1: building includes"
@echo "--------------------------------------------------------------"
${_+_}cd ${.CURDIR}; ${WMAKE} SHARED=symlinks par-includes
${_+_}cd ${.CURDIR}; ${WMAKE} SHARED=symlinks includes
_libraries:
@echo
@echo "--------------------------------------------------------------"
@ -632,7 +639,7 @@ _depend:
@echo "--------------------------------------------------------------"
@echo ">>> stage 4.3: make dependencies"
@echo "--------------------------------------------------------------"
${_+_}cd ${.CURDIR}; ${WMAKE} par-depend
${_+_}cd ${.CURDIR}; ${WMAKE} depend
everything:
@echo
@echo "--------------------------------------------------------------"
@ -1944,22 +1951,18 @@ _startup_libs: ${_startup_libs:S/$/__L/}
_prebuild_libs: ${_prebuild_libs:S/$/__L/}
_generic_libs: ${_generic_libs:S/$/__L/}
.for __target in all clean cleandepend cleandir depend includes obj
.for entry in ${SUBDIR}
${entry}.${__target}__D: .PHONY .MAKE
${_+_}@set -e; if test -d ${.CURDIR}/${entry}.${MACHINE_ARCH}; then \
${ECHODIR} "===> ${DIRPRFX}${entry}.${MACHINE_ARCH} (${__target})"; \
edir=${entry}.${MACHINE_ARCH}; \
cd ${.CURDIR}/$${edir}; \
else \
${ECHODIR} "===> ${DIRPRFX}${entry} (${__target})"; \
edir=${entry}; \
cd ${.CURDIR}/$${edir}; \
fi; \
${MAKE} ${__target} DIRPRFX=${DIRPRFX}$${edir}/
.endfor
par-${__target}: ${SUBDIR:S/$/.${__target}__D/}
.endfor
# Enable SUBDIR_PARALLEL when not calling 'make all', unless called as
# 'par-all'. This is because it is unlikely that running 'make all' from
# the top-level, especially with a SUBDIR_OVERRIDE or LOCAL_DIRS set,
# will have a reliable build if SUBDIRs are built in parallel. This is
# safe for the world stage of buildworld though since it has already
# built libraries in a proper order and installed includes into WORLDTMP.
# Special handling is done for SUBDIR ordering for 'install*' to avoid
# trashing a system if it crashes mid-install.
par-all: all .PHONY
.if !make(all)
SUBDIR_PARALLEL=
.endif
.include <bsd.subdir.mk>
@ -2267,7 +2270,7 @@ _xi-cross-tools:
.endfor
_xi-includes:
${_+_}cd ${.CURDIR}; ${CD2MAKE} -f Makefile.inc1 par-includes \
${_+_}cd ${.CURDIR}; ${CD2MAKE} -f Makefile.inc1 includes \
DESTDIR=${XDDESTDIR}
_xi-libraries: