freebsd-skq/share/mk/bsd.lib.mk

584 lines
17 KiB
Makefile
Raw Normal View History

# from: @(#)bsd.lib.mk 5.26 (Berkeley) 5/2/91
1999-08-28 00:22:10 +00:00
# $FreeBSD$
#
1994-05-30 19:09:18 +00:00
.include <bsd.init.mk>
.include <bsd.compiler.mk>
.include <bsd.linker.mk>
1994-05-30 19:09:18 +00:00
.if defined(LIB_CXX) || defined(SHLIB_CXX)
_LD= ${CXX}
.else
_LD= ${CC}
.endif
.if defined(LIB_CXX)
LIB= ${LIB_CXX}
.endif
.if defined(SHLIB_CXX)
SHLIB= ${SHLIB_CXX}
.endif
LIB_PRIVATE= ${PRIVATELIB:Dprivate}
# Set up the variables controlling shared libraries. After this section,
# SHLIB_NAME will be defined only if we are to create a shared library.
# SHLIB_LINK will be defined only if we are to create a link to it.
# INSTALL_PIC_ARCHIVE will be defined only if we are to create a PIC archive.
# BUILD_NOSSP_PIC_ARCHIVE will be defined only if we are to create a PIC archive.
.if defined(NO_PIC)
.undef SHLIB_NAME
.undef INSTALL_PIC_ARCHIVE
.undef BUILD_NOSSP_PIC_ARCHIVE
.else
.if !defined(SHLIB) && defined(LIB)
SHLIB= ${LIB}
.endif
.if !defined(SHLIB_NAME) && defined(SHLIB) && defined(SHLIB_MAJOR)
SHLIB_NAME= lib${LIB_PRIVATE}${SHLIB}.so.${SHLIB_MAJOR}
.endif
.if defined(SHLIB_NAME) && !empty(SHLIB_NAME:M*.so.*)
SHLIB_LINK?= ${SHLIB_NAME:R}
.endif
SONAME?= ${SHLIB_NAME}
.endif
.if defined(CRUNCH_CFLAGS)
CFLAGS+= ${CRUNCH_CFLAGS}
.endif
.if ${MK_ASSERT_DEBUG} == "no"
CFLAGS+= -DNDEBUG
# XXX: shouldn't we ensure that !asserts marks potentially unused variables as
# __unused instead of disabling -Werror globally?
MK_WERROR= no
.endif
.if defined(DEBUG_FLAGS)
CFLAGS+= ${DEBUG_FLAGS}
Add support for the Compact C Type (CTF) conversions throughout FreeBSD's system makefiles. Note that the CTF conversion defaults to off. We may choose to change this default later if DTrace proves popular and people are prepared to wear the compilation performance impact of compiling with debug symbols all the time. Setting NO_CTF in the make args or user environment turns off CTF conversion. Even if we choose to default CTF generation to on later, we still need NO_CTF so that the buildworld process can bootstrap the tools without needlessly generating CTF data for temporary tools. Setting WITH_CTF in the make args or user environment (and _NOT_ in /etc/make.conf) is the only way to enable CTF data conversion. Nore that this can't be implemented the same way that the WITH_ and WITHOUT_ stuff is implemented throughout the buildworld because the CTF conversion needs to work when building a simple object without a Makefile, using the default rules in sys.mk. Typing 'make test.o' with no makefile and just a source file test.c should work. Also, typing 'make WITH_CTF=1 test.o without a makefile and just a source file test.c should work and produce an object with a CTF elf section. Typing 'make WITH_CTF=1 CFLAGS=-g test.o' without a makefile and just a source file test.c should produce an object with both a CTF elf section and the debug elf sections. In the FreeBSD build where more .mk files are used than just sys.mk which is included my make by default, the use of DEBUG_FLAGS is the correct way to enable a debug build. The important thing to note here is that it is the DEBUG_FLAGS setting that prevents libraries and programs from being stripped on installation. So, for the addition of CTF data conversion, setting DEBUG_FLAGS to contain -g, without NO_CTF, will cause the ctfconvert and ctfmerge build programs to be executed also with the -g arg so that debug symbols are retained rather than being removed after the CTF data elf section has been added. Add DTrace libraries to the list of libnames.
2008-05-22 01:14:43 +00:00
.if ${MK_CTF} != "no" && ${DEBUG_FLAGS:M-g} != ""
Add support for the Compact C Type (CTF) conversions throughout FreeBSD's system makefiles. Note that the CTF conversion defaults to off. We may choose to change this default later if DTrace proves popular and people are prepared to wear the compilation performance impact of compiling with debug symbols all the time. Setting NO_CTF in the make args or user environment turns off CTF conversion. Even if we choose to default CTF generation to on later, we still need NO_CTF so that the buildworld process can bootstrap the tools without needlessly generating CTF data for temporary tools. Setting WITH_CTF in the make args or user environment (and _NOT_ in /etc/make.conf) is the only way to enable CTF data conversion. Nore that this can't be implemented the same way that the WITH_ and WITHOUT_ stuff is implemented throughout the buildworld because the CTF conversion needs to work when building a simple object without a Makefile, using the default rules in sys.mk. Typing 'make test.o' with no makefile and just a source file test.c should work. Also, typing 'make WITH_CTF=1 test.o without a makefile and just a source file test.c should work and produce an object with a CTF elf section. Typing 'make WITH_CTF=1 CFLAGS=-g test.o' without a makefile and just a source file test.c should produce an object with both a CTF elf section and the debug elf sections. In the FreeBSD build where more .mk files are used than just sys.mk which is included my make by default, the use of DEBUG_FLAGS is the correct way to enable a debug build. The important thing to note here is that it is the DEBUG_FLAGS setting that prevents libraries and programs from being stripped on installation. So, for the addition of CTF data conversion, setting DEBUG_FLAGS to contain -g, without NO_CTF, will cause the ctfconvert and ctfmerge build programs to be executed also with the -g arg so that debug symbols are retained rather than being removed after the CTF data elf section has been added. Add DTrace libraries to the list of libnames.
2008-05-22 01:14:43 +00:00
CTFFLAGS+= -g
.endif
.else
1994-05-30 19:09:18 +00:00
STRIP?= -s
.endif
1994-05-30 19:09:18 +00:00
2015-03-18 11:05:49 +00:00
.if ${SHLIBDIR:M*lib32*}
TAGS+= lib32
.endif
.if defined(NO_ROOT)
.if !defined(TAGS) || ! ${TAGS:Mpackage=*}
TAGS+= package=${PACKAGE:Uutilities}
.endif
TAG_ARGS= -T ${TAGS:[*]:S/ /,/g}
.endif
# ELF hardening knobs
.if ${MK_BIND_NOW} != "no"
LDFLAGS+= -Wl,-znow
.endif
.if ${MK_RETPOLINE} != "no"
.if ${COMPILER_FEATURES:Mretpoline} && ${LINKER_FEATURES:Mretpoline}
CFLAGS+= -mretpoline
CXXFLAGS+= -mretpoline
LDFLAGS+= -Wl,-zretpolineplt
.else
.warning Retpoline requested but not supported by compiler or linker
.endif
.endif
# Initialize stack variables on function entry
.if ${MK_INIT_ALL_ZERO} == "yes"
.if ${COMPILER_FEATURES:Minit-all}
CFLAGS+= -ftrivial-auto-var-init=zero \
-enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang
CXXFLAGS+= -ftrivial-auto-var-init=zero \
-enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang
.else
.warning InitAll (zeros) requested but not support by compiler
.endif
.elif ${MK_INIT_ALL_PATTERN} == "yes"
.if ${COMPILER_FEATURES:Minit-all}
CFLAGS+= -ftrivial-auto-var-init=pattern
CXXFLAGS+= -ftrivial-auto-var-init=pattern
.else
.warning InitAll (pattern) requested but not support by compiler
.endif
.endif
.if ${MK_DEBUG_FILES} != "no" && empty(DEBUG_FLAGS:M-g) && \
empty(DEBUG_FLAGS:M-gdwarf*)
CFLAGS+= ${DEBUG_FILES_CFLAGS}
CXXFLAGS+= ${DEBUG_FILES_CFLAGS}
CTFFLAGS+= -g
.endif
# clang currently defaults to dynamic TLS for mips64 object files without -fPIC
.if ${MACHINE_ARCH:Mmips64*} && ${COMPILER_TYPE} == "clang"
STATIC_CFLAGS+= -ftls-model=initial-exec
STATIC_CXXFLAGS+= -ftls-model=initial-exec
.endif
.if ${MACHINE_CPUARCH} == "riscv" && ${LINKER_FEATURES:Mriscv-relaxations} == ""
CFLAGS += -mno-relax
.endif
.include <bsd.libnames.mk>
1994-05-30 19:09:18 +00:00
# prefer .s to a .c, add .po, remove stuff not used in the BSD libraries
# .pico used for PIC object files
# .nossppico used for NOSSP PIC object files
# .pieo used for PIE object files
.SUFFIXES: .out .o .bc .ll .po .pico .nossppico .pieo .S .asm .s .c .cc .cpp .cxx .C .f .y .l .ln
.if !defined(PICFLAG)
PICFLAG=-fpic
PIEFLAG=-fpie
.endif
PO_FLAG=-pg
.c.po:
${CC} ${PO_FLAG} ${STATIC_CFLAGS} ${PO_CFLAGS} -c ${.IMPSRC} -o ${.TARGET}
${CTFCONVERT_CMD}
1994-05-30 19:09:18 +00:00
.c.pico:
${CC} ${PICFLAG} -DPIC ${SHARED_CFLAGS} ${CFLAGS} -c ${.IMPSRC} -o ${.TARGET}
${CTFCONVERT_CMD}
.c.nossppico:
${CC} ${PICFLAG} -DPIC ${SHARED_CFLAGS:C/^-fstack-protector.*$//} ${CFLAGS:C/^-fstack-protector.*$//} -c ${.IMPSRC} -o ${.TARGET}
${CTFCONVERT_CMD}
.c.pieo:
${CC} ${PIEFLAG} -DPIC ${SHARED_CFLAGS} ${CFLAGS} -c ${.IMPSRC} -o ${.TARGET}
${CTFCONVERT_CMD}
.cc.po .C.po .cpp.po .cxx.po:
${CXX} ${PO_FLAG} ${STATIC_CXXFLAGS} ${PO_CXXFLAGS} -c ${.IMPSRC} -o ${.TARGET}
.cc.pico .C.pico .cpp.pico .cxx.pico:
${CXX} ${PICFLAG} -DPIC ${SHARED_CXXFLAGS} ${CXXFLAGS} -c ${.IMPSRC} -o ${.TARGET}
.cc.nossppico .C.nossppico .cpp.nossppico .cxx.nossppico:
${CXX} ${PICFLAG} -DPIC ${SHARED_CXXFLAGS:C/^-fstack-protector.*$//} ${CXXFLAGS:C/^-fstack-protector.*$//} -c ${.IMPSRC} -o ${.TARGET}
.cc.pieo .C.pieo .cpp.pieo .cxx.pieo:
${CXX} ${PIEFLAG} ${SHARED_CXXFLAGS} ${CXXFLAGS} -c ${.IMPSRC} -o ${.TARGET}
.f.po:
2004-03-15 17:01:31 +00:00
${FC} -pg ${FFLAGS} -o ${.TARGET} -c ${.IMPSRC}
${CTFCONVERT_CMD}
.f.pico:
${FC} ${PICFLAG} -DPIC ${FFLAGS} -o ${.TARGET} -c ${.IMPSRC}
${CTFCONVERT_CMD}
.f.nossppico:
${FC} ${PICFLAG} -DPIC ${FFLAGS:C/^-fstack-protector.*$//} -o ${.TARGET} -c ${.IMPSRC}
${CTFCONVERT_CMD}
.s.po .s.pico .s.nossppico .s.pieo:
${AS} ${AFLAGS} -o ${.TARGET} ${.IMPSRC}
${CTFCONVERT_CMD}
.asm.po:
Add built-in ccache build support via WITH_CCACHE_BUILD option. ccache is mostly beneficial for frequent builds where -DNO_CLEAN is not used to achieve a safe pseudo-incremental build. This is explained in more detail upstream [1] [2]. It incurs about a 20%-28% hit to populate the cache, but with a full cache saves 30-50% in build times. When combined with the WITH_FAST_DEPEND feature it saves up to 65% since ccache does cache the resulting dependency file, which it does not do when using mkdep(1)/'CC -E'. Stats are provided at the end of this message. This removes the need to modify /etc/make.conf with the CC:= and CXX:= lines which conflicted with external compiler support [3] (causing the bootstrap compiler to not be built which lead to obscure failures [4]), incorrectly invoked ccache in various stages, required CCACHE_CPP2 to avoid Clang errors with parenthesis, and did not work with META_MODE. The option name was picked to match the existing option in ports. This feature is available for both in-src and out-of-src builds that use /usr/share/mk. Linking, assembly compiles, and pre-processing avoid using ccache since it is only overhead. ccache does nothing special in these modes, although there is no harm in calling it for them. CCACHE_COMPILERCHECK is set to 'content' when using the in-tree bootstrap compiler to hash the content of the compiler binary to determine if it should be a cache miss. For external compilers the 'mtime' option is used as it is more efficient and likely to be correct. Future work may optimize the 'content' check using the same checks as whether a bootstrap compiler is needed to be built. The CCACHE_CPP2 pessimization is currently default in our devel/ccache port due to Clang requiring it. Clang's -Wparentheses-equality, -Wtautological-compare, and -Wself-assign warnings do not mix well with compiling already-pre-processed code that may have expanded macros that trigger the warnings. GCC has so far not had this issue so it is allowed to disable the CCACHE_CPP2 default in our port. Sharing a cache between multiple checkouts, or systems, is explained in the ccache manual. Sharing a cache over NFS would likely not be worth it, but syncing cache directories between systems may be useful for an organization. There is also a memcached backend available [5]. Due to using an object directory outside of the source directory though you will need to ensure that both are in the same prefix and all users use the same layout. A possible working layout is as follows: Source: /some/prefix/src1 Source: /some/prefix/src2 Source: /some/prefix/src3 Objdir: /some/prefix/obj Environment: CCACHE_BASEDIR='${SRCTOP:H}' MAKEOBJDIRPREFIX='${SRCTOP:H}/obj' This will use src*/../obj as the MAKEOBJDIRPREFIX and tells ccache to replace all absolute paths to be relative. Using something like this is required due to -I and -o flags containing both SRC and OBJDIR absolute paths that ccache adds into its hash for the object without CCACHE_BASEDIR. distcc can be hooked into by setting CCACHE_PREFIX=/usr/local/bin/distcc. I have not personally tested this and assume it will not mix well with using the bootstrap compiler. The cache from buildworld can be reused in a subdir by first running 'make buildenv' (from r290424). Note that the cache is currently different depending on whether -j is used or not due to ccache enabling -fdiagnostics-color automatically if stderr is a TTY, which bmake only does if not using -j. The system I used for testing was: WITNESS Build options: -j20 WITH_LLDB=yes WITH_DEBUG_FILES=yes WITH_CCACHE_BUILD=yes DISK: ZFS 3-way mirror with very slow disks using SSD l2arc/log. The arc was fully populated with src tree files and ccache objects. RAM: 76GiB CPU: Intel(R) Xeon(R) CPU L5520 @2.27GHz 2 package(s) x 4 core(s) x 2 SMT threads = hw.ncpu=16 The WITH_FAST_DEPEND feature was used for comparison here as well to show the dramatic time savings with a full cache. buildworld: x buildworld-before + buildworld-ccache-empty * buildworld-ccache-full % buildworld-ccache-full-fastdep # buildworld-fastdep +-------------------------------------------------------------------------------+ |% * # +| |% * # +| |% * # xxx +| | |A | | A| | A | |A | | A | +-------------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 3 3744.13 3794.31 3752.25 3763.5633 26.935139 + 3 4519 4525.04 4520.73 4521.59 3.1104823 Difference at 95.0% confidence 758.027 +/- 43.4565 20.1412% +/- 1.15466% (Student's t, pooled s = 19.1726) * 3 1823.08 1827.2 1825.62 1825.3 2.0785572 Difference at 95.0% confidence -1938.26 +/- 43.298 -51.5007% +/- 1.15045% (Student's t, pooled s = 19.1026) % 3 1266.96 1279.37 1270.47 1272.2667 6.3971113 Difference at 95.0% confidence -2491.3 +/- 44.3704 -66.1952% +/- 1.17895% (Student's t, pooled s = 19.5758) # 3 3153.34 3155.16 3154.2 3154.2333 0.91045776 Difference at 95.0% confidence -609.33 +/- 43.1943 -16.1902% +/- 1.1477% (Student's t, pooled s = 19.0569) buildkernel: x buildkernel-before + buildkernel-ccache-empty * buildkernel-ccache-empty-fastdep % buildkernel-ccache-full # buildkernel-ccache-full-fastdep @ buildkernel-fastdep +-------------------------------------------------------------------------------+ |# @ % * | |# @ % * x + | |# @ % * xx ++| | MA | | MA| | A | | A | |A | | A | +-------------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 3 571.57 573.94 571.79 572.43333 1.3094401 + 3 727.97 731.91 728.06 729.31333 2.2492295 Difference at 95.0% confidence 156.88 +/- 4.17129 27.4058% +/- 0.728695% (Student's t, pooled s = 1.84034) * 3 527.1 528.29 528.08 527.82333 0.63516402 Difference at 95.0% confidence -44.61 +/- 2.33254 -7.79305% +/- 0.407478% (Student's t, pooled s = 1.02909) % 3 400.4 401.05 400.62 400.69 0.3306055 Difference at 95.0% confidence -171.743 +/- 2.16453 -30.0023% +/- 0.378128% (Student's t, pooled s = 0.954969) # 3 201.94 203.34 202.28 202.52 0.73020545 Difference at 95.0% confidence -369.913 +/- 2.40293 -64.6212% +/- 0.419774% (Student's t, pooled s = 1.06015) @ 3 369.12 370.57 369.3 369.66333 0.79033748 Difference at 95.0% confidence -202.77 +/- 2.45131 -35.4225% +/- 0.428227% (Student's t, pooled s = 1.0815) [1] https://ccache.samba.org/performance.html [2] http://www.mail-archive.com/ccache@lists.samba.org/msg00576.html [3] https://reviews.freebsd.org/D3484 [5] https://github.com/jrosdahl/ccache/pull/30 PR: 182944 [4] MFC after: 3 weeks Sponsored by: EMC / Isilon Storage Division Relnotes: yes
2015-11-08 00:50:18 +00:00
${CC:N${CCACHE_BIN}} -x assembler-with-cpp -DPROF ${PO_CFLAGS} \
${ACFLAGS} -c ${.IMPSRC} -o ${.TARGET}
${CTFCONVERT_CMD}
.asm.pico:
Add built-in ccache build support via WITH_CCACHE_BUILD option. ccache is mostly beneficial for frequent builds where -DNO_CLEAN is not used to achieve a safe pseudo-incremental build. This is explained in more detail upstream [1] [2]. It incurs about a 20%-28% hit to populate the cache, but with a full cache saves 30-50% in build times. When combined with the WITH_FAST_DEPEND feature it saves up to 65% since ccache does cache the resulting dependency file, which it does not do when using mkdep(1)/'CC -E'. Stats are provided at the end of this message. This removes the need to modify /etc/make.conf with the CC:= and CXX:= lines which conflicted with external compiler support [3] (causing the bootstrap compiler to not be built which lead to obscure failures [4]), incorrectly invoked ccache in various stages, required CCACHE_CPP2 to avoid Clang errors with parenthesis, and did not work with META_MODE. The option name was picked to match the existing option in ports. This feature is available for both in-src and out-of-src builds that use /usr/share/mk. Linking, assembly compiles, and pre-processing avoid using ccache since it is only overhead. ccache does nothing special in these modes, although there is no harm in calling it for them. CCACHE_COMPILERCHECK is set to 'content' when using the in-tree bootstrap compiler to hash the content of the compiler binary to determine if it should be a cache miss. For external compilers the 'mtime' option is used as it is more efficient and likely to be correct. Future work may optimize the 'content' check using the same checks as whether a bootstrap compiler is needed to be built. The CCACHE_CPP2 pessimization is currently default in our devel/ccache port due to Clang requiring it. Clang's -Wparentheses-equality, -Wtautological-compare, and -Wself-assign warnings do not mix well with compiling already-pre-processed code that may have expanded macros that trigger the warnings. GCC has so far not had this issue so it is allowed to disable the CCACHE_CPP2 default in our port. Sharing a cache between multiple checkouts, or systems, is explained in the ccache manual. Sharing a cache over NFS would likely not be worth it, but syncing cache directories between systems may be useful for an organization. There is also a memcached backend available [5]. Due to using an object directory outside of the source directory though you will need to ensure that both are in the same prefix and all users use the same layout. A possible working layout is as follows: Source: /some/prefix/src1 Source: /some/prefix/src2 Source: /some/prefix/src3 Objdir: /some/prefix/obj Environment: CCACHE_BASEDIR='${SRCTOP:H}' MAKEOBJDIRPREFIX='${SRCTOP:H}/obj' This will use src*/../obj as the MAKEOBJDIRPREFIX and tells ccache to replace all absolute paths to be relative. Using something like this is required due to -I and -o flags containing both SRC and OBJDIR absolute paths that ccache adds into its hash for the object without CCACHE_BASEDIR. distcc can be hooked into by setting CCACHE_PREFIX=/usr/local/bin/distcc. I have not personally tested this and assume it will not mix well with using the bootstrap compiler. The cache from buildworld can be reused in a subdir by first running 'make buildenv' (from r290424). Note that the cache is currently different depending on whether -j is used or not due to ccache enabling -fdiagnostics-color automatically if stderr is a TTY, which bmake only does if not using -j. The system I used for testing was: WITNESS Build options: -j20 WITH_LLDB=yes WITH_DEBUG_FILES=yes WITH_CCACHE_BUILD=yes DISK: ZFS 3-way mirror with very slow disks using SSD l2arc/log. The arc was fully populated with src tree files and ccache objects. RAM: 76GiB CPU: Intel(R) Xeon(R) CPU L5520 @2.27GHz 2 package(s) x 4 core(s) x 2 SMT threads = hw.ncpu=16 The WITH_FAST_DEPEND feature was used for comparison here as well to show the dramatic time savings with a full cache. buildworld: x buildworld-before + buildworld-ccache-empty * buildworld-ccache-full % buildworld-ccache-full-fastdep # buildworld-fastdep +-------------------------------------------------------------------------------+ |% * # +| |% * # +| |% * # xxx +| | |A | | A| | A | |A | | A | +-------------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 3 3744.13 3794.31 3752.25 3763.5633 26.935139 + 3 4519 4525.04 4520.73 4521.59 3.1104823 Difference at 95.0% confidence 758.027 +/- 43.4565 20.1412% +/- 1.15466% (Student's t, pooled s = 19.1726) * 3 1823.08 1827.2 1825.62 1825.3 2.0785572 Difference at 95.0% confidence -1938.26 +/- 43.298 -51.5007% +/- 1.15045% (Student's t, pooled s = 19.1026) % 3 1266.96 1279.37 1270.47 1272.2667 6.3971113 Difference at 95.0% confidence -2491.3 +/- 44.3704 -66.1952% +/- 1.17895% (Student's t, pooled s = 19.5758) # 3 3153.34 3155.16 3154.2 3154.2333 0.91045776 Difference at 95.0% confidence -609.33 +/- 43.1943 -16.1902% +/- 1.1477% (Student's t, pooled s = 19.0569) buildkernel: x buildkernel-before + buildkernel-ccache-empty * buildkernel-ccache-empty-fastdep % buildkernel-ccache-full # buildkernel-ccache-full-fastdep @ buildkernel-fastdep +-------------------------------------------------------------------------------+ |# @ % * | |# @ % * x + | |# @ % * xx ++| | MA | | MA| | A | | A | |A | | A | +-------------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 3 571.57 573.94 571.79 572.43333 1.3094401 + 3 727.97 731.91 728.06 729.31333 2.2492295 Difference at 95.0% confidence 156.88 +/- 4.17129 27.4058% +/- 0.728695% (Student's t, pooled s = 1.84034) * 3 527.1 528.29 528.08 527.82333 0.63516402 Difference at 95.0% confidence -44.61 +/- 2.33254 -7.79305% +/- 0.407478% (Student's t, pooled s = 1.02909) % 3 400.4 401.05 400.62 400.69 0.3306055 Difference at 95.0% confidence -171.743 +/- 2.16453 -30.0023% +/- 0.378128% (Student's t, pooled s = 0.954969) # 3 201.94 203.34 202.28 202.52 0.73020545 Difference at 95.0% confidence -369.913 +/- 2.40293 -64.6212% +/- 0.419774% (Student's t, pooled s = 1.06015) @ 3 369.12 370.57 369.3 369.66333 0.79033748 Difference at 95.0% confidence -202.77 +/- 2.45131 -35.4225% +/- 0.428227% (Student's t, pooled s = 1.0815) [1] https://ccache.samba.org/performance.html [2] http://www.mail-archive.com/ccache@lists.samba.org/msg00576.html [3] https://reviews.freebsd.org/D3484 [5] https://github.com/jrosdahl/ccache/pull/30 PR: 182944 [4] MFC after: 3 weeks Sponsored by: EMC / Isilon Storage Division Relnotes: yes
2015-11-08 00:50:18 +00:00
${CC:N${CCACHE_BIN}} -x assembler-with-cpp ${PICFLAG} -DPIC \
${CFLAGS} ${ACFLAGS} -c ${.IMPSRC} -o ${.TARGET}
${CTFCONVERT_CMD}
.asm.nossppico:
${CC:N${CCACHE_BIN}} -x assembler-with-cpp ${PICFLAG} -DPIC \
${CFLAGS:C/^-fstack-protector.*$//} ${ACFLAGS} -c ${.IMPSRC} -o ${.TARGET}
${CTFCONVERT_CMD}
.asm.pieo:
${CC:N${CCACHE_BIN}} -x assembler-with-cpp ${PIEFLAG} -DPIC \
${CFLAGS} ${ACFLAGS} -c ${.IMPSRC} -o ${.TARGET}
${CTFCONVERT_CMD}
.S.po:
Add built-in ccache build support via WITH_CCACHE_BUILD option. ccache is mostly beneficial for frequent builds where -DNO_CLEAN is not used to achieve a safe pseudo-incremental build. This is explained in more detail upstream [1] [2]. It incurs about a 20%-28% hit to populate the cache, but with a full cache saves 30-50% in build times. When combined with the WITH_FAST_DEPEND feature it saves up to 65% since ccache does cache the resulting dependency file, which it does not do when using mkdep(1)/'CC -E'. Stats are provided at the end of this message. This removes the need to modify /etc/make.conf with the CC:= and CXX:= lines which conflicted with external compiler support [3] (causing the bootstrap compiler to not be built which lead to obscure failures [4]), incorrectly invoked ccache in various stages, required CCACHE_CPP2 to avoid Clang errors with parenthesis, and did not work with META_MODE. The option name was picked to match the existing option in ports. This feature is available for both in-src and out-of-src builds that use /usr/share/mk. Linking, assembly compiles, and pre-processing avoid using ccache since it is only overhead. ccache does nothing special in these modes, although there is no harm in calling it for them. CCACHE_COMPILERCHECK is set to 'content' when using the in-tree bootstrap compiler to hash the content of the compiler binary to determine if it should be a cache miss. For external compilers the 'mtime' option is used as it is more efficient and likely to be correct. Future work may optimize the 'content' check using the same checks as whether a bootstrap compiler is needed to be built. The CCACHE_CPP2 pessimization is currently default in our devel/ccache port due to Clang requiring it. Clang's -Wparentheses-equality, -Wtautological-compare, and -Wself-assign warnings do not mix well with compiling already-pre-processed code that may have expanded macros that trigger the warnings. GCC has so far not had this issue so it is allowed to disable the CCACHE_CPP2 default in our port. Sharing a cache between multiple checkouts, or systems, is explained in the ccache manual. Sharing a cache over NFS would likely not be worth it, but syncing cache directories between systems may be useful for an organization. There is also a memcached backend available [5]. Due to using an object directory outside of the source directory though you will need to ensure that both are in the same prefix and all users use the same layout. A possible working layout is as follows: Source: /some/prefix/src1 Source: /some/prefix/src2 Source: /some/prefix/src3 Objdir: /some/prefix/obj Environment: CCACHE_BASEDIR='${SRCTOP:H}' MAKEOBJDIRPREFIX='${SRCTOP:H}/obj' This will use src*/../obj as the MAKEOBJDIRPREFIX and tells ccache to replace all absolute paths to be relative. Using something like this is required due to -I and -o flags containing both SRC and OBJDIR absolute paths that ccache adds into its hash for the object without CCACHE_BASEDIR. distcc can be hooked into by setting CCACHE_PREFIX=/usr/local/bin/distcc. I have not personally tested this and assume it will not mix well with using the bootstrap compiler. The cache from buildworld can be reused in a subdir by first running 'make buildenv' (from r290424). Note that the cache is currently different depending on whether -j is used or not due to ccache enabling -fdiagnostics-color automatically if stderr is a TTY, which bmake only does if not using -j. The system I used for testing was: WITNESS Build options: -j20 WITH_LLDB=yes WITH_DEBUG_FILES=yes WITH_CCACHE_BUILD=yes DISK: ZFS 3-way mirror with very slow disks using SSD l2arc/log. The arc was fully populated with src tree files and ccache objects. RAM: 76GiB CPU: Intel(R) Xeon(R) CPU L5520 @2.27GHz 2 package(s) x 4 core(s) x 2 SMT threads = hw.ncpu=16 The WITH_FAST_DEPEND feature was used for comparison here as well to show the dramatic time savings with a full cache. buildworld: x buildworld-before + buildworld-ccache-empty * buildworld-ccache-full % buildworld-ccache-full-fastdep # buildworld-fastdep +-------------------------------------------------------------------------------+ |% * # +| |% * # +| |% * # xxx +| | |A | | A| | A | |A | | A | +-------------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 3 3744.13 3794.31 3752.25 3763.5633 26.935139 + 3 4519 4525.04 4520.73 4521.59 3.1104823 Difference at 95.0% confidence 758.027 +/- 43.4565 20.1412% +/- 1.15466% (Student's t, pooled s = 19.1726) * 3 1823.08 1827.2 1825.62 1825.3 2.0785572 Difference at 95.0% confidence -1938.26 +/- 43.298 -51.5007% +/- 1.15045% (Student's t, pooled s = 19.1026) % 3 1266.96 1279.37 1270.47 1272.2667 6.3971113 Difference at 95.0% confidence -2491.3 +/- 44.3704 -66.1952% +/- 1.17895% (Student's t, pooled s = 19.5758) # 3 3153.34 3155.16 3154.2 3154.2333 0.91045776 Difference at 95.0% confidence -609.33 +/- 43.1943 -16.1902% +/- 1.1477% (Student's t, pooled s = 19.0569) buildkernel: x buildkernel-before + buildkernel-ccache-empty * buildkernel-ccache-empty-fastdep % buildkernel-ccache-full # buildkernel-ccache-full-fastdep @ buildkernel-fastdep +-------------------------------------------------------------------------------+ |# @ % * | |# @ % * x + | |# @ % * xx ++| | MA | | MA| | A | | A | |A | | A | +-------------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 3 571.57 573.94 571.79 572.43333 1.3094401 + 3 727.97 731.91 728.06 729.31333 2.2492295 Difference at 95.0% confidence 156.88 +/- 4.17129 27.4058% +/- 0.728695% (Student's t, pooled s = 1.84034) * 3 527.1 528.29 528.08 527.82333 0.63516402 Difference at 95.0% confidence -44.61 +/- 2.33254 -7.79305% +/- 0.407478% (Student's t, pooled s = 1.02909) % 3 400.4 401.05 400.62 400.69 0.3306055 Difference at 95.0% confidence -171.743 +/- 2.16453 -30.0023% +/- 0.378128% (Student's t, pooled s = 0.954969) # 3 201.94 203.34 202.28 202.52 0.73020545 Difference at 95.0% confidence -369.913 +/- 2.40293 -64.6212% +/- 0.419774% (Student's t, pooled s = 1.06015) @ 3 369.12 370.57 369.3 369.66333 0.79033748 Difference at 95.0% confidence -202.77 +/- 2.45131 -35.4225% +/- 0.428227% (Student's t, pooled s = 1.0815) [1] https://ccache.samba.org/performance.html [2] http://www.mail-archive.com/ccache@lists.samba.org/msg00576.html [3] https://reviews.freebsd.org/D3484 [5] https://github.com/jrosdahl/ccache/pull/30 PR: 182944 [4] MFC after: 3 weeks Sponsored by: EMC / Isilon Storage Division Relnotes: yes
2015-11-08 00:50:18 +00:00
${CC:N${CCACHE_BIN}} -DPROF ${PO_CFLAGS} ${ACFLAGS} -c ${.IMPSRC} \
-o ${.TARGET}
${CTFCONVERT_CMD}
.S.pico:
Add built-in ccache build support via WITH_CCACHE_BUILD option. ccache is mostly beneficial for frequent builds where -DNO_CLEAN is not used to achieve a safe pseudo-incremental build. This is explained in more detail upstream [1] [2]. It incurs about a 20%-28% hit to populate the cache, but with a full cache saves 30-50% in build times. When combined with the WITH_FAST_DEPEND feature it saves up to 65% since ccache does cache the resulting dependency file, which it does not do when using mkdep(1)/'CC -E'. Stats are provided at the end of this message. This removes the need to modify /etc/make.conf with the CC:= and CXX:= lines which conflicted with external compiler support [3] (causing the bootstrap compiler to not be built which lead to obscure failures [4]), incorrectly invoked ccache in various stages, required CCACHE_CPP2 to avoid Clang errors with parenthesis, and did not work with META_MODE. The option name was picked to match the existing option in ports. This feature is available for both in-src and out-of-src builds that use /usr/share/mk. Linking, assembly compiles, and pre-processing avoid using ccache since it is only overhead. ccache does nothing special in these modes, although there is no harm in calling it for them. CCACHE_COMPILERCHECK is set to 'content' when using the in-tree bootstrap compiler to hash the content of the compiler binary to determine if it should be a cache miss. For external compilers the 'mtime' option is used as it is more efficient and likely to be correct. Future work may optimize the 'content' check using the same checks as whether a bootstrap compiler is needed to be built. The CCACHE_CPP2 pessimization is currently default in our devel/ccache port due to Clang requiring it. Clang's -Wparentheses-equality, -Wtautological-compare, and -Wself-assign warnings do not mix well with compiling already-pre-processed code that may have expanded macros that trigger the warnings. GCC has so far not had this issue so it is allowed to disable the CCACHE_CPP2 default in our port. Sharing a cache between multiple checkouts, or systems, is explained in the ccache manual. Sharing a cache over NFS would likely not be worth it, but syncing cache directories between systems may be useful for an organization. There is also a memcached backend available [5]. Due to using an object directory outside of the source directory though you will need to ensure that both are in the same prefix and all users use the same layout. A possible working layout is as follows: Source: /some/prefix/src1 Source: /some/prefix/src2 Source: /some/prefix/src3 Objdir: /some/prefix/obj Environment: CCACHE_BASEDIR='${SRCTOP:H}' MAKEOBJDIRPREFIX='${SRCTOP:H}/obj' This will use src*/../obj as the MAKEOBJDIRPREFIX and tells ccache to replace all absolute paths to be relative. Using something like this is required due to -I and -o flags containing both SRC and OBJDIR absolute paths that ccache adds into its hash for the object without CCACHE_BASEDIR. distcc can be hooked into by setting CCACHE_PREFIX=/usr/local/bin/distcc. I have not personally tested this and assume it will not mix well with using the bootstrap compiler. The cache from buildworld can be reused in a subdir by first running 'make buildenv' (from r290424). Note that the cache is currently different depending on whether -j is used or not due to ccache enabling -fdiagnostics-color automatically if stderr is a TTY, which bmake only does if not using -j. The system I used for testing was: WITNESS Build options: -j20 WITH_LLDB=yes WITH_DEBUG_FILES=yes WITH_CCACHE_BUILD=yes DISK: ZFS 3-way mirror with very slow disks using SSD l2arc/log. The arc was fully populated with src tree files and ccache objects. RAM: 76GiB CPU: Intel(R) Xeon(R) CPU L5520 @2.27GHz 2 package(s) x 4 core(s) x 2 SMT threads = hw.ncpu=16 The WITH_FAST_DEPEND feature was used for comparison here as well to show the dramatic time savings with a full cache. buildworld: x buildworld-before + buildworld-ccache-empty * buildworld-ccache-full % buildworld-ccache-full-fastdep # buildworld-fastdep +-------------------------------------------------------------------------------+ |% * # +| |% * # +| |% * # xxx +| | |A | | A| | A | |A | | A | +-------------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 3 3744.13 3794.31 3752.25 3763.5633 26.935139 + 3 4519 4525.04 4520.73 4521.59 3.1104823 Difference at 95.0% confidence 758.027 +/- 43.4565 20.1412% +/- 1.15466% (Student's t, pooled s = 19.1726) * 3 1823.08 1827.2 1825.62 1825.3 2.0785572 Difference at 95.0% confidence -1938.26 +/- 43.298 -51.5007% +/- 1.15045% (Student's t, pooled s = 19.1026) % 3 1266.96 1279.37 1270.47 1272.2667 6.3971113 Difference at 95.0% confidence -2491.3 +/- 44.3704 -66.1952% +/- 1.17895% (Student's t, pooled s = 19.5758) # 3 3153.34 3155.16 3154.2 3154.2333 0.91045776 Difference at 95.0% confidence -609.33 +/- 43.1943 -16.1902% +/- 1.1477% (Student's t, pooled s = 19.0569) buildkernel: x buildkernel-before + buildkernel-ccache-empty * buildkernel-ccache-empty-fastdep % buildkernel-ccache-full # buildkernel-ccache-full-fastdep @ buildkernel-fastdep +-------------------------------------------------------------------------------+ |# @ % * | |# @ % * x + | |# @ % * xx ++| | MA | | MA| | A | | A | |A | | A | +-------------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 3 571.57 573.94 571.79 572.43333 1.3094401 + 3 727.97 731.91 728.06 729.31333 2.2492295 Difference at 95.0% confidence 156.88 +/- 4.17129 27.4058% +/- 0.728695% (Student's t, pooled s = 1.84034) * 3 527.1 528.29 528.08 527.82333 0.63516402 Difference at 95.0% confidence -44.61 +/- 2.33254 -7.79305% +/- 0.407478% (Student's t, pooled s = 1.02909) % 3 400.4 401.05 400.62 400.69 0.3306055 Difference at 95.0% confidence -171.743 +/- 2.16453 -30.0023% +/- 0.378128% (Student's t, pooled s = 0.954969) # 3 201.94 203.34 202.28 202.52 0.73020545 Difference at 95.0% confidence -369.913 +/- 2.40293 -64.6212% +/- 0.419774% (Student's t, pooled s = 1.06015) @ 3 369.12 370.57 369.3 369.66333 0.79033748 Difference at 95.0% confidence -202.77 +/- 2.45131 -35.4225% +/- 0.428227% (Student's t, pooled s = 1.0815) [1] https://ccache.samba.org/performance.html [2] http://www.mail-archive.com/ccache@lists.samba.org/msg00576.html [3] https://reviews.freebsd.org/D3484 [5] https://github.com/jrosdahl/ccache/pull/30 PR: 182944 [4] MFC after: 3 weeks Sponsored by: EMC / Isilon Storage Division Relnotes: yes
2015-11-08 00:50:18 +00:00
${CC:N${CCACHE_BIN}} ${PICFLAG} -DPIC ${CFLAGS} ${ACFLAGS} \
-c ${.IMPSRC} -o ${.TARGET}
${CTFCONVERT_CMD}
.S.nossppico:
${CC:N${CCACHE_BIN}} ${PICFLAG} -DPIC ${CFLAGS:C/^-fstack-protector.*$//} ${ACFLAGS} \
-c ${.IMPSRC} -o ${.TARGET}
${CTFCONVERT_CMD}
.S.pieo:
${CC:N${CCACHE_BIN}} ${PIEFLAG} -DPIC ${CFLAGS} ${ACFLAGS} \
-c ${.IMPSRC} -o ${.TARGET}
${CTFCONVERT_CMD}
_LIBDIR:=${LIBDIR}
_SHLIBDIR:=${SHLIBDIR}
.if defined(SHLIB_NAME)
.if ${MK_DEBUG_FILES} != "no"
SHLIB_NAME_FULL=${SHLIB_NAME}.full
# Use ${DEBUGDIR} for base system debug files, else .debug subdirectory
.if ${_SHLIBDIR} == "/boot" ||\
${SHLIBDIR:C%/lib(/.*)?$%/lib%} == "/lib" ||\
${SHLIBDIR:C%/usr/(tests/)?lib(32|exec)?(/.*)?%/usr/lib%} == "/usr/lib"
DEBUGFILEDIR=${DEBUGDIR}${_SHLIBDIR}
.else
DEBUGFILEDIR=${_SHLIBDIR}/.debug
.endif
.if !exists(${DESTDIR}${DEBUGFILEDIR})
DEBUGMKDIR=
.endif
.else
SHLIB_NAME_FULL=${SHLIB_NAME}
.endif
.endif
.include <bsd.symver.mk>
2008-04-09 20:27:53 +00:00
# Allow libraries to specify their own version map or have it
# automatically generated (see bsd.symver.mk above).
.if !empty(VERSION_MAP)
${SHLIB_NAME_FULL}: ${VERSION_MAP}
LDFLAGS+= -Wl,--version-script=${VERSION_MAP}
.endif
.if defined(LIB) && !empty(LIB) || defined(SHLIB_NAME)
OBJS+= ${SRCS:N*.h:${OBJS_SRCS_FILTER:ts:}:S/$/.o/}
BCOBJS+= ${SRCS:N*.[hsS]:N*.asm:${OBJS_SRCS_FILTER:ts:}:S/$/.bco/g}
LLOBJS+= ${SRCS:N*.[hsS]:N*.asm:${OBJS_SRCS_FILTER:ts:}:S/$/.llo/g}
CLEANFILES+= ${OBJS} ${BCOBJS} ${LLOBJS} ${STATICOBJS}
.endif
.if defined(LIB) && !empty(LIB)
_LIBS= lib${LIB_PRIVATE}${LIB}.a
2002-04-20 08:41:55 +00:00
lib${LIB_PRIVATE}${LIB}.a: ${OBJS} ${STATICOBJS}
@${ECHO} building static ${LIB} library
@rm -f ${.TARGET}
Stop using lorder and ranlib when building libraries Use of ranlib or lorder is no longer necessary with current linkers (probably anything newer than ~1990) and ar's ability to create an object index and symbol table in the archive. Currently the build system uses lorder+tsort to sort the .o files in dependency order so that a single-pass linker can use them. However, we can use the -s flag to ar to add an index to the .a file which makes lorder unnecessary. Running ar -s is equivalent to running ranlib afterwards, so we can also skip the ranlib invocation. Similarly, we don't have to pass the .o files for shared libraries in dependency order since both ld.bfd and ld.lld will correctly resolve references between the .o files. This removes many fork()+execve calls for each library so should speed up builds a bit. Additionally lorder.sh uses a regular expression that is not supported by the macOS libc or glibc and results in many warnings when cross-building (see D25989). There is one functional change: lorder.sh removed duplicated .o files from the linker command line which now no longer happens. I fixed the duplicates in the base system in r364649. I also checked the ports tree for uses of bsd.lib.mk and found one duplicate source file which I fixed in r548168. Most ports use CMake/autotools rather than bsd.lib.mk but if this breaks any ports that I missed in my search please let me know. Avoiding the shell script actually speeds up the linking step noticeably: I measured how long it takes to rebuild the .a and .so files for lib/libc using a basic benchmark: `rm $LIBC_OBJDIR/*.so* $LIBC_OBJDIR/*.a* && /usr/bin/time make -DWITHOUT_TESTS -s > /dev/null` Without this change ~4.5 seconds and afterwards ~3.1 seconds. Looking at truss -cf output we can see that the number fork() system calls goes down from 27 to 12 (and the speedup while tracing is more noticeable: 81 seconds -> 65 seconds). See also https://www.gnu.org/software/coreutils/manual/html_node/tsort-background.html for some more background: This whole procedure has been obsolete since about 1980, because Unix archives now contain a symbol table (traditionally built by ranlib, now generally built by ar itself), and the Unix linker uses the symbol table to effectively make multiple passes over an archive file. Or alternatively https://www.unix.com/man-page/osf1/1/lorder/: The lorder command is essentially obsolete. Use the following command in its place: % ar -ts file.a Reviewed By: emaste, imp, dim Differential Revision: https://reviews.freebsd.org/D26044
2020-09-17 15:07:25 +00:00
${AR} ${ARFLAGS} ${.TARGET} ${OBJS} ${STATICOBJS} ${ARADD}
.endif
.if !defined(INTERNALLIB)
1994-05-30 19:09:18 +00:00
.if ${MK_PROFILE} != "no" && defined(LIB) && !empty(LIB)
_LIBS+= lib${LIB_PRIVATE}${LIB}_p.a
POBJS+= ${OBJS:.o=.po} ${STATICOBJS:.o=.po}
DEPENDOBJS+= ${POBJS}
CLEANFILES+= ${POBJS}
lib${LIB_PRIVATE}${LIB}_p.a: ${POBJS}
@${ECHO} building profiled ${LIB} library
@rm -f ${.TARGET}
Stop using lorder and ranlib when building libraries Use of ranlib or lorder is no longer necessary with current linkers (probably anything newer than ~1990) and ar's ability to create an object index and symbol table in the archive. Currently the build system uses lorder+tsort to sort the .o files in dependency order so that a single-pass linker can use them. However, we can use the -s flag to ar to add an index to the .a file which makes lorder unnecessary. Running ar -s is equivalent to running ranlib afterwards, so we can also skip the ranlib invocation. Similarly, we don't have to pass the .o files for shared libraries in dependency order since both ld.bfd and ld.lld will correctly resolve references between the .o files. This removes many fork()+execve calls for each library so should speed up builds a bit. Additionally lorder.sh uses a regular expression that is not supported by the macOS libc or glibc and results in many warnings when cross-building (see D25989). There is one functional change: lorder.sh removed duplicated .o files from the linker command line which now no longer happens. I fixed the duplicates in the base system in r364649. I also checked the ports tree for uses of bsd.lib.mk and found one duplicate source file which I fixed in r548168. Most ports use CMake/autotools rather than bsd.lib.mk but if this breaks any ports that I missed in my search please let me know. Avoiding the shell script actually speeds up the linking step noticeably: I measured how long it takes to rebuild the .a and .so files for lib/libc using a basic benchmark: `rm $LIBC_OBJDIR/*.so* $LIBC_OBJDIR/*.a* && /usr/bin/time make -DWITHOUT_TESTS -s > /dev/null` Without this change ~4.5 seconds and afterwards ~3.1 seconds. Looking at truss -cf output we can see that the number fork() system calls goes down from 27 to 12 (and the speedup while tracing is more noticeable: 81 seconds -> 65 seconds). See also https://www.gnu.org/software/coreutils/manual/html_node/tsort-background.html for some more background: This whole procedure has been obsolete since about 1980, because Unix archives now contain a symbol table (traditionally built by ranlib, now generally built by ar itself), and the Unix linker uses the symbol table to effectively make multiple passes over an archive file. Or alternatively https://www.unix.com/man-page/osf1/1/lorder/: The lorder command is essentially obsolete. Use the following command in its place: % ar -ts file.a Reviewed By: emaste, imp, dim Differential Revision: https://reviews.freebsd.org/D26044
2020-09-17 15:07:25 +00:00
${AR} ${ARFLAGS} ${.TARGET} ${POBJS} ${ARADD}
.endif
.if defined(LLVM_LINK)
lib${LIB_PRIVATE}${LIB}.bc: ${BCOBJS}
${LLVM_LINK} -o ${.TARGET} ${BCOBJS}
lib${LIB_PRIVATE}${LIB}.ll: ${LLOBJS}
${LLVM_LINK} -S -o ${.TARGET} ${LLOBJS}
CLEANFILES+= lib${LIB_PRIVATE}${LIB}.bc lib${LIB_PRIVATE}${LIB}.ll
.endif
.if defined(SHLIB_NAME) || \
defined(INSTALL_PIC_ARCHIVE) && defined(LIB) && !empty(LIB)
SOBJS+= ${OBJS:.o=.pico}
DEPENDOBJS+= ${SOBJS}
CLEANFILES+= ${SOBJS}
.endif
.if defined(SHLIB_NAME)
_LIBS+= ${SHLIB_NAME}
SOLINKOPTS+= -shared -Wl,-x
.if defined(LD_FATAL_WARNINGS) && ${LD_FATAL_WARNINGS} == "no"
SOLINKOPTS+= -Wl,--no-fatal-warnings
.else
SOLINKOPTS+= -Wl,--fatal-warnings
.endif
SOLINKOPTS+= -Wl,--warn-shared-textrel
.if target(beforelinking)
beforelinking: ${SOBJS}
${SHLIB_NAME_FULL}: beforelinking
.endif
.if defined(SHLIB_LINK)
.if defined(SHLIB_LDSCRIPT) && !empty(SHLIB_LDSCRIPT) && exists(${.CURDIR}/${SHLIB_LDSCRIPT})
${SHLIB_LINK:R}.ld: ${.CURDIR}/${SHLIB_LDSCRIPT}
sed -e 's,@@SHLIB@@,${_SHLIBDIR}/${SHLIB_NAME},g' \
-e 's,@@LIBDIR@@,${_LIBDIR},g' \
${.ALLSRC} > ${.TARGET}
${SHLIB_NAME_FULL}: ${SHLIB_LINK:R}.ld
CLEANFILES+= ${SHLIB_LINK:R}.ld
.endif
CLEANFILES+= ${SHLIB_LINK}
.endif
${SHLIB_NAME_FULL}: ${SOBJS}
@${ECHO} building shared library ${SHLIB_NAME}
@rm -f ${SHLIB_NAME} ${SHLIB_LINK}
.if defined(SHLIB_LINK) && !commands(${SHLIB_LINK:R}.ld) && ${MK_DEBUG_FILES} == "no"
# Note: This uses ln instead of ${INSTALL_LIBSYMLINK} since we are in OBJDIR
@${LN:Uln} -fs ${SHLIB_NAME} ${SHLIB_LINK}
.endif
Add built-in ccache build support via WITH_CCACHE_BUILD option. ccache is mostly beneficial for frequent builds where -DNO_CLEAN is not used to achieve a safe pseudo-incremental build. This is explained in more detail upstream [1] [2]. It incurs about a 20%-28% hit to populate the cache, but with a full cache saves 30-50% in build times. When combined with the WITH_FAST_DEPEND feature it saves up to 65% since ccache does cache the resulting dependency file, which it does not do when using mkdep(1)/'CC -E'. Stats are provided at the end of this message. This removes the need to modify /etc/make.conf with the CC:= and CXX:= lines which conflicted with external compiler support [3] (causing the bootstrap compiler to not be built which lead to obscure failures [4]), incorrectly invoked ccache in various stages, required CCACHE_CPP2 to avoid Clang errors with parenthesis, and did not work with META_MODE. The option name was picked to match the existing option in ports. This feature is available for both in-src and out-of-src builds that use /usr/share/mk. Linking, assembly compiles, and pre-processing avoid using ccache since it is only overhead. ccache does nothing special in these modes, although there is no harm in calling it for them. CCACHE_COMPILERCHECK is set to 'content' when using the in-tree bootstrap compiler to hash the content of the compiler binary to determine if it should be a cache miss. For external compilers the 'mtime' option is used as it is more efficient and likely to be correct. Future work may optimize the 'content' check using the same checks as whether a bootstrap compiler is needed to be built. The CCACHE_CPP2 pessimization is currently default in our devel/ccache port due to Clang requiring it. Clang's -Wparentheses-equality, -Wtautological-compare, and -Wself-assign warnings do not mix well with compiling already-pre-processed code that may have expanded macros that trigger the warnings. GCC has so far not had this issue so it is allowed to disable the CCACHE_CPP2 default in our port. Sharing a cache between multiple checkouts, or systems, is explained in the ccache manual. Sharing a cache over NFS would likely not be worth it, but syncing cache directories between systems may be useful for an organization. There is also a memcached backend available [5]. Due to using an object directory outside of the source directory though you will need to ensure that both are in the same prefix and all users use the same layout. A possible working layout is as follows: Source: /some/prefix/src1 Source: /some/prefix/src2 Source: /some/prefix/src3 Objdir: /some/prefix/obj Environment: CCACHE_BASEDIR='${SRCTOP:H}' MAKEOBJDIRPREFIX='${SRCTOP:H}/obj' This will use src*/../obj as the MAKEOBJDIRPREFIX and tells ccache to replace all absolute paths to be relative. Using something like this is required due to -I and -o flags containing both SRC and OBJDIR absolute paths that ccache adds into its hash for the object without CCACHE_BASEDIR. distcc can be hooked into by setting CCACHE_PREFIX=/usr/local/bin/distcc. I have not personally tested this and assume it will not mix well with using the bootstrap compiler. The cache from buildworld can be reused in a subdir by first running 'make buildenv' (from r290424). Note that the cache is currently different depending on whether -j is used or not due to ccache enabling -fdiagnostics-color automatically if stderr is a TTY, which bmake only does if not using -j. The system I used for testing was: WITNESS Build options: -j20 WITH_LLDB=yes WITH_DEBUG_FILES=yes WITH_CCACHE_BUILD=yes DISK: ZFS 3-way mirror with very slow disks using SSD l2arc/log. The arc was fully populated with src tree files and ccache objects. RAM: 76GiB CPU: Intel(R) Xeon(R) CPU L5520 @2.27GHz 2 package(s) x 4 core(s) x 2 SMT threads = hw.ncpu=16 The WITH_FAST_DEPEND feature was used for comparison here as well to show the dramatic time savings with a full cache. buildworld: x buildworld-before + buildworld-ccache-empty * buildworld-ccache-full % buildworld-ccache-full-fastdep # buildworld-fastdep +-------------------------------------------------------------------------------+ |% * # +| |% * # +| |% * # xxx +| | |A | | A| | A | |A | | A | +-------------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 3 3744.13 3794.31 3752.25 3763.5633 26.935139 + 3 4519 4525.04 4520.73 4521.59 3.1104823 Difference at 95.0% confidence 758.027 +/- 43.4565 20.1412% +/- 1.15466% (Student's t, pooled s = 19.1726) * 3 1823.08 1827.2 1825.62 1825.3 2.0785572 Difference at 95.0% confidence -1938.26 +/- 43.298 -51.5007% +/- 1.15045% (Student's t, pooled s = 19.1026) % 3 1266.96 1279.37 1270.47 1272.2667 6.3971113 Difference at 95.0% confidence -2491.3 +/- 44.3704 -66.1952% +/- 1.17895% (Student's t, pooled s = 19.5758) # 3 3153.34 3155.16 3154.2 3154.2333 0.91045776 Difference at 95.0% confidence -609.33 +/- 43.1943 -16.1902% +/- 1.1477% (Student's t, pooled s = 19.0569) buildkernel: x buildkernel-before + buildkernel-ccache-empty * buildkernel-ccache-empty-fastdep % buildkernel-ccache-full # buildkernel-ccache-full-fastdep @ buildkernel-fastdep +-------------------------------------------------------------------------------+ |# @ % * | |# @ % * x + | |# @ % * xx ++| | MA | | MA| | A | | A | |A | | A | +-------------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 3 571.57 573.94 571.79 572.43333 1.3094401 + 3 727.97 731.91 728.06 729.31333 2.2492295 Difference at 95.0% confidence 156.88 +/- 4.17129 27.4058% +/- 0.728695% (Student's t, pooled s = 1.84034) * 3 527.1 528.29 528.08 527.82333 0.63516402 Difference at 95.0% confidence -44.61 +/- 2.33254 -7.79305% +/- 0.407478% (Student's t, pooled s = 1.02909) % 3 400.4 401.05 400.62 400.69 0.3306055 Difference at 95.0% confidence -171.743 +/- 2.16453 -30.0023% +/- 0.378128% (Student's t, pooled s = 0.954969) # 3 201.94 203.34 202.28 202.52 0.73020545 Difference at 95.0% confidence -369.913 +/- 2.40293 -64.6212% +/- 0.419774% (Student's t, pooled s = 1.06015) @ 3 369.12 370.57 369.3 369.66333 0.79033748 Difference at 95.0% confidence -202.77 +/- 2.45131 -35.4225% +/- 0.428227% (Student's t, pooled s = 1.0815) [1] https://ccache.samba.org/performance.html [2] http://www.mail-archive.com/ccache@lists.samba.org/msg00576.html [3] https://reviews.freebsd.org/D3484 [5] https://github.com/jrosdahl/ccache/pull/30 PR: 182944 [4] MFC after: 3 weeks Sponsored by: EMC / Isilon Storage Division Relnotes: yes
2015-11-08 00:50:18 +00:00
${_LD:N${CCACHE_BIN}} ${LDFLAGS} ${SSP_CFLAGS} ${SOLINKOPTS} \
Stop using lorder and ranlib when building libraries Use of ranlib or lorder is no longer necessary with current linkers (probably anything newer than ~1990) and ar's ability to create an object index and symbol table in the archive. Currently the build system uses lorder+tsort to sort the .o files in dependency order so that a single-pass linker can use them. However, we can use the -s flag to ar to add an index to the .a file which makes lorder unnecessary. Running ar -s is equivalent to running ranlib afterwards, so we can also skip the ranlib invocation. Similarly, we don't have to pass the .o files for shared libraries in dependency order since both ld.bfd and ld.lld will correctly resolve references between the .o files. This removes many fork()+execve calls for each library so should speed up builds a bit. Additionally lorder.sh uses a regular expression that is not supported by the macOS libc or glibc and results in many warnings when cross-building (see D25989). There is one functional change: lorder.sh removed duplicated .o files from the linker command line which now no longer happens. I fixed the duplicates in the base system in r364649. I also checked the ports tree for uses of bsd.lib.mk and found one duplicate source file which I fixed in r548168. Most ports use CMake/autotools rather than bsd.lib.mk but if this breaks any ports that I missed in my search please let me know. Avoiding the shell script actually speeds up the linking step noticeably: I measured how long it takes to rebuild the .a and .so files for lib/libc using a basic benchmark: `rm $LIBC_OBJDIR/*.so* $LIBC_OBJDIR/*.a* && /usr/bin/time make -DWITHOUT_TESTS -s > /dev/null` Without this change ~4.5 seconds and afterwards ~3.1 seconds. Looking at truss -cf output we can see that the number fork() system calls goes down from 27 to 12 (and the speedup while tracing is more noticeable: 81 seconds -> 65 seconds). See also https://www.gnu.org/software/coreutils/manual/html_node/tsort-background.html for some more background: This whole procedure has been obsolete since about 1980, because Unix archives now contain a symbol table (traditionally built by ranlib, now generally built by ar itself), and the Unix linker uses the symbol table to effectively make multiple passes over an archive file. Or alternatively https://www.unix.com/man-page/osf1/1/lorder/: The lorder command is essentially obsolete. Use the following command in its place: % ar -ts file.a Reviewed By: emaste, imp, dim Differential Revision: https://reviews.freebsd.org/D26044
2020-09-17 15:07:25 +00:00
-o ${.TARGET} -Wl,-soname,${SONAME} ${SOBJS} ${LDADD}
.if ${MK_CTF} != "no"
${CTFMERGE} ${CTFFLAGS} -o ${.TARGET} ${SOBJS}
.endif
.if ${MK_DEBUG_FILES} != "no"
CLEANFILES+= ${SHLIB_NAME_FULL} ${SHLIB_NAME}.debug
${SHLIB_NAME}: ${SHLIB_NAME_FULL} ${SHLIB_NAME}.debug
${OBJCOPY} --strip-debug --add-gnu-debuglink=${SHLIB_NAME}.debug \
${SHLIB_NAME_FULL} ${.TARGET}
.if defined(SHLIB_LINK) && !commands(${SHLIB_LINK:R}.ld)
# Note: This uses ln instead of ${INSTALL_LIBSYMLINK} since we are in OBJDIR
@${LN:Uln} -fs ${SHLIB_NAME} ${SHLIB_LINK}
.endif
${SHLIB_NAME}.debug: ${SHLIB_NAME_FULL}
${OBJCOPY} --only-keep-debug ${SHLIB_NAME_FULL} ${.TARGET}
.endif
.endif #defined(SHLIB_NAME)
.if defined(INSTALL_PIC_ARCHIVE) && defined(LIB) && !empty(LIB) && ${MK_TOOLCHAIN} != "no"
_LIBS+= lib${LIB_PRIVATE}${LIB}_pic.a
lib${LIB_PRIVATE}${LIB}_pic.a: ${SOBJS}
@${ECHO} building special pic ${LIB} library
@rm -f ${.TARGET}
${AR} ${ARFLAGS} ${.TARGET} ${SOBJS} ${ARADD}
.endif
1994-05-30 19:09:18 +00:00
.if defined(BUILD_NOSSP_PIC_ARCHIVE) && defined(LIB) && !empty(LIB)
NOSSPSOBJS+= ${OBJS:.o=.nossppico}
DEPENDOBJS+= ${NOSSPSOBJS}
CLEANFILES+= ${NOSSPSOBJS}
_LIBS+= lib${LIB_PRIVATE}${LIB}_nossp_pic.a
lib${LIB_PRIVATE}${LIB}_nossp_pic.a: ${NOSSPSOBJS}
@${ECHO} building special nossp pic ${LIB} library
@rm -f ${.TARGET}
${AR} ${ARFLAGS} ${.TARGET} ${NOSSPSOBJS} ${ARADD}
.endif
.endif # !defined(INTERNALLIB)
.if defined(INTERNALLIB) && ${MK_PIE} != "no"
PIEOBJS+= ${OBJS:.o=.pieo}
DEPENDOBJS+= ${PIEOBJS}
CLEANFILES+= ${PIEOBJS}
_LIBS+= lib${LIB_PRIVATE}${LIB}_pie.a
lib${LIB_PRIVATE}${LIB}_pie.a: ${PIEOBJS}
@${ECHO} building pie ${LIB} library
@rm -f ${.TARGET}
${AR} ${ARFLAGS} ${.TARGET} ${PIEOBJS} ${ARADD}
.endif
.if defined(_SKIP_BUILD)
all:
.else
.if defined(_LIBS) && !empty(_LIBS)
all: ${_LIBS}
.endif
.if ${MK_MAN} != "no" && !defined(LIBRARIES_ONLY)
all: all-man
1994-05-30 19:09:18 +00:00
.endif
.endif
1994-05-30 19:09:18 +00:00
CLEANFILES+= ${_LIBS}
_EXTRADEPEND:
2004-12-21 09:42:10 +00:00
.if !defined(NO_EXTRADEPEND) && defined(SHLIB_NAME)
.if defined(DPADD) && !empty(DPADD)
echo ${SHLIB_NAME_FULL}: ${DPADD} >> ${DEPENDFILE}
.endif
.endif
1994-05-30 19:09:18 +00:00
.if !target(install)
.if defined(PRECIOUSLIB)
2004-12-21 09:53:14 +00:00
.if !defined(NO_FSCHG)
SHLINSTALLFLAGS+= -fschg
.endif
.endif
# Install libraries with -S to avoid risk of modifying in-use libraries when
# installing to a running system. It is safe to avoid this for NO_ROOT builds
# that are only creating an image.
#
# XXX: Since Makefile.inc1 ends up building lib/libc both as part of
# _startup_libs and as part of _generic_libs it ends up getting installed a
# second time during the parallel build, and although the .WAIT in lib/Makefile
# stops that mattering for lib, other directories like secure/lib are built in
# parallel at the top level and are unaffected by that, so can sometimes race
# with the libc.so.7 reinstall and see a missing or corrupt file. Ideally the
# build system would be fixed to not build/install libc to WORLDTMP the second
# time round, but for now using -S ensures the install is atomic and thus we
# never see a broken intermediate state, so use it even for NO_ROOT builds.
.if !defined(NO_SAFE_LIBINSTALL) #&& !defined(NO_ROOT)
SHLINSTALLFLAGS+= -S
SHLINSTALLSYMLINKFLAGS+= -S
.endif
_INSTALLFLAGS:= ${INSTALLFLAGS}
.for ie in ${INSTALLFLAGS_EDIT}
_INSTALLFLAGS:= ${_INSTALLFLAGS${ie}}
.endfor
_SHLINSTALLFLAGS:= ${SHLINSTALLFLAGS}
_SHLINSTALLSYMLINKFLAGS:= ${SHLINSTALLSYMLINKFLAGS}
.for ie in ${INSTALLFLAGS_EDIT}
_SHLINSTALLFLAGS:= ${_SHLINSTALLFLAGS${ie}}
.endfor
.if defined(PCFILES)
.for pcfile in ${PCFILES}
installpcfiles: installpcfiles-${pcfile}
installpcfiles-${pcfile}: ${pcfile}
${INSTALL} ${TAG_ARGS:D${TAG_ARGS},dev} -o ${LIBOWN} -g ${LIBGRP} -m ${LIBMODE} \
${_INSTALLFLAGS} \
${.ALLSRC} ${DESTDIR}${LIBDATADIR}/pkgconfig
.endfor
.endif
installpcfiles: .PHONY
.if !defined(INTERNALLIB)
realinstall: _libinstall installpcfiles
.ORDER: beforeinstall _libinstall
_libinstall:
.if defined(LIB) && !empty(LIB) && ${MK_INSTALLLIB} != "no"
${INSTALL} ${TAG_ARGS:D${TAG_ARGS},dev} -C -o ${LIBOWN} -g ${LIBGRP} -m ${LIBMODE} \
${_INSTALLFLAGS} lib${LIB_PRIVATE}${LIB}.a ${DESTDIR}${_LIBDIR}/
.endif
.if ${MK_PROFILE} != "no" && defined(LIB) && !empty(LIB)
${INSTALL} ${TAG_ARGS:D${TAG_ARGS},dev} -C -o ${LIBOWN} -g ${LIBGRP} -m ${LIBMODE} \
${_INSTALLFLAGS} lib${LIB_PRIVATE}${LIB}_p.a ${DESTDIR}${_LIBDIR}/
1994-05-30 19:09:18 +00:00
.endif
.if defined(SHLIB_NAME)
${INSTALL} ${TAG_ARGS} ${STRIP} -o ${LIBOWN} -g ${LIBGRP} -m ${LIBMODE} \
${_INSTALLFLAGS} ${_SHLINSTALLFLAGS} \
${SHLIB_NAME} ${DESTDIR}${_SHLIBDIR}/
.if ${MK_DEBUG_FILES} != "no"
.if defined(DEBUGMKDIR)
${INSTALL} ${TAG_ARGS:D${TAG_ARGS},dbg} -d ${DESTDIR}${DEBUGFILEDIR}/
.endif
${INSTALL} ${TAG_ARGS:D${TAG_ARGS},dbg} -o ${LIBOWN} -g ${LIBGRP} -m ${DEBUGMODE} \
${_INSTALLFLAGS} \
${SHLIB_NAME}.debug ${DESTDIR}${DEBUGFILEDIR}/
.endif
.if defined(SHLIB_LINK)
.if commands(${SHLIB_LINK:R}.ld)
${INSTALL} ${TAG_ARGS:D${TAG_ARGS},dev} -S -C -o ${LIBOWN} -g ${LIBGRP} -m ${LIBMODE} \
${_INSTALLFLAGS} ${SHLIB_LINK:R}.ld \
${DESTDIR}${_LIBDIR}/${SHLIB_LINK}
.for _SHLIB_LINK_LINK in ${SHLIB_LDSCRIPT_LINKS}
${INSTALL_LIBSYMLINK} ${_SHLINSTALLSYMLINKFLAGS} ${TAG_ARGS} ${SHLIB_LINK} \
${DESTDIR}${_LIBDIR}/${_SHLIB_LINK_LINK}
.endfor
.else
.if ${_SHLIBDIR} == ${_LIBDIR}
.if ${SHLIB_LINK:Mlib*}
${INSTALL_RSYMLINK} ${_SHLINSTALLSYMLINKFLAGS} ${TAG_ARGS:D${TAG_ARGS},dev} \
${SHLIB_NAME} ${DESTDIR}${_LIBDIR}/${SHLIB_LINK}
.else
${INSTALL_RSYMLINK} ${_SHLINSTALLSYMLINKFLAGS} ${TAG_ARGS} ${DESTDIR}${_SHLIBDIR}/${SHLIB_NAME} \
${DESTDIR}${_LIBDIR}/${SHLIB_LINK}
.endif
.else
.if ${SHLIB_LINK:Mlib*}
${INSTALL_RSYMLINK} ${_SHLINSTALLSYMLINKFLAGS} ${TAG_ARGS:D${TAG_ARGS},dev} \
${DESTDIR}${_SHLIBDIR}/${SHLIB_NAME} ${DESTDIR}${_LIBDIR}/${SHLIB_LINK}
.else
${INSTALL_RSYMLINK} ${_SHLINSTALLSYMLINKFLAGS} ${TAG_ARGS} \
${DESTDIR}${_SHLIBDIR}/${SHLIB_NAME} ${DESTDIR}${_LIBDIR}/${SHLIB_LINK}
.endif
.if exists(${DESTDIR}${_LIBDIR}/${SHLIB_NAME})
-chflags noschg ${DESTDIR}${_LIBDIR}/${SHLIB_NAME}
rm -f ${DESTDIR}${_LIBDIR}/${SHLIB_NAME}
.endif
.endif # _SHLIBDIR == _LIBDIR
.endif # SHLIB_LDSCRIPT
.endif # SHLIB_LINK
.endif # SHIB_NAME
.if defined(INSTALL_PIC_ARCHIVE) && defined(LIB) && !empty(LIB) && ${MK_TOOLCHAIN} != "no"
${INSTALL} ${TAG_ARGS:D${TAG_ARGS},dev} -o ${LIBOWN} -g ${LIBGRP} -m ${LIBMODE} \
${_INSTALLFLAGS} lib${LIB}_pic.a ${DESTDIR}${_LIBDIR}/
.endif
.endif # !defined(INTERNALLIB)
.if !defined(LIBRARIES_ONLY)
.include <bsd.nls.mk>
.include <bsd.confs.mk>
.include <bsd.files.mk>
#No need to install header for INTERNALLIB
.if !defined(INTERNALLIB)
.include <bsd.incs.mk>
.endif
.endif
LINKOWN?= ${LIBOWN}
LINKGRP?= ${LIBGRP}
LINKMODE?= ${LIBMODE}
SYMLINKOWN?= ${LIBOWN}
SYMLINKGRP?= ${LIBGRP}
.include <bsd.links.mk>
1994-05-30 19:09:18 +00:00
.if ${MK_MAN} != "no" && !defined(LIBRARIES_ONLY)
realinstall: maninstall
.ORDER: beforeinstall maninstall
1994-05-30 19:09:18 +00:00
.endif
1994-05-30 19:09:18 +00:00
.endif
.if ${MK_MAN} != "no" && !defined(LIBRARIES_ONLY)
1994-05-30 19:09:18 +00:00
.include <bsd.man.mk>
.endif
.if defined(LIB) && !empty(LIB)
OBJS_DEPEND_GUESS+= ${SRCS:M*.h}
.for _S in ${SRCS:N*.[hly]}
OBJS_DEPEND_GUESS.${_S:${OBJS_SRCS_FILTER:ts:}}.po+= ${_S}
.endfor
.endif
.if defined(SHLIB_NAME) || \
defined(INSTALL_PIC_ARCHIVE) && defined(LIB) && !empty(LIB)
.for _S in ${SRCS:N*.[hly]}
OBJS_DEPEND_GUESS.${_S:${OBJS_SRCS_FILTER:ts:}}.pico+= ${_S}
.endfor
.endif
.if defined(BUILD_NOSSP_PIC_ARCHIVE) && defined(LIB) && !empty(LIB)
.for _S in ${SRCS:N*.[hly]}
OBJS_DEPEND_GUESS.${_S:${OBJS_SRCS_FILTER:ts:}}.nossppico+= ${_S}
.endfor
.endif
Add limited sandbox capability to "make check" == Rationale == r295380 introduced "make check" and consolidated means for running test code in an attempt to simplify running tests. One could either install files/libraries/programs and run "make check", or run "make check" with an explicit CHECKDIR, e.g., `make check CHECKDIR=$(make -V.OBJDIR)``. One criticism that was received is that "make check" should be run with the intent of making dev->test->commit easier, which means that the target audience's workflow should be developers. One developer pattern available in other opensource projects is to run test code from a developer sandbox, instead of installing to a system. == Method == This approach is slightly different from the standard approach, in the sense that it builds and installs into a deterministic directory under .OBJDIR (as I call it, the "sandbox"), then runs "make check" against that. In the event the test run is successful, the deterministic directory is removed to save space. == Approach == bsd.lib.mk, bsd.prog.mk: To support this functionality, a new variable `HAS_TESTS` is being added. HAS_TESTS enables appropriate behavior with bsd.lib.mk and bsd.prog.mk, as follows: - Add "make check" as an available target from the directory. - Pass down appropriate variables via ${TESTS_ENV}, i.e., ${TESTS_LD_LIBRARY_PATH} and ${TESTS_PATH}. One should add "HAS_TESTS" to directories containing tests in them, e.g. from bin/sh/Makefile, HAS_TESTS= SUBDIR.${MK_TESTS}+= tests HAS_TESTS doesn't automatically add the tests subdirectory for flexibility reasons. bsd.opts.mk, src.opts.mk: - The knob ${MK_MAKE_CHECK_USE_SANDBOX} has been added, both to explicitly direct (internally) when to set a deterministic ${DESTDIR} and to also allow users to disable this behavior globally, i.e., via src.conf. - MK_TESTS has been promoted from src.opts.mk to bsd.opts.mk to leverage syntactic sugar for having MK_TESTS be a dependency for MK_MAKE_CHECK_USE_SANDBOX, but to also ensure that src.opts.mk isn't required to use suite.test.mk (which is a dependency of bsd.test.mk). suite.test.mk: - beforecheck behavior (when MK_MAKE_CHECK_USE_SANDBOX is enabled) is modified from a no-op to: -- Build. -- Run "make hierarchy" on the sandbox dir. -- Install the tests/files to the sandbox dir. - aftercheck behavior (when MK_MAKE_CHECK_USE_SANDBOX is enabled) is modified from a no-op to: -- Remove the sandbox dir. Again, because the dependency order set in bsd.test.mk is beforecheck -> check -> aftercheck, "make check" will not be run unless "beforecheck" completes successfully, and "aftercheck" will not be run unless "beforecheck" and "check" complete successfully. == Caveats == - This target must either be run with MK_INSTALL_AS_USER or as root. Otherwise it will fail when running "make install" as the default user/group for many makefiles when calling INSTALL is root/wheel. - This target must be run from a suitable top-level directory. For example, running tests from `tests/sys/fs/tmpfs` won't work, but `tests/sys/fs` will, because `tests/sys/fs/tmpfs` relies on files installed by `tests/sys/fs`. - Running MK_INSTALL_AS_USER may introduce determinism issues. However, using it could identify deficiences in tests in terms of needing to be run as root, which are not properly articulated in the test requirements. - The doesn't negate the need for running "make installworld" and "make checkworld", etc. Again, this just is intended to simplify the dev->test->commit workflow. == Cleanup done == - CHECKDIR is removed; one can use "MK_MAKE_CHECK_USE_SANDBOX=no" to enable "legacy" (r295380) behavior. MFC after: 2 months Relnotes: yes (CHECKDIR removed; "make check" behavior changed) Requested by: jhb Reviewed by: arch (silence), testing (silence) Differential Revision: D11905
2017-08-14 19:03:05 +00:00
.if defined(HAS_TESTS)
MAKE+= MK_MAKE_CHECK_USE_SANDBOX=yes
SUBDIR_TARGETS+= check
TESTS_LD_LIBRARY_PATH+= ${.OBJDIR}
.endif
.include <bsd.dep.mk>
.include <bsd.clang-analyze.mk>
.include <bsd.obj.mk>
.include <bsd.sys.mk>