Commit Graph

2290 Commits

Author SHA1 Message Date
Bryan Drewery
4e461ce12d Don't hide stderr when checking ${CC} --version.
This can have important debugging information such as 'cc: not found' or
'ccache: error: Could not find compiler "cc" in PATH'.

MFC after:	2 weeks
Sponsored by:	EMC / Isilon Storage Division
2015-10-31 02:07:30 +00:00
Enji Cooper
d04fda6cc4 Unbreak bsd.progs.mk with PROGS (but not PROGS_CXX) and when invoking the
"one of many" targets, e.g. `make hello_world`, where hello_world is a C
program

Tested with: PROGS and PROGS_CXX
MFC after: 1 week
X-MFC with: r289289
Sponsored by: EMC / Isilon Storage Division
2015-10-30 06:07:41 +00:00
Bryan Drewery
0edd283235 Fix regression from using .USEBEFORE in _SUBDIR in r289705.
Using .USEBEFORE had the unintended side-effect of changing the directory for
the real target ran in the current directory.  For example this meant that
the 'make clean' would run in one of the SUBDIR.

Sponsored by:	EMC / Isilon Storage Division
Pointyhat to:	bdrewery
2015-10-29 00:41:03 +00:00
Bryan Drewery
523e46d486 Use proper CONFDIR after r289148 2015-10-27 23:49:32 +00:00
Bryan Drewery
222c700312 Spell CONFGROUPS properly. 2015-10-27 02:04:21 +00:00
Bryan Drewery
99330d7a24 META MODE: Fix after addition of bsd.confs.mk: Stage in "files" set.
The "files" staging name is not the same as "bsd.files.mk" but seems to just be
a group of extra files that are not the essential includes or libraries, which
include .

Sponsored by:	EMC / Isilon Storage Division
2015-10-27 01:29:38 +00:00
Bryan Drewery
b0be2b128f META MODE: Fix FILESNAME not being respected sans other FILES_group overrides.
This was fixed in bsd.incs.mk in r242801 already.

Sponsored by:	EMC / Isilon Storage Division
2015-10-26 23:28:35 +00:00
Bryan Drewery
7847b2337e META MODE: bsd.files.mk and bsd.confs.mk don't handle symlinks so there is no
need to set STAGE_SYMLINKS_DIR.${STAGE_SET}.

Sponsored by:	EMC / Isilon Storage Division
2015-10-26 22:31:57 +00:00
Bryan Drewery
8870ad3609 Sort properly.
MFC after:	1 week
X-MFC-With:	r289870
2015-10-24 05:00:20 +00:00
Bryan Drewery
ce3c9a36c5 Add bsd.crunchgen.mk to bsd.README.
MFC after:	1 week
2015-10-24 04:55:17 +00:00
Bryan Drewery
8758abd5b3 Slightly rework the comments and logic for default Clang/GCC.
This is because the previous version was very obscure about the fact
that despite having Clang "on by default" for architectures such as powerpc, it
does not actually build due to the GCC it uses not having C++11 support.
Using an external compiler that supports C++11 does allow this to work.
This whole block should be rethought more given "on by default" is not
really default without extra work which could actually be surprising for
why Clang is showing up when using a newer GCC.

Sponsored by:	EMC / Isilon Storage Division
2015-10-24 04:03:32 +00:00
Bryan Drewery
de606f83e0 Configs should not be under MK_INCLUDES control.
'buildconfig' is connected to 'all', but 'installconfig' is only called
manually.  There is not much need to conditionalize this file right
now due to how it is hooked up and its impact on various build phases.

Sponsored by:	EMC / Isilon Storage Division
2015-10-24 04:03:29 +00:00
Bryan Drewery
ccfb12d63a Rework r289778 to always parallelize known targets, without ordering.
- Rather than allow 'make clean*' to ignore dependencies, make a static
  list of targets in STANDALONE_SUBDIR_TARGETS that are known to be safe.
  This allows a user to override them if needed and avoids adding this feature
  to user-defined targets that are in ${SUBDIR_TARGETS}. [1]
- This now also allows to force SUBDIR_PARALLEL when calling these
  targets, since no dependencies are needed.

Reported by:	ian [1]
Sponsored by:	EMC / Isilon Storage Division
MFC after:	3 weeks
X-MFC-With:	r289778
2015-10-23 21:30:27 +00:00
Bryan Drewery
0d2a1539d1 Fix regression from r289734 that caused crunchgen "subdirs" to not be
properly recursed.

The .for loop was defining a ${__dir} variable that was being set at a
different evaluation time than the target itself, so every 'cd ${__dir}'
became the last value that was in ${__dir}.  This resulted in 'make obj'
not properly being ran in the tree that would leave .depend files
scattered around when 'make all' was ran in rescue/.

To fix this, define a CRUNCH_SRCDIR_* for every prog if it does not
already have one and then use that variable in every relevant place.
This allows simplifying some logic as well.

Reported by:	emaste
X-MFC-With:	r289734
MFC after:	3 weeks
Sponsored by:	EMC / Isilon Storage Division
2015-10-23 19:41:58 +00:00
Bryan Drewery
2e35a44ce5 For SUBDIR_PARALLEL, when doing 'make clean*' or 'make obj' there is no need to
respect SUBDIR_DEPEND_* or .WAIT.

MFC after:	2 weeks
Sponsored by:	EMC / Isilon Storage Division
2015-10-22 23:41:56 +00:00
Bryan Drewery
c22afe52f4 Fix installation of manpages with WITHOUT_COMPRESS broken since r284685.
This partially reverts r284685.  An attempt was made in r285295 to fix this
but was not enough.  There were still $${} vars in the code that should have
been using the ${_page} and ${_sect} vars, but the bigger problem was that
.for cannot be used on .ALLSRC as it is not defined when the .for is evaluated.

Using ${MAN} here in a .for loop doesn't work either as the paths are not
expanded right for lib/libc/ subdirs despite having a .PATH set for all
of them.

Add some comments around long .else and .endif as well.

Sponsored by:	EMC / Isilon Storage Division
2015-10-22 22:29:25 +00:00
Bryan Drewery
cc104c812d Fix incorrect defined() usage from style clean up in r289735.
Submitted by:	ngie
MFC after:	2 weeks
X-MFC-With:	r289735
Sponsored by:	EMC / Isilon Storage Division
2015-10-22 04:52:37 +00:00
Bryan Drewery
d4f564c588 Fix style. Namely use {} rather than ().
MFC after:	2 weeks
Sponsored by:	EMC / Isilon Storage Division
2015-10-22 04:47:52 +00:00
Bryan Drewery
5583e49be6 Rewrite crunchgen target handling for progs so that it can be parallelized.
This covers 'clean', 'cleandepend', 'cleandir', 'obj', 'objlink' and
'build-tools'.

This uses the same method as bsd.subdir.mk.

MFC after:	2 weeks
X-MFC-With:	r289731
Sponsored by:	EMC / Isilon Storage Division
2015-10-22 04:42:17 +00:00
Bryan Drewery
01d591d45b Clean up some bsd.crunchgen.mk issues.
- Remove handling of 'make -P' since that is for fmake only.
- Add '+' where appropriate for sub-make calls.
- Pass MK_TESTS=no to all of the sub-makes to prevent recursing into test
  directories for targets such as 'obj', 'clean', 'depend', etc.

Sponsored by:	EMC / Isilon Storage Division
2015-10-22 04:28:22 +00:00
Bryan Drewery
7f5f4b449d Remove indirection of _sub target for using _SUBDIR.
This reverts r266473 as the need for it, working around .MAKE and '+'
issues, is no longer needed after r289460.  This avoids extra log
output in -j builds of '-- _sub.TARGET --' that are redundant with the
'-- TARGET --' and '-- TARGET_subdir_DIR --' entries already showing.

r266473 also made a subtle change in the ordering of _SUBDIR handling.  Before
the change, SUBDIRS were recursed into after building the TARGET due to the
.USE of _SUBDIR *appending* the commands onto the TARGET.  After the change
though the indirection caused TARGET to depend on _sub.TARGET which had the
_SUBDIR handling in it.  This TARGET would run after recursing.  However, the
SUBDIR_PARALLEL handling from r263778 has this ordering as well.  Since
this has so far not been a problem, for now make this behavior for
non-SUBDIR_PARALLEL use of _SUBDIR explicit by using .USEBEFORE.
Further research may change this back to .USE as well as the
SUBDIR_PARALLEL handling and bsd.progs.mk recursing.

Sponsored by:	EMC / Isilon Storage Division
2015-10-21 16:24:44 +00:00
Bryan Drewery
5d6be39a4b Fix building in a directory with SUBDIRs and SUBDIR_PARALLEL.
The SUBDIR_PARALLEL feature uses a .for dir in ${SUBDIR} loop.  The old code
here for recursing was setting SUBDIR= as a make *argument*.  The SUBDIR=
replacement was not actually handled until after the .for loop was unrolled.
This could be seen with a '.info ${SUBDIR} ${dir}' inside of the loop which
showed an empty ${SUBDIR} and a set ${dir}.  Setting NO_SUBIDR= before calling
${MAKE} as an *environment* variable handles the case fine and is a more
proper mechanism for disabling subdir handling.

This could be seen with 'make -C tests/sys/kern -j15 SUBDIR_PARALLEL=yes'.

MFC after:	2 weeks
Sponsored by:	EMC / Isilon Storage Division
2015-10-21 00:25:18 +00:00
Bryan Drewery
13569c0ae5 Improve safety of caching from r289659 by only importing of none of the
variables are already set.  This should cover odd cases such as the
COMPILER_TYPE override in lib/csu/powerpc64.

X-MFC-With:	r289659
MFC after:	3 weeks
Sponsored by:	EMC / Isilon Storage Division
2015-10-20 20:37:00 +00:00
Bryan Drewery
16a5c008cc Pass COMPILER_TYPE and COMPILER_VERSION to sub-makes to avoid redundant
lookups.

This uses a special variable name based on a hash of ${CC}, ${PATH}, and
${MACHINE} to ensure that a cached value is not used if any of these
values changes to use a new compiler.

Before this there were 34,620 fork/exec from bsd.compiler.mk during a buildworld.
After this there are 608.  More improvement is needed to cache a value from
the top-level before descending into subdirs in the various build phases.

Reviewed by:	brooks (earlier version)
MFC after:	3 weeks
Sponsored by:	EMC / Isilon Storage Division
Differential Revision:	https://reviews.freebsd.org/D3898
2015-10-20 20:15:25 +00:00
Bryan Drewery
65d88005ef Replace all of the duplicated logic for recursing into a subdir with one
implementation.  It is duplicated at run-time but is more easily
maintainable now.

Sponsored by:	EMC / Isilon Storage Division
2015-10-19 23:34:35 +00:00
Bryan Drewery
e92818f2e6 Add missing .PHONY for parallel subdir target.
MFC after:	1 week
Sponsored by:	EMC / Isilon Storage Division
2015-10-19 22:27:32 +00:00
Enji Cooper
550d2b80ec Make libxo depend on libutil because it uses humanize_number after r287111
Remove overlinking in lib/libxo/tests, sbin/savecore, and
usr.bin/{iscsictl,wc,xo}

PR: 203673
Sponsored by: EMC / Isilon Storage Division
2015-10-18 07:30:50 +00:00
Enji Cooper
a52bef9c99 Only enable -fstack-protector-strong on gcc 4.9+ and default to -fstack-protector
when -fstack-protector-strong is not available, like it was implicitly before
r288669

As noted by antoine@, devel/gcc (which is 4.8.5) lacks -fstack-protector-strong
support, whereas 4.8.4i (devel/gcc48) has the support.

Until a version is available which has -fstack-protector-strong support, be
conservative and only enable support with 4.9+.

Reviewed by: pfg
X-MFC with: r288669, r289465
Differential Revision: https://reviews.freebsd.org/D3924
2015-10-18 04:07:40 +00:00
Enji Cooper
835f328986 Only use -fstack-protector-strong with supported compilers
This includes clang 3.5.0+, gcc 4.2.1, gcc 4.8.4+

This allows me to do subdirectory makes again after setting
MAKESYSPATH on 10.2-RELEASE as it comes with clang 3.4.1.

As a sidenote: this isn't technically correct for all vintages
of gcc 4.2.1, but will be correct when gcc is rebuilt/reinstalled
after r286074, so this version check should be good enough.

X-MFC with: r288669
Differential Revision: https://reviews.freebsd.org/D3924
Reviewed by: emaste, pfg
2015-10-17 19:48:17 +00:00
Bryan Drewery
56585ab576 Rework the 'make -n -n' feature such that '-n' recurses and '-N' does not.
Bmake has a documented feature of '-N' to skip executing commands which is
specifically intended for debugging top-level builds and not recursing into
sub-directories.  This matches the older 'make -n' behavior we added which made
'-n -n' the recursing target and '-n' a non-recursing target.

Removing the '-n -n' feature allows the build to work as documented in
the bmake manpage with '-n' and '-N'.  The older '-n -n' feature was also
not documented anywhere that I could see.

Note that the ${_+_} var is still needed as currently bmake incorrectly
executes '+' commands when '-N' is specified.

The '-n' and '-n -n' features were broken for several reasons prior to this.
r251748 made '_+_' never expand with '-n -n' which resulted in many
sub-directories not being visited until fixed 2 years later in r288391, and
many targets were given .MAKE over the past few years which resulted in
non-sub-make commands, such as rm and ln and mtree, to be executed.

This should also allow removing some indirection hacks in bsd.subdir.mk and
other cases of .USE that have a .MAKE by using '+'.

Sponsored by:	EMC / Isilon Storage Division
Discussed on:	arch@ (mostly silence)
2015-10-17 16:42:54 +00:00
Bryan Drewery
29f297420d Similar to r289355, /usr/tests is within the base system so put the symbols
into /usr/lib/debug.

This covers some missing files:
  /usr/tests/libexec/rtld-elf/.debug
  /usr/tests/libexec/rtld-elf/.debug/libpythagoras.so.0.debug
  /usr/tests/lib/libc/tls/.debug
  /usr/tests/lib/libc/tls/.debug/libh_tls_dynamic.so.1.debug
  /usr/tests/lib/libc/tls/.debug/h_tls_dlopen.so.debug
  /usr/tests/lib/libthr/dlopen/.debug
  /usr/tests/lib/libthr/dlopen/.debug/h_pthread_dlopen.so.1.debug
  /usr/tests/lib/libxo/.debug
  /usr/tests/lib/libxo/.debug/libenc_test.so.debug

Sponsored by:	EMC / Isilon Storage Division
2015-10-16 05:06:43 +00:00
Bryan Drewery
61c20fc782 Make installing to a non-existent directory an error.
Before this, if a file was installed to DESTDIR/some/dir and that directory
was missing due to not having ran 'make distrib-dirs' yet, the file would
be installed as 'some/dir'.  For something like bsd.incs.mk with INCLUDEDIR
being a sub-directory of /usr/include, this could result in all of the headers
being installed to a file rather than getting a directory of them.

Now it will error that the file/directory does not exist rather than hide
the issue.

Another option being discussed is to implement GNU's install -D flag which
would auto create any missing directories.

This is a mitigation of the problem.  The proper order to the build is to
run 'make distrib-dirs' first, but that can be forgotten if building from
a sub-directory after updating the source code to the latest revision.

MFC after:	2 weeks
Sponsored by:	EMC / Isilon Storage Division
2015-10-15 22:49:56 +00:00
Bryan Drewery
8251c561bf Fix wrong use of .for; the iteration variable is not used in the loop.
MFC after:	1 week
Sponsored by:	EMC / Isilon Storage Division
2015-10-15 18:55:43 +00:00
Bryan Drewery
dcbf2188c1 Remove excess .else
MFC after:	1 week
Sponsored by:	EMC / Isilon Storage Division
2015-10-15 18:53:26 +00:00
Ed Maste
692432bb20 /usr/tests is part of the base system (for *.debug files)
Sponsored by:	The FreeBSD Foundation
2015-10-15 02:52:30 +00:00
Bryan Drewery
08cf86fef8 Add missing targets to PHONY_NOTMAIN.
- buildconfig, installconfig (missed in r289085)
- files (missed in r241298)

Sponsored by:	EMC / Isilon Storage Division
2015-10-14 20:38:51 +00:00
Bryan Drewery
90df76ab62 Recurse on 'buildconfig' and 'installconfig'. Remove the 'config' pseudo target.
The 'config' target isn't really needed right now so just remove it to avoid
any clashes with config(8) building.  It's also likely misspelled and should
be 'configs' if we decide to add it back.  This was just a convenience
target recently added.

Sponsored by:	EMC / Isilon Storage Division
2015-10-14 20:30:32 +00:00
Bryan Drewery
e1adfb9a5a Re-indent the ALL_SUBDIR_TARGETS list 2015-10-14 20:28:15 +00:00
Bryan Drewery
b9b3f1e65a Revert r289282 for now as the interaction with a directory containing
bsd.files.mk and bsd.subdir.mk is recursing too many times.
2015-10-14 19:30:04 +00:00
Ed Maste
5ea4b6d5f6 /libexec subdirs are part of the base system (for *.debug files)
Sponsored by:	The FreeBSD Foundation
2015-10-14 19:19:44 +00:00
Bryan Drewery
add9ac4043 Fix support for building a PROG_CXX, and PROG, directly.
For example in lib/atf/libatf-c++/tests/detail it is now possible to
run 'make application_test'.  This was intended to worked for PROGS,
but lacked support for PROGS_CXX.

Also fix redefining the main PROG target to recurse.  This isn't needed
since the main process is setting PROG/PROG_CXX to handle it directly
via bsd.prog.mk.

MFC after:	3 weeks
Sponsored by:	EMC / Isilon Storage Division
2015-10-14 05:50:16 +00:00
Bryan Drewery
3be00685d6 Follow-up r288218 by ensuring common objects are built before recursing.
Some example where this is a problem:
  lib/atf/libatf-c++/tests/Makefile:SRCS.${_T}=   ${_T}.cpp test_helpers.cpp
  lib/atf/libatf-c++/tests/detail/Makefile:SRCS.${_T}=    ${_T}.cpp test_helpers.cpp
  lib/atf/libatf-c/tests/Makefile:SRCS.${_T}=     ${_T}.c test_helpers.c
  lib/atf/libatf-c/tests/detail/Makefile:SRCS.${_T}=      ${_T}.c test_helpers.c
  lib/libpam/libpam/tests/Makefile:SRCS.${test} = ${test}.c ${COMMONSRC}

A similar change may be needed for FILES, SCRIPTS, or INCS, but for now stay
with just SRCS.

Reported by:	rodrigc
MFC after:	3 weeks
X-MFC-With:	r288218
Sponsored by:	EMC / Isilon Storage Division
2015-10-14 04:42:05 +00:00
Bryan Drewery
7b87723296 Replace the out-of-place includes/files/config handling in bsd.subdir.mk with
more typical ALL_SUBDIR_TARGETS entries and target hooks in bsd.incs.mk,
bsd.files.mk and bsd.confs.mk.

This allows the targets to be NOPs if unneeded and still work with the
shortcut 'make includes' to build and then install in a parallel-safe manner.

Sort and re-indent the ALL_SUBDIR_TARGETS with the new entries.

Sponsored by:	EMC / Isilon Storage Division
2015-10-14 02:37:30 +00:00
Bryan Drewery
16927922a5 Correct a comment in bsd.incs.mk forgotten in r274662 and copied into bsd.confs.mk.
The bsd.confs.mk may be wrong but for now fix it.

Sponsored by:	EMC / Isilon Storage Division
2015-10-14 00:43:29 +00:00
Bryan Drewery
1b828f96d8 Add a note about the mysterious files/includes/config block.
This originated from r96668.
2015-10-14 00:36:33 +00:00
Bryan Drewery
0b2580be07 bsd.subdir.mk: Move all of the targets into ALL_SUBDIR_TARGETS.
Also improve documentation.

The SUBDIR_TARGETS variable should really be named LOCAL_SUBDIR_TARGETS, but
renaming it may be a surprise for downstream vendors who use this variable.

MFC after:	2 weeks
Sponsored by:	EMC / Isilon Storage Division
2015-10-13 19:42:57 +00:00
Bryan Drewery
85564c69ca bsd.subdir.mk: Handle cleanobj.
Before this, the target was unknown.  Now it will recurse on subdirs and run
the target in the current directory.  It is required to recurse as there
may be subdirs that have objs in their directory or in the object directory,
so it is not enough to just delete the objdir of the subdir parent.

MFC after:	2 weeks
Sponsored by:	EMC / Isilon Storage Division
2015-10-13 19:11:22 +00:00
Enji Cooper
da3110ed56 Default TESTSDIR to /usr/tests/${RELDIR:H}
When run from bin/ls/tests, for example, the value of TESTSDIR would be
${TESTSBASE}/${RELDIR:H} -> /usr/tests/bin/ls/tests/.. ->
/usr/tests/bin/ls

Document the new behavior in bsd.README.

While here, also document TESTSBASE

Relnotes: yes
Differential Revision: D1022
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
2015-10-11 21:29:24 +00:00
Enji Cooper
dfff069863 Simplify netbsd-tests.test.mk
- projects/bmake and subsequent commits provide SRCTOP; there's no need to
  manually specify it now.
- Compute a sane default for OBJTOP based on .OBJDIR and RELDIR. Manually
  specifying this is probably no longer needed, but it persists just in case
  (supporting commits will need to be made to move it out of some of the meta
  .mk files).
- Compute a sane default for TESTSRC. Error out if the path cannot be found.

MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
2015-10-11 20:02:10 +00:00
Baptiste Daroussin
f98f68bd50 Use existing CONFDIR instead of adding a new CONFIGDIR
Reported by:	jbeich
2015-10-11 19:25:39 +00:00