Commit Graph

151 Commits

Author SHA1 Message Date
sjg
8425f75b1c Use OBJS_SRCS_FILTER to control setting OBJS from SRCS
Some makefiles do reachover builds.
In some cases it is convenient to list subdirs of the distribution
in SRCS.

It is not very convenient, or always even desirable to have corresponding
subdirs in .OBJDIR, so OBJS_SRCS_FILTER allows the makefile to choose.
The default value 'R' matches existing practice.

But a makefile can set OBJS_SRCS_FILTER= T (the R gets added by
bsd.init.mk) to avoid the need for subdirs in .OBJDIR

Differential Revision:	https://reviews.freebsd.org/D12218
Reviewed by:	bdrewery
2017-09-16 05:42:27 +00:00
bdrewery
17e3f83790 Allow -DNO_SKIP_DEPEND to override the _SKIP_DEPEND logic.
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-07-25 16:46:12 +00:00
bdrewery
e007570379 Allow disabling dependency tracking if DEPEND_CFLAGS is empty.
This falls back on using the guesssed dependencies if so.
Also remove a pre-bmake check while here.

MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-07-24 23:32:40 +00:00
bdrewery
4497a71c5a Slightly simplify logic for which depend file is expected.
This is a NOP.

MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-07-24 23:32:36 +00:00
bdrewery
63cec4ec0f Rework logic for skipping .depend/.meta file read/stat/writes.
- Rename _SKIP_READ_DEPEND to _SKIP_DEPEND since it also avoids writing.
- This now uses .NOMETA to avoid reading any .meta files related to
  DEPENDOBJS.  Objects not in OBJS/DEPENDOBJS may still have their .meta
  files read in if they are in the dependency graph.
- This also avoids statting .meta and .depend files in the META_MODE +
  -DNO_FILEMON case.

MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-06-22 05:34:41 +00:00
bdrewery
6eab5891b5 Remove logic for setting .MAKE.DEPENDFILE=/dev/null already covered by _SKIP_BUILD.
_SKIP_BUILD is defined in bsd.init.mk.

MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-06-21 20:10:58 +00:00
bdrewery
57488d2ac8 Fix 'make clean all' to work again.
This likely broke completely with r308599.

Apply the same fix for 'make destroy' which is a DIRDEPS_BUILD thing.

PR:		219819
Reported by:	trasz
MFC after:	3 days
Sponsored by:	Dell EMC Isilon
2017-06-20 20:52:06 +00:00
bdrewery
810baa9812 META_MODE: Fix not using .depend files when no OBJDIR is present.
By default bmake does not allow meta mode to work unless an OBJDIR is
present.  It allows this if curdirok= is set with a value not starting
with [0NnFf], but usually it is "yes".

MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-03-17 18:08:36 +00:00
bdrewery
a72457c14f DIRDEPS_BUILD: Consolidate some logic for skipping of reading Makefile.depend.
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2016-11-13 00:11:05 +00:00
bdrewery
1ef06821e2 Consolidate the "don't build" optimizations into _SKIP_BUILD.
_SKIP_BUILD will be set when nothing is expected to be built.  This can
be used to optimize some tree-walks and operations which don't need to
load dependency files or generate dependencies via beforebuild-style
hacks.

MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2016-11-13 00:11:02 +00:00
marcel
17592f2cf9 When MAKEOBJDIRPREFIX points to a case-insensitive file system, the
build can break when different source files create the same target
files (case-insensitivity speaking).  This is the case for object
files compiled with -fpic and shared libraries. The former uses
an extension of ".So", and the latter an extension ".so".  Rename
shared object files from *.So to *.pico to match what NetBSD does.

See also r305855

MFC after:	1 month
Sponsored by:	Bracket Computing
Differential Revision:	https://reviews.freebsd.org/D7906
2016-09-24 15:11:27 +00:00
bdrewery
5ac77e13d9 FAST_DEPEND/META_MODE: Fix several issues with SRCS containing '/'.
- For FAST_DEPEND, properly apply the -M flags when compiling by
  enduring that the condition also has the s,/,_,g filter applied to it
  first.
- For FAST_DEPEND, properly read from the filtered filename.
- For META_MODE, it needs to read from a full-pathed and s,/,_,g replaced
  filename based on bmake's meta_name() function which names the .meta
  file for SRCS with '/' in them.

This support has not been extended to the kernel build yet but may be in the
future.

MFC after:	2 weeks
Reported by:	dim
Sponsored by:	EMC / Isilon Storage Division
2016-09-01 17:36:52 +00:00
bdrewery
b526555a34 Add a 'make print-dir' that simply traverses all directories and prints them.
This is useful for finding connected directories.

MFC after:	1 week
Sponsored by:	EMC / Isilon Storage Division
2016-08-31 19:30:52 +00:00
bdrewery
d5d515e9ff Allow parallelizing clean/cleandepend/depend with all.
This now makes 'make -jX clean all install' safe.

MFC after:	3 days
Sponsored by:	EMC / Isilon Storage Division
2016-08-29 18:31:34 +00:00
bdrewery
7782c2dc96 FAST_DEPEND: Fix 'make all install' not properly rebuilding based on .depend.* files.
An optimization is in place to skip reading the .depend.* files with
'make install'.  This was too strong and broke 'make all install' and
'make foo.o foo install'.  Now only skip reading the dependency files
if all make targets ran are install targets.

The problem comes about because headers are only added in as a guessed
dependency if .depend.* files do not yet exist.  If they do exist, even
if being skipped from being read, then the header dependencies are not
applied.  This applies to all #included files, and not just headers.

Reported by:	kib
MFC after:	1 day
Sponsored by:	EMC / Isilon Storage Division
2016-08-23 19:37:18 +00:00
bdrewery
8f9b4b00a2 META_MODE: Don't generate or read _EXTRADEPEND dependencies when using filemon.
The DPADD data in .depend will be redundant with what is in the .meta file.

Also extend NO_EXTRADEPEND support to bsd.prog.mk.

Approved by:	re (blanket, META_MODE)
Sponsored by:	EMC / Isilon Storage Division
2016-06-21 21:55:03 +00:00
bdrewery
e36cb3f2ce WITH_META_MODE: Do include headers for specific guessed dependencies
This is a follow-up to r300343.

This is important for the OBJS_DEPEND_GUESS usage in
gnu/usr.bin/cc/cc_tools.

See comments for more details.

Approved by:	re (implicit)
Sponsored by:	EMC / Isilon Storage Division
2016-06-15 23:57:50 +00:00
bdrewery
a9f6ba59c0 Don't truncate OBJS_DEPEND_GUESS.target from Makefile
This is important to allow a Makefile to override OBJS_DEPEND_GUESS for
handling in META_MODE when its depend files are missing.

Approved by:	re (implicit)
Sponsored by:	EMC / Isilon Storage Division
2016-06-15 23:57:32 +00:00
bdrewery
7e8741e191 WITH_META_MODE: Avoid "building" .depend if there is nothing to do.
This avoids 'Building /path/.depend' when it will not actually produce a
file.

Sponsored by:	EMC / Isilon Storage Division
2016-06-03 21:35:19 +00:00
bdrewery
f12a82914c WITH_META_MODE: Don't expect meta files for side-effect generated files.
The first file in these lists will generate everything else so only
it should be getting a .meta file.  With bmake's missing=yes meta
feature these would otherwise cause a rebuild without the
.NOMETA hint.

Sponsored by:	EMC / Isilon Storage Division
2016-06-03 19:25:41 +00:00
bdrewery
cb5c2c054b Revert r301079.
This breaks cross-building with WITH_META_MODE since it will rebuild
'build-tools' during the 'everything' phase.

A more proper fix is coming to bmake to implicitly require .META unless
.NOMETA (and other restrictions) are in place.
2016-06-03 19:25:36 +00:00
bdrewery
0dcfafcc05 WITH_META_MODE: Mitigate switching from without to with META_MODE.
Adding .META to targets-to-build will ensure that they will rebuild if there
is no .meta file.

Adding it to all SUFFIXES and objects ensures that at least objects will
rebuild if there is no .meta file.

This will be reverted if bmake's behavior changes to rebuild on missing .meta
files.

Sponsored by:	EMC / Isilon Storage Division
2016-05-31 21:22:19 +00:00
bdrewery
e3900077b6 WITH_META_MODE: Fix suffix transformation rules with guessed dependencies.
This is the same problem as r290629.  With META_MODE we do not generate
.depend files, so there is no proper dependency to lookup.  Guessed
dependencies must be used.  If this proves to be a problem then we will
have to generate and use .depend files even with META_MODE.

Sponsored by:	EMC / Isilon Storage Division
2016-05-21 01:31:48 +00:00
bdrewery
77ec731d14 META_MODE: Don't hide the .depend rm -f command.
Otherwise the build command changes every build.  META_MODE will
only remove it if something changes to warrant rebuilding
the file.

Sponsored by:	EMC / Isilon Storage Division
2016-04-18 21:11:55 +00:00
bdrewery
3842fa6c22 META_MODE: Clean .meta files with 'make cleandepend'.
Sponsored by:	EMC / Isilon Storage Division
2016-04-18 18:12:12 +00:00
bdrewery
c027e2497f Implement the dependency condition more safely.
Nested : are not handled well without "".

Sponsored by:	EMC / Isilon Storage Division
2016-04-14 21:04:49 +00:00
bdrewery
59f4cb814d Remove the old depend (mkdep) code and make FAST_DEPEND the one true way.
Reviewed by:	emaste, hselasky (partial), brooks (brief)
Discussed on:	arch@
Sponsored by:	EMC / Isilon Storage Division
Differential Revision:	https://reviews.freebsd.org/D5742
2016-03-30 23:50:23 +00:00
bdrewery
f0db244394 FAST_DEPEND: Use .dinclude to enable full .depend logic in bmake.
The inclusion of .MAKE.DEPENDFILE (.depend) has special logic in make
to ignore stale/missing dependencies.  bmake 20160220 added a '.dinclude'
directive that uses the special logic for .depend when including the file.

This fixes a build error when a file is moved or deleted that exists in a
.depend.OBJ file.  This happened in r292782 when sha512c.c "moved" and an
incremental build of lib/libmd would fail with:
  make: don't know how to make /usr/src/lib/libcrypt/../libmd/sha512c.c. Stop

Now this will just be seen as a stale dependency and cause a rebuild:
  make: /usr/obj/usr/src/lib/libmd/.depend.sha512c.o, 13: ignoring stale .depend for /usr/src/lib/libcrypt/../libmd/sha512c.c
  --- sha512c.o ---
  ...
This rebuild will only be done once since the .depend.sha512c.o will
be updated on the build with the -MF flags.

This also removes -MP being passed for the .depend.OBJ generation (which
would create fake targets for system headers) since the logic is no
longer needed to protect from missing files.

Sponsored by:	EMC / Isilon Storage Division
2016-03-11 04:09:56 +00:00
bdrewery
32f8969e48 FAST_DEPEND: Prefer .OBJDIR depend files.
Sponsored by:	EMC / Isilon Storage Division
2016-02-29 21:10:47 +00:00
bdrewery
9f2509a6de Import bsd.clang-analyze.mk based on NetBSD's version.
This allows 'make analyze' or 'make OBJ.clang-analyzer' to run the
Clang static analyzer and present results on stdout.

Obtained from:	NetBSD (CVS Rev. 1.3)
Sponsored by:	EMC / Isilon Storage Division
Differential Revision:	https://reviews.freebsd.org/D5449
2016-02-26 22:14:00 +00:00
bdrewery
91180d20bc Let CLEANDEPENDFILES actually be set before bsd.dep.mk.
This fixes 'cleandepend' for kmods after r295983.

Pointyhat to:	bdrewery
Sponsored by:	EMC / Isilon Storage Division
2016-02-26 01:52:33 +00:00
bdrewery
a9202a0906 Fix build without FAST_DEPEND and without running 'make depend' after r295985.
The OBJS_DEPEND_GUESS mechanism required moving the bsd.dep.mk inclusion
to after the checks, but left DEPENDFILE not-yet-set.  Move it to
bsd.own.mk to resolve this.

Pointyhat to:	bdrewery
Reported by:	antoine (ports failures)
Sponsored by:	EMC / Isilon Storage Division
2016-02-25 20:02:42 +00:00
bdrewery
a6a90cbd3a FAST_DEPEND: Still need to remove DEPENDFILE when rebuilding.
This avoids _EXTRADEPEND adding duplicate entries.

Sponsored by:	EMC / Isilon Storage Division
2016-02-24 20:28:45 +00:00
bdrewery
8501e83329 FAST_DEPEND: Always run depend via beforebuild which removes many hacks.
This will generate dependencies rather than depending on the previous behavior
of depending on the guessed OBJS: *.h dependecies or a user running
'make depend'.

Experimentation showed that depending only on headers was not enough and
prone to .ORDER errors.  Downstream users may also have added
dependencies into beforedepend or afterdepend targets.  The safest way to
ensure dependencies are generated before build is to run 'make depend'
beforehand rather than just depending on DPSRCS+SRCS.

Note that the OBJS_DEPEND_GUESS mechanism (a.k.a .if !exists(.depend) then
foo.o: *.h) is still useful as it improves incremental builds with missing
.depend.* files and allows 'make foo.o' to usually work, while this
'beforebuild: depend' ensures that the build will always find all dependencies.
The 'make foo.o' case has no means of a 'beforebuild' hook.

This also removes several hacks in the DIRDEPS_BUILD:
- NO_INSTALL_INCLUDES is no longer needed as it mostly was to work around
  .ORDER problems with building the needed headers early.
- DIRDEPS_BUILD: It is no longer necesarry to track "local dependencies" in
  Makefile.depend.

  These were only in Makefile.depend for 'clean builds' since nothing would
  generate the files due to skipping 'make depend' and early dependency
  bugs that have been fixed, such as adding headers into SRCS for the
  OBJS_DEPEND_GUESS mechanism.  Normally if a .depend file does not exist then
  a dependency is added by bsd.lib.mk/bsd.prog.mk from OBJS: *.h.  However,
  meta.autodep.mk creates a .depend file from created meta files and inserts
  that into Makefile.depend.  It also only tracks *.[ch] files though which can
  miss some dependencies that are hooked into 'make depend'.  This .depend
  that is created then breaks incremental builds due to the !exists(.depend)
  checks for OBJS_DEPEND_GUESS.  The goal was to skip 'make depend' yet it only
  really works the first time.  After that files are not generated as expected,
  which r288966 tried to address but was using buildfiles: rather than
  beforebuild: and was reverted in r291725.  As noted previously,
  depending only on headers in beforebuild: would create .ORDER errors
  in some cases.

  meta.autodep.mk is still used to generate Makefile.depend though via:
    gendirdeps: Makefile.depend
    .END: gendirdeps

  This commit allows removing all of the "local dependencies" in
  Makefile.depend which cuts down on churn and removes some of the
  arch-dependent Makefile.depend files.

  The "local dependencies" were also problematic for bootstrapping.

Sponsored by:	EMC / Isilon Storage Division
2016-02-24 17:19:18 +00:00
bdrewery
5c7565c503 Hook the meta/nofilemon build into using FAST_DEPEND.
FAST_DEPEND is intended to be the "skip 'make depend' and mkdep"
feature.  Since DIRDEPS_BUILD does this already with some of its own
hacks, and filemon doesn't need this, and nofilemon does, teach it how
to handle each of these cases.

In meta+filemon mode filemon will handle dependencies itself via the
meta mode logic in bmake.  We still want to set MK_FAST_DEPEND=yes to
enable some logic that indicates that 'make depend' is skipped in the
traditional sense.  The actual .depend.* files will be skipped.

When nofilemon is set though we still need to track and generate dependencies.

Sponsored by:	EMC / Isilon Storage Division
2016-02-24 17:19:13 +00:00
bdrewery
7171d0bbd8 FAST_DEPEND: Don't waste time generating an empty .depend file.
The .depend file will still be generated if _EXTRADEPEND is used.  The target
is kept with a dependency on DPSRCS though so that 'make depend' will generate
all files.

Sponsored by:	EMC / Isilon Storage Division
2016-02-24 17:19:09 +00:00
bdrewery
99c769b1d7 FAST_DEPEND: Rework how guessed dependencies are handled.
Rather than depend on .depend not existing, check the actual
.depend.OBJ file that will be used for that object.  If it doesn't
exist then use the guessed dependencies.

FAST_DEPEND may never have a .depend file.  Not having one means all of the
previous logic would over-depend all object files on all headers which is not
what we wanted.  It also means that if a .depend is generated before a build
is done for _EXTRADEPEND (such as for PROG or LIB) then all of these
dependencies would not be used since the .depend wasn't generated from mkdep
and the real .depend.* files are not generated until the build.

Sponsored by:	EMC / Isilon Storage Division
2016-02-24 17:19:05 +00:00
bdrewery
3f10674738 Follow-up r295667 with fixes for SRCS defined.
cleandepend should always remove CLEANDEPEND* if they are not empty,
but bsd.dep.mk should not add the tags entries unless SRCS is defined
as it did before.  The .depend file itself it still always removed
to avoid accidentally keeping a stale one around as done in r295666.

Sponsored by:	EMC / Isilon Storage Division
2016-02-24 17:18:55 +00:00
bdrewery
b4d8043355 Avoid reading .depend.* in simple cases where not needed.
This will speed up some tree-walks with FAST_DEPEND which otherwise
would include length(SRCS) .depend files.

This also uses a trick suggested by sjg@ to still read them in when
specifying _V_READ_DEPEND=1 in the env/make args.

Sponsored by:	EMC / Isilon Storage Division
2016-02-19 00:41:24 +00:00
bdrewery
cdcd6aa074 Add CLEANDEPEND[FILES|DIRS] and simplify the tags cleanup to use it.
Sponsored by:	EMC / Isilon Storage Division
2016-02-16 19:11:17 +00:00
bdrewery
69888e41a1 Always remove .depend.* regardless of WITH_FAST_DEPEND in case of changing
which is used.

Sponsored by:	 EMC / Isilon Storage Division
2016-02-16 19:02:04 +00:00
bdrewery
b76e69080a Remove temporary hack from r294370 for SSH upgrades.
MFC after:	1 week
Sponsored by:	EMC / Isilon Storage Division
2016-02-16 19:00:22 +00:00
bdrewery
d04bd9e924 Fix -include .depend hack from r294370 for headers not in .PATH.
This hack will be removed in a few weeks.  It is here to fix incremental
builds of SSH between r291941 and r294370.

Reported by:	jmallett
MFC after:	1 day
Sponsored by:	EMC / Isilon Storage Division
2016-01-28 18:57:47 +00:00
bdrewery
bda9fd0b43 Revert yacc dependency back to pre-r241298.
Several attempts to fix this logic was done after r241298, which were
all reverted, yet this change was not.

The .h file does not depend on the .c file, so do not impose such a
dependency on it.  They are generated by the same command but do not
depend on each other.  Restore the .ORDER which should handle parallel build
issues.  This fixes an actual bug where the .h file is not recreated
when missing [1].  For example:
  cd lib/libc
  make cleanobj
  make nsparser.h
  rm nsparser.h
  make nsparser.h # will not rebuild nsparser.h

I have been trying to track down a build problem where nsparser.h is
missing when nslexer.o is built.  It is possible this is related.

Reported by:	bde [1]
		https://lists.freebsd.org/pipermail/svn-src-all/2012-October/059481.html
		https://lists.freebsd.org/pipermail/svn-src-all/2012-October/060038.html
MFC after:	3 weeks
Sponsored by:	EMC / Isilon Storage Division
2016-01-27 01:33:26 +00:00
bdrewery
64d0bd2951 Fix DIRDEPS_BUILD after r294752.
DIRDEPS_BUILD does not yet support PROGS having their own dependency
file.

Overriding .MAKE.DEPENDFILE here causes major problems with the meta
mode logic since it creates the Makefile.depend as '.depend' resulting
in infinite loops in make due to dirdeps.mk including .depend endlessly.

X-MFC-With:	r294752
MFC after:	1 week
Sponsored by:	EMC / Isilon Storage Division
2016-01-27 01:24:05 +00:00
bdrewery
78a86ea4e5 Fix PROGS not reading .depend files after r284288 by making DEPENDFILE work.
We have had this user-modifable DEPENDFILE variable forever that does nothing
relevant for the user since fmake always used '.depend'.  Bmake
introduced the .MAKE.DEPENDFILE variable that can be modified to change
the name of '.depend'.

Prior to r284288, bsd.progs.mk was setting .MAKE.DEPENDFILE to allow
working incremental builds.  This was modified most likely to not
conflict with the META MODE handling of .MAKE.DEPENDFILE as it has a lot
more special logic for that variable.

MFC after:	1 week
Sponsored by:	EMC / Isilon Storage Division
2016-01-25 22:29:44 +00:00
bdrewery
e15e109c7d Remove unneeded beforebuild hacks for dtrace probes.
This still allows skipping 'make depend'.  Headers in SRCS generate
properly.

MFC after:	1 week
Sponsored by:	EMC / Isilon Storage Division
2016-01-25 22:29:41 +00:00
bdrewery
be8db63a6d Fix incremental build of dtrace probes.
Currently dtrace(1) -Go does not properly rebuild the target if it
exists.  It results in missing symbols.

  dtrace -C -x nolibs -G -o usdt.o -s /root/git/freebsd/cddl/contrib/opensolaris/cmd/dtrace/test/tst/common/json/usdt.d tst.usdt.o
  dtrace: target object (usdt.o) already exists. Please remove the target
  dtrace: object and rebuild all the source objects if you wish to run the DTrace
  dtrace: linking process again
  cc -O2 -pipe -O0 -g -I/root/git/freebsd/cddl/usr.sbin/dtrace/tests/common/json -std=gnu99 -fstack-protector-strong -Qunused-arguments  -o tst.usdt.exe.full tst.usdt.o usdt.o
  tst.usdt.o: In function `main':
  /root/git/freebsd/cddl/contrib/opensolaris/cmd/dtrace/test/tst/common/json/tst.usdt.c:56: undefined reference to `__dtrace_bunyan_fake___log__debug'
  /root/git/freebsd/cddl/contrib/opensolaris/cmd/dtrace/test/tst/common/json/tst.usdt.c:60: undefined reference to `__dtrace_bunyan_fake___log__debug'
  cc: error: linker command failed with exit code 1 (use -v to see invocation)
  *** [tst.usdt.exe.full] Error code 1

This is a consequence of r212358.

MFC after:	1 week
Sponsored by:	EMC / Isilon Storage Division
2016-01-25 22:29:32 +00:00
bdrewery
6c8338f92c mkdep: Fix -include not being added for .depend tracking.
This fixes incremental build of OpenSSH after the recent upgrade.

For example, in secure/lib/libssh, -include ssh_namespace.h is used on
all files.  This is not tracked in the .depend file though due to
MKDEP_CFLAGS not including it.  The ssh example was broken in r291941
when not using FAST_DEPEND due to the .depend bug.  FAST_DEPEND was not
affected by this because it generates dependencies at compile time and
thus sees the -include.

This ugly make syntax could be simpler for bmake by using :tW but
fmake-compatible syntax is used since this needs to be MFC'd all the way
to stable/9.

Also add a temporary hack to workaround existing checkouts building
incrementally with a .depend file not having these headers.

MFC after:	1 week
Sponsored by:	EMC / Isilon Storage Division
2016-01-20 01:40:18 +00:00
bdrewery
9ee7889344 FAST_DEPEND: Still use if filemon is not used.
If filemon is used then there is no need to generate dependency files during
compilation as the .meta files will achieve the same result.

This is a temporary solution until FAST_DEPEND is default.  Once that is
default there will be an option to disable dependency generation entirely
as it is only useful if an incremental build is planned, thus META_MODE+filemon
can enable that option to short-circuit all FAST_DEPEND-related logic.

Sponsored by:	EMC / Isilon Storage Division
2016-01-19 22:41:58 +00:00