2690 Commits

Author SHA1 Message Date
bdrewery
7c9332316e META_MODE: Don't expect .meta files for side-effect generated files.
This is similar to r301285.

MFC after:	1 week
Sponsored by:	EMC / Isilon Storage Division
2016-09-01 23:20:54 +00:00
bdrewery
5ac77e13d9 FAST_DEPEND/META_MODE: Fix several issues with SRCS containing '/'.
- For FAST_DEPEND, properly apply the -M flags when compiling by
  enduring that the condition also has the s,/,_,g filter applied to it
  first.
- For FAST_DEPEND, properly read from the filtered filename.
- For META_MODE, it needs to read from a full-pathed and s,/,_,g replaced
  filename based on bmake's meta_name() function which names the .meta
  file for SRCS with '/' in them.

This support has not been extended to the kernel build yet but may be in the
future.

MFC after:	2 weeks
Reported by:	dim
Sponsored by:	EMC / Isilon Storage Division
2016-09-01 17:36:52 +00:00
bdrewery
b526555a34 Add a 'make print-dir' that simply traverses all directories and prints them.
This is useful for finding connected directories.

MFC after:	1 week
Sponsored by:	EMC / Isilon Storage Division
2016-08-31 19:30:52 +00:00
bdrewery
7ff669ea6a AUTO_OBJ: Support SRCS/DPSRCS that contain '/' and require a nested OBJDIR.
This is to be used by the new clang3.9 build and extends functionality
added to 'make obj' in r279980.

MFC after:	2 weeks
Sponsored by:	EMC / Isilon Storage Division
2016-08-30 22:48:05 +00:00
bdrewery
d5d515e9ff Allow parallelizing clean/cleandepend/depend with all.
This now makes 'make -jX clean all install' safe.

MFC after:	3 days
Sponsored by:	EMC / Isilon Storage Division
2016-08-29 18:31:34 +00:00
bapt
f14653e4cd Remove warning on struct-overflow on gcc 5.3.0 as zic(8) dies on it 2016-08-27 01:28:00 +00:00
bdrewery
33af5bbb36 WITH_META_MODE: Don't let subdir traversals for 'make install' re-enable meta mode.
MFC after:	2 weeks
Sponsored by:	EMC / Isilon Storage Division
2016-08-25 23:55:56 +00:00
kp
a5e10af8bc Add libifc, a library implementing core functionality that exists in ifconfig(8) today.
libifc (pronounced lib-ifconfig) aims to be a light abstraction layer between
programs and the kernel APIs for managing the network configuration.
This should hopefully make programs easier to maintain, and reduce code
duplication.

Work will begin on making ifconfig(8) use this library in the near future.

This code is still evolving. The interface should not be considered stable until
it is announced as such.

Submitted By: Marie Helene Kvello-Aune <marieheleneka@gmail.com>
Reviewed By: kp
Differential Revision: https://reviews.freebsd.org/D7529
2016-08-25 19:40:25 +00:00
bdrewery
7782c2dc96 FAST_DEPEND: Fix 'make all install' not properly rebuilding based on .depend.* files.
An optimization is in place to skip reading the .depend.* files with
'make install'.  This was too strong and broke 'make all install' and
'make foo.o foo install'.  Now only skip reading the dependency files
if all make targets ran are install targets.

The problem comes about because headers are only added in as a guessed
dependency if .depend.* files do not yet exist.  If they do exist, even
if being skipped from being read, then the header dependencies are not
applied.  This applies to all #included files, and not just headers.

Reported by:	kib
MFC after:	1 day
Sponsored by:	EMC / Isilon Storage Division
2016-08-23 19:37:18 +00:00
dim
cb0b718c02 In addition to creating subdirectories under .OBJDIR for SRCS with
relative paths, also create them for DPSRCS.  This is needed for builds
that generate files during the depend stage, which cannot be compiled by
themselves, since those have to be put in DPSRCS.
2016-08-23 19:31:43 +00:00
bdrewery
3fce81858a Re-enable WITH_SYSTEM_COMPILER with WITH_META_MODE.
This was disabled in r301468 due to -target/--sysroot sometimes being used in
the build and other times not being used.  Now that it is always used since
r304681, it is safe to combine the features.

MFC after:	3 days
Sponsored by:	EMC / Isilon Storage Division
2016-08-23 15:22:17 +00:00
bdrewery
adb3ac2dae For 'make <directory>' hook into the all_subdir_<directory> targets.
This fixes parallel build issues when trying to depend on ${SUBDIR}.  An
example of this in share/i18n/csmapper/Makefile where mapper.dir depends
on ${SUBDIR} having been traversed and built already.  Before this
change running make in that directory would build the subdirectories
twice.  This led to obscure build races.  While reworking that build
may be possible, the framework should not so easily allow creating such
problems.

Now depending on <directory> will properly redirect to the
all_subdir_<directory> target rather than invoking the inline shell.

This also makes 'make -jX <directory>' now respect any
SUBDIR_DEPEND_<directory> statements when SUBDIR_PARALLEL is defined.
This is not entirely intended and may be changed later.

MFC after:	2 weeks
Sponsored by:	EMC / Isilon Storage Division
2016-08-22 22:51:07 +00:00
bdrewery
b4aae46029 Always define the various <target>_subdir_<directory> targets, even if not used.
This is part of an effort to cleanup handling of some edge cases
involving 'make <directory>'.  It also provides the targets for
other uses.

MFC after:	2 weeks
Sponsored by:	EMC / Isilon Storage Division
2016-08-22 22:51:04 +00:00
bdrewery
46aba11fa0 Stop using _SUBDIR internally for non-SUBDIR_PARALLEL builds.
This is unifying more of the logic.  Rather than create targets such
as 'all: all_subdir_foo' when using SUBDIR_PARALLEL and using
'all: _SUBDIR' when not using SUBDIR_PARALLEL, always use the
expanded out <target>_subdir_<directory> pattern.  When not using
SUBDIR_PARALLEL, have each directory-target depend on the previously
defined targets as to respect the *order* of SUBDIR.

Using 'make -N' now prints all directory traversals individually rather
than using a loop, since a loop is no longer used to traverse.

This is part of an effort to cleanup handling of some edge cases
involving 'make <directory>' and making it simpler in the sense
that the pattern used to build is the same for all modes.

MFC after:	2 weeks
Sponsored by:	EMC / Isilon Storage Division
2016-08-22 22:51:01 +00:00
emaste
0d4d3262d1 Forcibly disable MK_TESTS if building without C++
Several atf components require C++, and the test suite is not usable
if building WITHOUT_CXX.

Reviewed by:	bdrewery, jmmv
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D7597
2016-08-22 17:45:30 +00:00
bdrewery
cbfb8e93eb PROGS: Support INTERNALPROG.prog=yes to not install it.
MFC after:	3 days
Sponsored by:	EMC / Isilon Storage Division
2016-08-11 17:06:48 +00:00
br
f555afc5a1 Revert r303911 "Remove extra -msoft-float flags settings."
This was not properly tested.
2016-08-11 13:42:31 +00:00
br
5d5018f88a Remove extra -msoft-float flags settings.
This helps to build firmware modules.

Sponsored by:	DARPA, AFRL
Sponsored by:	HEIF5
2016-08-10 13:32:27 +00:00
markj
3ded4ca51d libproc: Add proc_getmodel().
This is used by libdtrace to determine the data model of target processes.
This allows for the creation of pid provider probes in 32-bit processes on
amd64.

MFC after:	1 month
2016-07-30 03:09:23 +00:00
markj
da3555a6b6 librtld_db: Use the auxv to figure out where to look up loader symbols.
Previously, librtld_db just hardcoded /libexec/ld-elf.so, which isn't
correct for processes that aren't using the native ABI. With this change,
librtld_db can be used to inspect non-native processes; in particular,
dtrace -c now works for 32-bit executables on amd64.

MFC after:	1 month
2016-07-30 03:05:23 +00:00
br
8d8cea7deb Normalise the CWARNFLAGS inter-word spacing: remove all leading
and trailing space, and convert multiple consecutive spaces to
single space.

This helps to keep build output looking good.
2016-07-28 17:18:02 +00:00
emaste
d20793840b Enable LLVM libunwind by default on amd64 and i386
It is a maintained and updated runtime exception stack unwinder that
should be a drop-in replacement.

It can be disabled by setting WITHOUT_LLVM_LIBUNWIND in src.conf.

PR:		206039 [exp-run]
Sponsored by:	The FreeBSD Foundation
2016-07-27 16:01:44 +00:00
emaste
2c03309f28 Remove ${OBJDUMP} as it is not used by the base system
It was added to sys.mk relatively recently (r274503) for EFI builds
but is no longer used by the base system. The in-tree binutils are
outdated, will not be updated, and will be removed in the future.
Remove it from the toolchain build now to slightly simplify the build
and make sure we don't grow an accidental dependency.

Note that this affects only the toolchain build, and does not affect
/usr/bin/objdump in the built world.

Reviewed by:	bdrewery
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D6460
2016-07-27 14:58:15 +00:00
br
899e79ab1c Fix style. 2016-07-24 18:04:12 +00:00
br
f96b095193 Set the soft-float flag for assembly code as well.
This fixes compilation with GCC 6.1.

Sponsored by:	DARPA, AFRL
2016-07-22 15:22:49 +00:00
br
702f33ca74 Add warn flags for GCC 6.1 compiler.
Sponsored by:	DARPA, AFRL
2016-07-22 15:00:38 +00:00
bdrewery
c9ee3263a5 Revert r302670 and r302671 for now.
MACHINE_CPUARCH smells like MACHINE except for arm64/aarch64 which
has it backwards.
2016-07-13 01:35:53 +00:00
bdrewery
b97599e1f2 Create one list of replacements for MACHINE_CPUARCH as MACHINE_CPUARCH_SUB.
This also adds missing s/aarch64/arm64 to the sys.mk version and also
adds back armv6hf for universe since it was added to the sys.mk version
in r300438.

MFC after:	3 days
Sponsored by:	EMC / Isilon Storage Division
Differential Revision:	https://reviews.freebsd.org/D7159
2016-07-12 18:57:25 +00:00
bdrewery
5c68448b2f Don't warn about needing filemon for makeman.
r287879 wasn't quite enough here since Makefile.inc1 is running
'make -C release -V VERSION' and hitting the warning.

Reported by:	lattera
Sponsored by:	EMC / Isilon Storage Division
Approved by:	re (blanket, META_MODE)
2016-06-29 16:45:01 +00:00
bdrewery
81ddc3b19a WITH_SYSTEM_COMPILER: Enable by default
This improves buildworld, toolchain, kernel-toolchain, and universe targets.
See r300354 or src.conf(5) for more details.

Approved by:	re (gjb)
Relnotes:	yes (r300354)
Sponsored by:	EMC / Isilon Storage Division
2016-06-24 19:55:59 +00:00
emaste
dbe14047dd bsd.prog.mk: add "/usr/lib" to list of base system directories
kgzldr.o is installed into /usr/lib but using bsd.prog.mk. Add
/usr/lib to the base system directory list so that debug files are
installed into /usr/lib/debug/usr/lib, not /usr/lib/.debug .

Approved by:	re (gjb)
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2016-06-24 18:45:16 +00:00
bdrewery
e6811f9793 Add AFLAGS.IMPSRC and document A[C]FLAGS
Approved by:	re (gjb)
Sponsored by:	EMC / Isilon Storage Division
2016-06-22 20:31:49 +00:00
bdrewery
8f9b4b00a2 META_MODE: Don't generate or read _EXTRADEPEND dependencies when using filemon.
The DPADD data in .depend will be redundant with what is in the .meta file.

Also extend NO_EXTRADEPEND support to bsd.prog.mk.

Approved by:	re (blanket, META_MODE)
Sponsored by:	EMC / Isilon Storage Division
2016-06-21 21:55:03 +00:00
bdrewery
c0c29c5585 Mark targets with _SUBDIR as .PHONY.
This is mostly fixing META_MODE with realinstall wanting a .meta
file when it does not need one.  These targets really should
always run though since they have _SUBDIR on them.

Approved by:	re (implicit)
Sponsored by:	EMC / Isilon Storage Division
2016-06-15 23:58:09 +00:00
bdrewery
42d9d45fb8 Fix native powerpc64 build of lib32 with in-tree GCC.
- This was broken by r300350 and r300885.
- Add some comments around the external GCC logic since it is spread out
  and in need of some cleanup.
- The problem was that X_COMPILER_TYPE is always defined from CC->XCC's
  default, so if /usr/bin/cc is GCC (as it is on native powerpc64) then
  X_COMPILER_TYPE was getting GCC and triggering the external logic in
  Makefile.libcompat.  It was intended to always provide -isystem with
  GCC since --sysroot is used into the lib32 sysroot which won't modify
  the header path without the -isystem.  The use of the libc++/std=c++11
  override was only intended to be used for external compilers though
  (more accurately GCC 4.8+ but that's a separate assumption to
  cleanup).  Apply the same logic from Makefile.inc1 to Makefile.libcompat
  to only add the libc++ override when needed for external compilers.

Pointyhat to:	bdrewery
Tested with:	native ppc64 (swills), universe, ppc64 xtoolchain,
		amd64 xtoolchain, sparc64 cross-build of ppc64 (host GCC 4.2)
Reported by:	andreast, swills
Approved by:	re (gjb)
Sponsored by:	EMC / Isilon Storage Division
2016-06-15 23:58:03 +00:00
bdrewery
9b42513fd1 WITH_META_MODE: Keep .MAKE.MODE/META_MODE clean
Due to META_MODE being passed into the environment it tends
to keep growing with the defaults.

Approved by:	re (implicit)
Sponsored by:	EMC / Isilon Storage Division
2016-06-15 23:57:57 +00:00
bdrewery
e36cb3f2ce WITH_META_MODE: Do include headers for specific guessed dependencies
This is a follow-up to r300343.

This is important for the OBJS_DEPEND_GUESS usage in
gnu/usr.bin/cc/cc_tools.

See comments for more details.

Approved by:	re (implicit)
Sponsored by:	EMC / Isilon Storage Division
2016-06-15 23:57:50 +00:00
bdrewery
a9f6ba59c0 Don't truncate OBJS_DEPEND_GUESS.target from Makefile
This is important to allow a Makefile to override OBJS_DEPEND_GUESS for
handling in META_MODE when its depend files are missing.

Approved by:	re (implicit)
Sponsored by:	EMC / Isilon Storage Division
2016-06-15 23:57:32 +00:00
bdrewery
107ea631d3 WITH_META_MODE: Enable printing of some of make's environment on error.
This will print a set of variables from make on error using
MAKE_PRINT_VAR_ON_ERROR.  It is already enabled for the DIRDEPS_BUILD.
It may make sense to enable this in the non-meta mode as well once
people are more used to its more verbose error output.

This makes it much simpler to see which .meta file is used when a
command files so that it may be inspected for the build command.

Suggested by:	sjg
Approved by:	re (implicit)
Sponsored by:	EMC / Isilon Storage Division
2016-06-14 16:20:25 +00:00
bdrewery
2e92e3aeda WITH_META_MODE: Lessen the filemon(4) requirement scope.
- Move the sys.mk filemon requirement to bsd.init.mk as a warning.
  This is intended only to show when building directly in a subdirectory
  without filemon loaded.
- Move the error into Makefile and only apply it when building
  from the META_TGT_WHITELIST target list.

-DNO_FILEMON can be used to suppress both the warning and the error but
makes WITH_META_MODE less useful.  It will only compare build commands
in this mode rather than track all dependencies.

This fixes installing from a jail which doesn't need filemon in this
phase [1].

Reported by:	Nikolai Lifanov <lifanov@mail.lifanov.com> [1]
Approved by:	re (implicit)
Sponsored by:	EMC / Isilon Storage Division
2016-06-14 16:20:19 +00:00
bdrewery
8c2481d2a9 WITH_META_MODE: Set MK_META_MODE=no with -B.
Using -B already sets .MAKE.MODE=compat but it was leaving
MK_META_MODE set which could still cause other MK_META_MODE==yes
checks to trigger.

Approved by:	re (implicit)
Sponsored by:	EMC / Isilon Storage Division
2016-06-14 16:20:11 +00:00
bdrewery
abaeb3d946 WITH_META_MODE: Fix rescue rebuilding build-tools.
This is the same issue as r297997.

Approved by:	re (implicit)
Sponsored by:	EMC / Isilon Storage Division
2016-06-14 16:19:54 +00:00
bdrewery
7d00fcc3d0 WITH_META_MODE+WITH_DEBUG_FILES: Fix library symlinks causing bogus rebuilds.
A simplified example of the library targets with WITH_DEBUG_FILES is:

  libgeom.so.5: libgeom.so.5.full
     cp libgeom.so.5.full libgeom.so.5

  libgeom.so.5.full:
     ln -s libgeom.so.5 libgeom.so
     cc -o libgeom.so.5.full *.o

Before, or without, WITH_DEBUG_FILES it is:

  libgeom.so.5:
     ln -s libgeom.so.5 libgeom.so
     cc -o libgeom.so.5 *.o

The problem is that bmake considers the link source for the libgeom.so
link in the libgeom.so.5.full target as being a dependency for
libgeom.so.5.full.  That resolves to libgeom.so.5.  Thus a cyclic
dependency is created.  The result of this is that if libgeom.so.5 is
created with a newer timestamp than libgeom.so.5.full, then
libgeom.so.5.full will be rebuilt on the next build.  This causes a
chain reaction of everything in the build relinking, or hitting the
problem itself.

Moving the link creation to the target that actually creates
libgeom.so.5 fixes the problem.  The simplest fix here is to just
duplicate the logic.

Submitted by:	sjg
Approved by:	re (implicit)
2016-06-14 16:19:44 +00:00
bdrewery
5f4cfb164d Don't force filemon for makeman/showconfig
Reported by:	lidl
2016-06-08 14:02:21 +00:00
bdrewery
f6ed7b9188 WITH_SYSTEM_COMPILER: Disable with WITH_META_MODE for now.
The WITH_META_MODE build is intended to be a working incremental build.
It spies on the build command to see if things should be rebuilt if the
command changes.  If you run buildworld, it builds a cross-compiler,
then do installworld and buildworld again it will invoke the
WITH_SYSTEM_COMPILER logic.  This then adds on -target/--sysroot, etc,
and causes rebuilds due to the changed build command even though the
compiler used is technically the same revision.  Since the incremental
build is not cleaning anything by default then there is much
less risk to rebuilding the already-existing cross-compiler.  Just
disable the combined logic and always use and build the cross-compiler.

An alternative to this would be to always pass -target/--sysroot.  Doing
so may occur in the future.

Sponsored by:	EMC / Isilon Storage Division
2016-06-05 23:05:04 +00:00
bdrewery
1e55525ea3 WITH_META_MODE: Avoid host tool timestamps causing a rebuild.
Using buildworld, installworld, buildworld.  It is expected that nothing
should rebuild.  However any host tool used could have its timestamp
updated.  Any library used by dynamic tools could have its timestamp
updated.  The filemon(4) data in the .meta files captures all reads to
these files.  This causes the 2nd buildworld to rebuild everything since
host tools and files have been updated.

Because the build is self-reliant and bootstraps itself, it should be
safe to ignore mtime changes on host files used during the build.  Host
files should only impact the build of legacy, build-tools, bootstrap-tools,
cross-tools, but those are already intended to be reproducible from its
own bootstrapping.  It is possible in a rare case that a bug in a host
file does produce a broken build tool.  If that happens it will just
have to be communicated properly.

An alternative solution would be to update the mtime of all files in the
object directory after installworld so that the host files are not newer
than the object files.  That also requires special care for read-only
obj directories and special care to not mess with any intended timestamps in
the build, such as done for reproducibility.

Reported by:	many
Sponsored by:	EMC / Isilon Storage Division
2016-06-05 23:05:01 +00:00
bdrewery
39ee029a9c WITH_META_MODE: Enable bmake's missing meta rebuild feature 2016-06-05 23:04:42 +00:00
bdrewery
633cc97eae Import latest meta.stage.mk 1.45 (r301462) 2016-06-05 21:21:41 +00:00
bdrewery
7e8741e191 WITH_META_MODE: Avoid "building" .depend if there is nothing to do.
This avoids 'Building /path/.depend' when it will not actually produce a
file.

Sponsored by:	EMC / Isilon Storage Division
2016-06-03 21:35:19 +00:00
bdrewery
0eeb947db6 WITH_CCACHE_BUILD + WITH_META_MODE: Ignore ccache changes.
Ccache will not affect the output of the objects, so just ignore it for
meta mode handling.  This avoids having everything rebuild if ccache is
updated.

Sponsored by:	EMC / Isilon Storage Division
2016-06-03 19:25:45 +00:00