2863 Commits

Author SHA1 Message Date
bdrewery
f5c5b3c9c9 CCACHE_BUILD: Follow-up r321880: Fix some PATH issues with buildworld.
- bsd.compiler.mk: Must ensure that the CCACHE_WRAPPER_PATH comes first
  in PATH.
- Makefile.inc1: Must prepend the CCACHE_WRAPPER_PATH into BPATH as it
  overrides the PATH set in bsd.compiler.mk in sub-makes.  The PATH
  set in bsd.compiler.mk is not exported and doing so would cause it to
  then override the BPATH set from environment.  The only sane solution
  is to prepend into BPATH as needed.
CCACHE_PATH could possibly be used for some of this as well.

Sponsored by:	Dell EMC Isilon
2017-08-01 18:26:20 +00:00
bdrewery
dfd301532b CCACHE_BUILD: Allow setting CCACHE_BUILD_TYPE=wrapper.
This uses the /usr/local/libexec/ccache/<cc,c++> wrappers rather than
modifying CC to be '/usr/local/bin/ccache cc'.  Some forms of compilation
do not support the 'command' type.

Sponsored by:	Dell EMC Isilon
2017-08-01 16:15:08 +00:00
bdrewery
17e3f83790 Allow -DNO_SKIP_DEPEND to override the _SKIP_DEPEND logic.
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-07-25 16:46:12 +00:00
andrew
f5a021507a Build the 32-bit ARM libstand and loader parts with -fPIC. Many of them are
already built with this flag so libstand should also be build as such.
This will be needed when moving to lld as it refuses to link due to
incompatible relocations.

Sponsored by:	DARPA, AFRL
2017-07-25 10:41:34 +00:00
bdrewery
9628714bcb cleandir: Fix ESTALE errors from parallel removals.
This fixes 'make cleandir' to use the same ordering as 'make cleanobj'.
Meaning that SUBDIR will be recursed before the current directory is
handled.  This avoids an 'rm -rf /usr/obj/usr/src/lib/libc' while
a child 'rm -rf /usr/obj/usr/src/lib/libc/tests' is being ran next,
or even removing the current directory and then recursing into a child
and using the 'missing OBJDIR' logic to remove files rather than the
directory.

The most ideal ordering here would be for 'cleanobj' and 'cleandir' to
simply remove the .OBJDIR and then not recurse at all.  This is only
safe if it is guaranteed that all children directories have no orphaned
files in their source checkout and are only using obj directories.  This
is usually safe from the top-level build targets and when using
WITH_AUTO_OBJ.  Improving the build for those cases is coming.

Reported by:	cperciva, scottl
X-MFC-With:	r321427
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-07-25 00:12:48 +00:00
bdrewery
e007570379 Allow disabling dependency tracking if DEPEND_CFLAGS is empty.
This falls back on using the guesssed dependencies if so.
Also remove a pre-bmake check while here.

MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-07-24 23:32:40 +00:00
bdrewery
4497a71c5a Slightly simplify logic for which depend file is expected.
This is a NOP.

MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-07-24 23:32:36 +00:00
ngie
f602f06427 Use __DEFAULT_DEPENDENT_OPTIONS for articulating dependency relationship
between MK_STALE_STAGED and MK_STAGING instead of using equivalent ad hoc
logic.

MFC after:	1 month
2017-07-24 18:21:01 +00:00
bdrewery
635b1e3eb5 PROGS: Fix ESTALE errors on NFS while cleaning in directories with PROGS.
- Only recurse on cleanobj/cleandir if there is no .OBJDIR being used.
  If we don't recurse then bsd.obj.mk will just rm -rf the OBJDIR dir.
- When recursing on cleanobj/cleandir don't remove dependfiles/dirs
  redundantly from the child and main processes.  Meaning '.depend', and
  'tags', and '.depend.*' will now only be removed from the main
  process.
- Stop recursing on 'cleandepend' since the main process can handle
  removing all files via the default glob patterns in CLEANDEPENDFILES.
- This reverts r288201, by readding recursion on 'cleanobj', due to
  r291635 changing how bsd.subdir.mk handles recursion.

This is primarily targeting ESTALE NFS errors from rm(1) during a
buildworld but is also a performance optimization as both issues fixed
were redundant anyway.

Reported by:	cperciva, scottl
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-07-24 17:54:03 +00:00
bdrewery
0355687f0f cleanobj: Unhide removal of directory.
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-07-24 17:53:45 +00:00
emaste
f2930be3fb lld 5.0 supports filter libraries, so enable linker feature flag
Also switch the logic to enable this for any non-lld linker, since
filter library support is fairly simple and is very likely supported
by any other linker capable of linking the FreeBSD base system.

MFC after:	2 months
MFC with:	r321369
Sponsored by:	The FreeBSD Foundation
2017-07-24 15:39:09 +00:00
bdrewery
3abbcde98f Respect INSTALL_AS_USER for FILES.
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-07-21 16:14:35 +00:00
br
c8b8ae9609 Add warning flags for GCC 7.1.0 compiler.
Sponsored by:	DARPA, AFRL
2017-07-21 14:50:32 +00:00
ngie
c14f6dfb65 Sort the tests alphabetically before adding them to the Kyuafiles
This is being done to aid in debugging test runs, in the event the
output shifts due to refactored Makefiles, added tests, etc.

MFC after:	1 month
2017-07-17 18:20:54 +00:00
kib
84be924362 Provide libdl.
Create libdl.so.1 as a filter for libc.so.7 which exports public dl*
functions. The functions are resolved from the rtld instead, the goal
of creating library is to avoid errors from the static linker due to
missed libdl. For static binaries, an empty .o is compiled into
libdl.a so that static binaries still get dl stubs from libc.a.

Right now lld cannot create filter objects, disable libdl on arm64
when binutils are not used.

Reviewed by:	bdrewery, dim (previos version); emaste
Exp run:	PR 220525, done by antoine
Sponsored by:	The FreeBSD Foundation
MFC after:	1 month
Differential revision:	https://reviews.freebsd.org/D11504
2017-07-10 14:59:21 +00:00
ngie
a448519a3c Formalize LEAPSECONDS and OLDTIMEZONES in share/zoneinfo/... as
`MK_ZONEINFO_LEAPSECONDS_SUPPORT == yes` and
`MK_ZONEINFO_OLD_TIMEZONES_SUPPORT == yes`.

Keep `LEAPSECONDS` and `OLDTIMEZONES` for backwards compatibility,
but print out a warning notifying users that they should use the new
variables, in an effort to migrate them to the variables. This is being
done mostly for automated build tools, etc, that might rely on these
variables being set. The variables will be removed in the future on
^/head, e.g., after ^/stable/12 is cut.

MFC after:      1 month
Relnotes:       yes
Reviewed by:	bdrewery
Differential Revision:	D11376
2017-07-06 04:19:33 +00:00
emaste
16cc7b4e31 Do not build clang for all riscv*, not just riscv64
Previous test matching on "riscv64" was invalidated by the addition of
riscv64sf.

Sponsored by:	The FreeBSD Foundation
2017-07-01 19:10:59 +00:00
jlh
5c2b155b69 Disable RCMDS by default.
This was announced in this thread:
https://lists.freebsd.org/pipermail/freebsd-arch/2017-June/018239.html

Applying plan proposed by ngie@ in:
https://lists.freebsd.org/pipermail/freebsd-arch/2017-June/018249.html

The port has been submitted as net/bsdrcmds in r444814.

Approved by:	bapt, roberto, and others
2017-07-01 10:04:42 +00:00
emaste
e386983c91 bsd.linker.mk: add band-aid for linker invocation failure
In some cases bsd.linker.mk reports an error like:

  make[4]: ".../share/mk/bsd.linker.mk" line 56:
  Unknown linker from LD=ld -m elf32ppc_fbsd:"

For now change this to a .warning, and then assume GNU ld 2.17.50.
At present the linker type detection is used only for enabling build-id,
and we can carry on without it when type detection fails.

Also, show errors from ${LD} --version to aid in failure diagnosis.
Successful invocations of ${LD} --version produce no output on stderr
so this will not create any spam in non-failing builds.

Tested by:	swills
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D11424
2017-06-30 16:34:17 +00:00
emaste
3bde37655a Introduce LINKER_FEATURES to avoid duplicating version logic
Submitted by:	bdrewery
Reported by:	kib
2017-06-23 17:21:37 +00:00
emaste
5816cc6184 change GNU ld LINKER_TYPE from binutils to bfd
GNU binutils includes two linkers: ld.bfd and ld.gold.  For clarity use
LINKER_TYPE=bfd to refer to ld.bfd, the original binutils linker that
identifies itself as "GNU ld".

Discussed with:	bdrewery
2017-06-23 03:55:00 +00:00
bdrewery
656ec7cc2c Provide proper values for X_LINKER_TYPE/VERSION when XLD == LD.
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-06-22 22:53:10 +00:00
bdrewery
3132dadd89 Don't overwrite already-set LINKER_VERSION/LINKER_TYPE.
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-06-22 22:53:07 +00:00
bdrewery
fe8fd0e30d LINKER_VERSION: Support external binutils.
The ports binutils stores the version in the 5th word so just look for
a version using a pattern instead.

Reported by:	rpokala
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-06-22 21:22:12 +00:00
bdrewery
74c01096d1 Support cached linker values in environment.
This is similar to r289659 for bsd.compiler.mk.

MFC after:	2 weeks
Reviewed by:	emaste
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D11310
2017-06-22 21:03:27 +00:00
bdrewery
d53b98880f Support XLD for setting X_LINKER_TYPE and X_LINKER_VERSION.
This is similar to r300350 for bsd.compiler.mk.

MFC after:	2 weeks
Reviewed by:	emaste
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D11309
2017-06-22 21:03:24 +00:00
bdrewery
7cf0e0ad69 Add basic bsd.linker.mk auto included from bsd.compiler.mk.
This will provide LINKER_TYPE and LINKER_VERSION.

MFC after:	2 weeks
Reviewed by:	emaste
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D11308
2017-06-22 21:03:20 +00:00
bdrewery
4a42972a1d Fix spelling error.
Reported by:	arc
Sponsored by:	Dell EMC Isilon
2017-06-22 20:32:23 +00:00
bdrewery
63cec4ec0f Rework logic for skipping .depend/.meta file read/stat/writes.
- Rename _SKIP_READ_DEPEND to _SKIP_DEPEND since it also avoids writing.
- This now uses .NOMETA to avoid reading any .meta files related to
  DEPENDOBJS.  Objects not in OBJS/DEPENDOBJS may still have their .meta
  files read in if they are in the dependency graph.
- This also avoids statting .meta and .depend files in the META_MODE +
  -DNO_FILEMON case.

MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-06-22 05:34:41 +00:00
bdrewery
6eab5891b5 Remove logic for setting .MAKE.DEPENDFILE=/dev/null already covered by _SKIP_BUILD.
_SKIP_BUILD is defined in bsd.init.mk.

MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-06-21 20:10:58 +00:00
bdrewery
70a89a6800 Similar to r296013 for NO_ROOT, force SUBDIR_PARALLEL for buildworld WORLDTMP staging.
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-06-21 18:56:53 +00:00
bdrewery
4f4f919cf3 objwarn should be .PHONY.
Otherwise in META_MODE it may create an objwarn.meta if only bsd.obj.mk
is included;  bsd.sys.mk already had .PHONY: objwarn.

MFC after:	3 days
Sponsored by:	Dell EMC Isilon
2017-06-21 17:11:49 +00:00
bdrewery
57488d2ac8 Fix 'make clean all' to work again.
This likely broke completely with r308599.

Apply the same fix for 'make destroy' which is a DIRDEPS_BUILD thing.

PR:		219819
Reported by:	trasz
MFC after:	3 days
Sponsored by:	Dell EMC Isilon
2017-06-20 20:52:06 +00:00
bdrewery
690a0d91eb LIBADD: Try to support partial tree checkouts in some limited cases.
LIBADD is only supported for in-tree builds because we do not install
share/mk/src.libnames.mk (which provides LIBADD support) into /usr/share/mk.
So if a partial checkout is done then the LIBADDs are ignored and no LDADD is
ever added.

Provide limited support for this case for when LIBADD is composed entirely of
base libraries.  This is to avoid clashes with ports and other out-of-tree
LIBADD uses that should not be mapped to LDADD and because we do not want to
support LIBADD out-of-tree right now.

Reported by:	mckusick, kib
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-06-20 20:34:30 +00:00
emaste
c4e2c09d06 Fall back to GPL dtc(1) when we lack a C++11 compiler
The BSD licensed device tree compiler is written in C++11.

Reported by:	jhibbits
Reviewed by:	jhibbits, manu
Sponsored by:	The FreeBSD Foundation
2017-06-20 15:51:09 +00:00
manu
482531ced3 Switch back to the BSDL DTC (Device Tree Compiler).
The BSDL dtc has grown the needed features (overlays mostly) and is able to
compile all of our base DTS.
You can use WITH_GPL_DTC is you need the GPL one or DTC= in make.conf(5)
to specify an alternate location for the compiler to use.

Discussed with: emaste, imp
2017-06-20 02:09:50 +00:00
bdrewery
6c4b8e235b Utilize SYSROOT from r320119 in places where DESTDIR may be wanting WORLDTMP.
Since buildenv exports SYSROOT all of these uses will now look in
WORLDTMP by default.

sys/boot/efi/loader/Makefile
        A LIBSTAND hack is no longer required for buildenv.

MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-06-19 20:47:24 +00:00
bdrewery
c3d7eb1d7b Follow-up r320061: Need to respect make.conf/env LIBDIR overrides.
This fixes the lib32 build from creating all stale .depend files.

X-MFC-With:	320061
MFC after:	1 week
Sponsored by:	Dell EMC Isilon
2017-06-19 18:08:02 +00:00
bdrewery
b40354c0a7 Fix Makefiles which override LIBDIR to not add incorrect dependencies into .depend.
This fixes these cases which would rebuild every time:
    make[6]: /usr/obj/usr/src/libexec/rtld-elf/tests/libpythagoras/.depend, 1: ignoring stale .depend for /usr/obj/usr/src/tmp/usr/tests/libexec/rtld-elf/libm.a
    make[6]: /usr/obj/usr/src/lib/libxo/tests/encoder/.depend, 1: ignoring stale .depend for /usr/obj/usr/src/tmp/usr/tests/lib/libxo/libxo.a
    make[7]: /usr/obj/usr/src/lib/libthr/tests/dlopen/dso/.depend, 1: ignoring stale .depend for /usr/obj/usr/src/tmp/usr/tests/lib/libthr/dlopen/libpthread.a

The problem is that some Makefiles will override LIBDIR to where they want
their library to install.  bsd.libnames.mk will then use ${LIBDIR} to define
where *existing* libraries are.  This then leads to looking for the
libraries in the *target* place rather than the *expected* place.

We may want to expand this (and all of the other *DIR variables in bsd.own.mk)
into something like what Ports has, a PREFIX and a LOCALBASE.  PREFIX being
where things are being installed to and LOCALBASE being where they already are.

For now store the default expected LIBDIR into LIBDIR_BASE and use that for
library locations.

Reported by:	sbruno
MFC after:	1 week
Sponsored by:	Dell EMC Isilon
2017-06-17 20:33:11 +00:00
bdrewery
df1398b14a WITH_META_MODE: End each ERROR_CMD CMD line with ';'.
This makes it easier to debug multi-line command failures.

X-MFC-With:	r319862
MFC after:	1 week
Sponsored by:	Dell EMC Isilon
2017-06-16 21:48:42 +00:00
bdrewery
488ac365fc Fix more incorrect library directories fix 'stale .depend' rebuilds.
Reported by:	sbruno
MFC after:	3 days
Sponsored by:	Dell EMC Isilon
2017-06-16 21:37:05 +00:00
bdrewery
3ed41bbad6 Fix LIBAMU location to fix 'stale .depend' rebuilds in usr.sbin/amd.
This originally came in r275052.

Reported by:	sbruno
MFC after:	3 days
Sponsored by:	Dell EMC Isilon
2017-06-16 20:47:12 +00:00
bdrewery
af137d191a WITH_META_MODE: Don't try showing command if .ERROR_META_FILE is empty.
This was sed'ing on stdin for failing .PHONY targets.

Reported by:	Mark Millard
X-MFC-With:	r319862
MFC after:	1 week
Sponsored by:	Dell EMC Isilon
2017-06-16 04:28:10 +00:00
bdrewery
55bd2850e7 META_MODE: Show .ERROR_CMD in error.
This uses a hack to get the CMD from the meta file rather than
.ERROR_CMD since bmake currently blanks the value for non-jobs
mode.

Reviewed by:	sjg (indirectly)
MFC after:	1 week
Sponsored by:	Dell EMC Isilon
2017-06-12 19:13:29 +00:00
bdrewery
659ce4bc88 META_MODE: NO_FILEMON should imply nofilemon.
This fixes NO_FILEMON to properly still use .depend.OBJ files
for dependency tracking.

MFC after:	1 week
Sponsored by:	Dell EMC Isilon
2017-06-12 18:44:14 +00:00
bapt
c0d513aaf1 Finish removal of groff 2017-06-07 23:26:28 +00:00
bapt
ac6edc194c Remove groff from base
All manpages in base are now compatible with mandoc(1), all roff documentation
will be relocated in the doc tree. man(1) can now use groff from the ports tree
if it needs.

Also remove checknr(1) and colcrt(1) which are only useful with groff.

Approved by:	(no objections on the mailing lists)
2017-06-07 23:00:34 +00:00
bdrewery
3deaa65c41 META_MODE: Add framework to force rebuilding for major ABI changes.
Normally META_MODE ignores host files for "meta mode" decisions on whether a
file should be rebuilt or not.  This is because a simple installworld can
update timestamps and cause the next build to rebuild all host tools, when the
previous ones may not have any changes in the source tree.  These tools are
normally still ABI compatible.  They are only rebuilt if NO_META_IGNORE_HOST is
set from the workaround/hack in r301467.

One of the major problems with this is when a host tool has objects spread
across many revisions that have mixed-ABI.  For example, if struct stat were to
change on the host, some objects for a tool may have different ideas of that
struct's definition.  If just 1 source file were modified and rebuilt and
linked into the tool, then that toll will have mixed-ABI objects and crash.
This exact thing happened with the ino64 commit in r301467 followed by a
trivial update to libbfd in r318750.  The resulting binary would crash in
buildworld.

Sponsored by:	Dell EMC Isilon
2017-06-05 05:17:39 +00:00
bdrewery
e146f9c0bb META_MODE: Allow not ignoring host headers with NO_META_IGNORE_HOST_HEADERS.
See r301467 for more details on NO_META_IGNORE_HOST.  Usually the full
list of host ignores should have no real impact on the host tools.  The
headers however may reliably define what the ABI is for the host.  It
may be useful to allow using the headers for the build but still not
caring about things like /bin/sh, /lib/libedit.so, etc.

Sponsored by:	Dell EMC Isilon
2017-06-05 05:17:36 +00:00
bdrewery
f436c649ba META_MODE: Move ignoring of /usr/local/etc/libmap.d to proper place.
This was added in r318194 but local.meta.sys.mk is only used for
DIRDEPS_BUILD.
2017-06-01 23:04:44 +00:00