Commit Graph

1454 Commits

Author SHA1 Message Date
Bryan Drewery
49a1108ae7 Move SVNVERSION_CMD into the one place that uses it.
This code, which is basically `svnversion || svnliteversion`, generates
2 fstatat(2) for every directory in PATH for every Makefile parsed that
includes bsd.own.mk.  This can add up for things like generating a Ports
index (Poudriere) or building a dependency graph for base.

Sponsored by:	Dell EMC
MFC after:	2 weeks
2018-02-20 21:48:16 +00:00
Warner Losh
c1c35c1065 sort needs to be in ITOOLS since find -s was changed to find | sort. 2018-02-09 18:47:00 +00:00
Warner Losh
207efdb345 Add a note about why we have the conditional before including
bsd.compiler.mk. It's so fmake from older 9.x systems still
works (still a supported build config, and having the note here
will let us know when we can cull it more easily).

Also pull in a related change from include to sinclude from
arichardson@'s cross building work, as well as it's companion in
Makefile.inc1 with a note about why we do the odd thing there.

Submitted by: archardson
Differential Revision: https://reviews.freebsd.org/D14241
2018-02-07 16:28:26 +00:00
Ed Maste
4816408016 add retpoline compiler and linker feature flags
These features indicate that the compiler and linker support the
retpoline speculative execution vulnerability (CVE-2017-5715)
mitigation.

Reviewed by:	dim, imp
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D14228
2018-02-07 14:50:06 +00:00
Alex Richardson
fb1df20368 Don't hardcode /usr/bin as the path for mktemp in build tools
It won't work e.g. when crossbuilding from Ubuntu Linux as mktemp is in
/bin there.

Reviewed By:	bdrewery
Approved By:	jhb (mentor)
Differential Revision: https://reviews.freebsd.org/D13937
2018-02-06 15:41:35 +00:00
Wolfram Schneider
a564880c29 `make installkernel' should display a completed message if done
PR:		225159
Reviewed by:	bdrewery
Approved by: 	cem (mentor)
Differential Revision:	https://reviews.freebsd.org/D13940
2018-01-25 21:36:26 +00:00
Bryan Drewery
553b3bb89d test-system-compiler: Display X_ variants for compiler/linker.
Sponsored by:	Dell EMC
2018-01-24 18:09:44 +00:00
Bryan Drewery
1c0cac23cf X_COMPILER_* may not be defined.
Sponsored by:	Dell EMC
2018-01-24 18:08:37 +00:00
Alex Richardson
ebe45c64a0 Don't build share/syscons in build-tools stage if MK_SYSCONS == "no"
Reviewed By:	emaste, jhb
Approved By:	jhb (mentor)
Differential Revision: https://reviews.freebsd.org/D12926
2018-01-16 21:43:36 +00:00
Bryan Drewery
fe8be58826 Ensure MK_LDD for bootstrap-tools matches MK_LLD_BOOTSTRAP.
This fixes building libllvmminimal with MK_LLD support when WITHOUT_LLD
or WITHOUT_TOOLCHAIN are set, which are install flags not build tools
flags.

Reported by:	sbruno
Tested by:	sbruno
Sponsored by:	Dell EMC
2018-01-12 20:40:27 +00:00
Bryan Drewery
d52c29e7f2 Cache LINKER_FEATURES to fix the wrong ones being used.
Submitted by:	Dan McGregor <dan.mcgregor@usask.ca>
2018-01-12 01:09:30 +00:00
Ian Lepore
e93e5bb0fa Allow use of .WAIT in the LOCAL_DIRS and LOCAL_LIB_DIRS lists.
A comment in Makefile.inc1 has long stated that LOCAL_DIRS are built last,
after the base system.  Incremental improvements in parallel building over
the years have led to LOCAL_DIRS being built in parallel with base system
directories.  This change allows the .WAIT directive to appear in LOCAL_DIRS
and LOCAL_LIB_DIRS lists to give the user some control over parallel
building of local additions.

Differential Revision:	https://reviews.freebsd.org/D13622
2017-12-31 18:53:13 +00:00
Ed Maste
3670d4c79a Makefile.inc1: map mips MACHINE_/TARGET_ARCH to triples
This helps Clang MIPS builds.

Reviewed by:	bdrewery, jhb (earlier version)
Differential Revision:	https://reviews.freebsd.org/D12171
2017-12-05 21:42:09 +00:00
Bryan Drewery
9b43e3da5e native-xtools: Ensure GCC files are cleaned up.
Because we force enable MK_GCC when building we need to also force
enable it for the cleaning phase.  Otherwise the NXB_TARGET files
are found in the next build's kernel-toolchain phase and cause
an error.

Reported by:	sbruno
X-MFC-With:	r325001
MFC after:	1 month
Sponsored by:	Dell EMC
2017-12-05 21:36:08 +00:00
Bryan Drewery
4a51c7085e native-xtools: Fix build without META_MODE for GCC archs.
The initial kernel-toolchain is built with TARGET=MACHINE but
we want GCC to have files generated for TARGET=NXB_TARGET
instead later on.  Just clean the files between building of
the toolchain and nxb binaries which will let it rebuild
when needed.

Reported by:	sbruno
X-MFC-With:	r325001
MFC after:	1 month
Sponsored by:	Dell EMC
2017-12-05 02:23:36 +00:00
Hans Petter Selasky
388290b8be Fix for multi-threaded lib32 compat build on amd64.
Sponsored by:	Mellanox Technologies
2017-11-24 14:29:32 +00:00
Hans Petter Selasky
e833305789 Simplify the build dependencies when building the OFED libraries.
Suggested by:	bdrewery@
Sponsored by:	Mellanox Technologies
2017-11-24 09:01:08 +00:00
Hans Petter Selasky
937d37fc6c Merge ^/head r325842 through r325998. 2017-11-19 12:36:03 +00:00
Bryan Drewery
3faecc70e8 Fix 'make -n' for {reinstall,distribute}kernel.
PR:		201779
Sponsored by:	Dell
2017-11-18 20:01:02 +00:00
Hans Petter Selasky
7a310b0696 Update list of hardcoded OFED build order dependencies for
the buildworld target.

Sponsored by:	Mellanox Technologies
2017-11-16 15:13:28 +00:00
Ed Maste
92b800cc1a Sort pkgbase mtree metadata, for reproducible builds
Packaged base packages are created by running the stageworld and
stagekernel targets with -DNO_ROOT, and converting the resulting mtree
file into a set of pkg plists.  If stage* is run with multiple processes
the order of entries in the mtree file may be nondeterministic, and the
resulting package tbz also had nondeterministic file ordering.

The mtree file generated by -DNO_ROOT builds consists of one line per
file, with the filename starting in the first column, so is easily
sorted.  There's one exception: the first line of the mtree file is a
comment, but the # character sorts before the filenames anyhow and needs
no special treatment.

PR:		223673
Reviewed by:	bapt, gjb
Sponsored by:	The Linux Foundation, Core Infrastructure Initiative
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D13103
2017-11-15 18:03:31 +00:00
Warner Losh
ca987d4641 Move sys/boot to stand. Fix all references to new location
Sponsored by:	Netflix
2017-11-14 23:02:19 +00:00
Alex Richardson
7202739f7b Only build the libmagic build-tools if MK_FILE != no
Before this patch libmagic was always built in the build-tools phase
even if -DWITHOUT_FILE was specified.

Reviewed by:	emaste, jhb
Approved by:	jhb (mentor)
Differential Revision: https://reviews.freebsd.org/D12925
2017-11-13 20:07:08 +00:00
Bryan Drewery
f649ab27a3 cleanworld only needs a tree walk if the expected .OBJDIR is .CURDIR.
Sponsored by:	Dell EMC Isilon
2017-11-11 01:10:08 +00:00
Bryan Drewery
9cde684d49 native-xtools: Fix for GCC archs.
- This also adds in a _cleanobj step as needed.
- This redirects TARGET/TARGET_ARCH to NXB_TARGET/NXB_TARGET_ARCH in
  Makefile.inc1 as the main build needs to be for MACHINE rather
  than TARGET.

  First build the toolchain and then use that as an external toolchain
  to build the needed directories and NXB_TARGET-toolchain, all as
  MACHINE files though via TARGET_TRIPLE=MACHINE_TRIPLE.

  The NXBDIRS is evaluated in the 'everything' submake as it needs to be
  based on TARGET's src.opts.mk values, such as MK_GCC=yes when building
  on a MK_CLANG=yes MACHINE.  This can likely be changed to a specific
  _native-xtools-everything target later and the funky late evaluation
  of SUBDIR_OVERRIDE removed.

X-MFC-With:	r325001
Pointyhat to:	bdrewery
MFC after:	2 months
Sponsored by:	Dell EMC Isilon
2017-11-10 19:53:07 +00:00
Bryan Drewery
ae9f7248f9 rescue ipf: Remove hacks and link in libipf directly.
Sponsored by:	Dell EMC Isilon
2017-11-10 07:52:58 +00:00
Bryan Drewery
fa65e3a53d Handle some .OBJDIR == .CURDIR cases.
- If OBJROOT is SRCTOP then don't add on TARGET.TARGET_ARCH.  This
  only happens at the top-level, and for sub-directories when the
  user is clever with MAKEOBJDIRPREFIX=/.
- Don't bother checking 'test -w' on .CURDIR.
- Properly set OBJTOP/OBJROOT to SRCTOP in various needed cases.
- Check if the OBJDIR is writable even for *clean* targets since it
  determines which .OBJDIR the user gets;  If they cannot write to an
  existing eligible .OBJDIR then it needs to clean in .CURDIR instead.
- Add guard to cleanworld/cleanuniverse from removing SRCTOP.
- Ensure OBJTOP is proper for .OBJDIR=.CURDIR which fixes finding
  libraries since src.libnames.mk is based on OBJTOP.
- Avoid some chdir(2) for modifying .OBJDIR

Sponsored by:	Dell EMC Isilon
2017-11-10 02:09:33 +00:00
Warner Losh
3fa561a45b Note interactive shell errors for make buildenv and add a warning for
people tempted to add back the || true to get rid of them.

Sponsored by: Netflix
2017-11-07 15:01:38 +00:00
Bryan Drewery
996e475962 Fix nested MAKEOBJDIRPREFIX breaking various release/buildworld/toolchain targets.
This problem was caused by r325329 and r325350.

For the release(7) targets, some will run mm-mtree.sh which itself runs make
with a MAKEOBJDIRPREFIX.  The execution of that script leaks OBJROOT,
MAKEOBJDIR, and MAKELEVEL=1 in the environment.  This causes the mm-mtree makes
to not do some basic setup of OBJROOT and only use this special
MAKEOBJDIRPREFIX case which fails to empty out MAKEOBJDIRPREFIX for further
nested makes, such as a tree walk.  If that tree walk sets OBJROOT/OBJTOP such
as r325329 is doing, then the wrong OBJDIRs end up being used due to the
unemptied MAKEOBJDIRPREFIX being preferred over the proper MAKEOBJDIR.

Pointyhat to:	bdrewery
Sponsored by:	Dell EMC Isilon
2017-11-05 00:11:51 +00:00
Bryan Drewery
2abeba9d7a Follow-up r318105: Don't use NO_OBJ at top-level, use NO_OBJWALK.
NO_OBJ has a very specific meaning in sub-directories in that no object
directory will be made.  If a user wanted to skip the 'make obj' phase then
passing -DNO_OBJ would break all sub-directories from building properly.  Using
NO_OBJ internally also causes issue with NO_OBJ handling being added in
share/mk/bsd.init.mk soon.

Sponsored by:	Dell EMC Isilon
2017-11-03 23:22:03 +00:00
Bryan Drewery
2c36cefeaf Follow-up r325329: Store all WORLDTMP objects back in the same directory.
This still keeps the reduced MAKEOBJDIRPREFIX (SRCTOP) redundancy
removed in the OBJDIR, but now keeps all early phase objects
in the same directory rather than split per phase.

The problem of splitting per phase is that later phases want to link in
libraries from earlier phases and base their location on ${OBJTOP}.

Pointyhat to:	bdrewery
Reported by:	mjoras, Mark Millard
Sponsored by:	Dell EMC Isilon
2017-11-03 00:29:42 +00:00
Bryan Drewery
fe480405ec Follow-up r325342: Always ensure WORLDTMP exists.
Reported by:	Herbert J. Skuhra
Sponsored by:	Dell EMC Isilon
2017-11-02 21:58:18 +00:00
Bryan Drewery
89bd07e927 DIRDEPS_BUILD: Cleanup leftover WORLDTMP files.
This will force any existing objects to rebuild if their .meta
files reference files from WORLDTMP.  This is a problem after
the UNIFIED_OBJDIR effort caused buildworld and DIRDEPS_BUILD
to share an OBJDIR.  Without cleaning these files the
Makefile.depend files end up with odd tmp/legacy/... entries
since some bootstrap-tools linger from there and otherwise
don't rebuild.  Removing the files causes anything having
used WORLDTMP to rebuild.

Sponsored by:	Dell EMC Isilon
2017-11-02 20:45:04 +00:00
Bryan Drewery
4dc89c4d51 Reduce MAKEOBJDIRPREFIX path spam by specifying a direct objdir to use.
Sponsored by:	Dell EMC Isilon
2017-11-02 18:08:36 +00:00
Bryan Drewery
3d165b2f1e native-xtools/xdev: Store these in a location including both MACHINE/TARGET.
Given MACHINE/MACHINE_ARCH=amd64, TARGET=arm64 and TARGET_ARCH=aarch64,
this change will build them in a location such as:
    /usr/obj/usr/src/amd64.amd64/nxb/arm64.aarch64/
and
    /usr/obj/usr/src/amd64.amd64/xdev/aarch64-freebsd/

Sponsored by:	Dell EMC Isilon
2017-11-01 21:22:13 +00:00
Bryan Drewery
c10062b9bf Add a 'make cleanuniverse'.
This will remove all build files for the source directory
when MK_UNIFIED_OBJDIR is enabled.

Sponsored by:	Dell EMC Isilon
2017-11-01 21:22:10 +00:00
Bryan Drewery
dfa099890c Add option UNIFIED_OBJDIR, on by default, which moves the default build OBJDIR.
This changes the build OBJDIR from the older style of /usr/obj/<srcdir> for
native builds, and /usr/obj/<target>.<target_arch>/<srcdir> for cross builds to
a new simpler format of /usr/obj/<srcdir>/<target>.<target_arch>.  This
new format is used regardless of cross or native build.  It allows
easier management of multiple source tree object directories.

The UNIFIED_OBJDIR option will be removed and its feature made permanent
for the 12.0 release.

Relnotes:	yes (don't note UNIFIED_OBJDIR option since it will be removed)
Prior work:	D3711 D874
Reviewed by:	gjb, sjg
Discussed at:	https://lists.freebsd.org/pipermail/freebsd-arch/2016-May/017805.html
Discussed with:	emaste
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D12840
2017-11-01 21:22:05 +00:00
Bryan Drewery
f069674019 xdev: Follow-up r325087: Need to build lib/clang before lld.
LLD needs headers generated by the full libllvm.

X-MFC-With:	r325087
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-10-31 23:48:57 +00:00
Bryan Drewery
651c6845c7 kernel-toolchain: Fix improper build order after r325244.
Due to removing some targets that the previous .ORDER: ${WMAKE_TGTS}
set, it was no longer being respected; _build_tools was coming
immediately.

Pointyhat to:	bdrewery
X-MFC-with:	r325244
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-10-31 21:51:33 +00:00
Bryan Drewery
5a4121a800 kernel-toolchain: Skip world _obj and _cleanobj phases.
There's no good reason to treewalk the entire tree removing old OBJDIRS
and creating new ones when 'includes', 'libraries', and 'everything' are
all skipped.  The only shared directory in the existing toolchain target
and world is build-tools.  So handle cleaning in it directly if needed
only for the directories it wants to build.

The extra _obj/_cleanobj walks came in the initial kernel-toolchain
addition in r128189.

MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-10-31 19:02:14 +00:00
Bryan Drewery
7cd84b5641 build-tools: De-special-case the gcc tools build.
It merely wanted to use 'all' rather than 'build-tools' so just
add a build-tools target to the Makefile.

MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-10-31 19:02:05 +00:00
Bryan Drewery
555c6e60bc Follow-up r297998: Remove redundant TOOLS_PREFIX in XMAKE.
Sponsored by:	Dell EMC Isilon
2017-10-31 17:16:46 +00:00
Bryan Drewery
337698b78d native-xtools: Allow SYSTEM_COMPILER to be used after r325001.
Now that a proper sysroot is used and a separate target list, it should
be safe to skip building the initial cross-compiler.

X-MFC-With:	r325001
MFC after:	1 month
Sponsored by:	Dell EMC Isilon
2017-10-31 02:52:36 +00:00
Bryan Drewery
601594ed6a cleanworld: No need to cleandir if MK_AUTO_OBJ is enabled.
Sponsored by:	Dell EMC Isilon
2017-10-31 01:45:14 +00:00
Warner Losh
0255bc1370 Return proper status from buildenv.
make buildenv BUILDENV_SHELL=<some command> more useful. Remove '||
true' from the command line so that errors are properly
returned. There appears to be no reason for it, and it dates back to
the original commit by ru@.

Sponsored by: Netflix
2017-10-30 03:12:38 +00:00
Bryan Drewery
be1ef37b94 xdev: Consolidate duplicate cross-tools build/install list into XDEVDIRS.
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-10-29 01:21:52 +00:00
Bryan Drewery
86c4ac993d xdev: Remove excess '/' since XDTP is asserted to be an absolute path.
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-10-29 01:21:49 +00:00
Bryan Drewery
c16a698b5e xdev: Fix build for platforms needing LLD.
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-10-29 01:21:46 +00:00
Bryan Drewery
57820ca8df xdev: Add missing 'make obj' treewalk for 'make includes'.
It is possible that building headers requires an OBJDIR.
The other phases of xdev have their own 'make obj' calls
where needed, such as inside 'make libraries' itself.

MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-10-29 01:21:42 +00:00
Bryan Drewery
5bd47cc56f Change native-xtools to not install by default; add a native-xtools-install.
Without this the user has to mess with 'make -f Makefile.inc1 ...' to figure
out where the files are installed in the OBJDIR and then they need to copy them
to where they really wanted them. Using DESTDIR may be problematic after
r325001 as well.

The files will be installed to DESTDIR/NXTP where NXTP defaults to /nxb-bin.

MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-10-29 01:21:29 +00:00