Now it can be used to effectively "build in a subdir". It will use the
'cross-tools', 'libraries', and 'includes' phases of 'buildworld' to properly
setup a WORLDTMP to use. Then it will build 'everything' only in the
listed SUBDIR_OVERRIDE directories. It is still required to list custom
library directories in LOCAL_LIB_DIRS if SUBDIR_OVERRIDE is something
that contains libraries outside of the normal area (such as
SUBDIR_OVERRIDE=contrib/ofed needing LOCAL_LIB_DIRS=contrib/ofed/usr.lib)
Without these changes, SUBDIR_OVERRIDE with buildworld was broken or hit
obscure failures due to missing libraries, includes, or cross compiler.
SUBDIR_OVERRIDE with 'make <target that is not buildworld>' will continue to
work as it did before although its usefulness is questionable.
With a fully populated WORLDTMP, building with a SUBDIR_OVERRIDE with
-DNO_CLEAN only takes a few minutes to start building the target
directories. This is still much better than building unneeded things via
'everything' when testing small subset changes. A BUILDFAST or
SKIPWORLDTMP might make sense for this as well.
- Add in '_worldtmp' as we still need to create WORLDTMP as later targets,
such as '_libraries' and '_includes' use it. This probably was avoiding
calling '_worldtmp' to not remove WORLDTMP for debugging purposes, but
-DNO_CLEAN can be used for that.
- '_legacy' must be included since '_build-tools' uses -legacy.
The SUBDIR_OVERRIDE change came in r95509, while -legacy being part
of build-tools came in r113136.
- 'bootstrap-tools' is still skipped as this feature is not for
upgrades.
- Fix buildworld combined with SUBDIR_OVERRIDE not installing all includes.
The original change for SUBDIR_OVERRIDE in r95509 kept '_includes'
and '_libraries' as building everything possible as the SUBDIR_OVERRIDE
could need anything from them. However in r96462 the real 'includes'
target was changed from manual sub-makes to just recursing 'includes'
on SUBDIR, thus not all includes have been installed into WORLDTMP since then
when combined with 'buildworld'.
This is not done unless calling 'make buildworld' as it would be
unexpected to have it go into all directories when doing 'make
SUBDIR_OVERRIDE=mydir includes'.
- Also need to build the cross-compiler so it is used with --sysroot.
If this is burdensome then telling the build to use the local compiler
as an external compiler (thus using a proper --sysroot to WORLDTMP) is
possible by setting CC=/usr/bin/cc, CXX=/usr/bin/c++, etc.
- Don't build the lib32 distribution with SUBDIR_OVERRIDE in buildworld
since it won't contain anything related to SUBDIR_OVERRIDE. Testing
of the lib32 build can be done with 'make build32'.
- Document these changes in build.7
Sponsored by: EMC / Isilon Storage Division
MFC after: 2 weeks
Extend OptionalObsoleteFiles.inc to delete all lib32 files when MK_LIB32 is
set to no on a system that previously had lib32 libraries installed.
Also, to prevent "make delete-old-dirs" from always deleting lib32 directories
after an installworld, move the lib32 subtree to its own mtree file that only
gets applied when MK_LIB32=yes.
Test: Ran "make delete-old" and "make delete-old-libs" on a system that never
had MK_LIB32 enabled, and on a system where MK_LIB32 was enabled and later
disabled. Did this both on amd64 and powerpc64.
Test: Ran "make tinderbox" without errors.
Reviewed by: emaste
Differential Revision: https://reviews.freebsd.org/D3923
This has been handled since r228158 made MK_CTF dependent on MK_CDDL
in share/mk/bsd.opts.mk.
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
The 'includes' target is currently a pseudo target in bsd.subdir.mk that
does 'cd ${.CURDIR} && ${MAKE} buildincludes && ${MAKE} installincludes',
versus all over targets that just recurse.
In Makefile.inc1 the older duplicated bsd.subdir.mk logic for calling
'includes' was being executed in each subdir directly, meaning 'cd lib && make
includes' became 'cd lib && make buildincludes && make installincludes'. Now
that the bsd.subdir.mk logic is used it is calling 'make buildincludes && make
installincludes' from the top-level which pulls in the PATH=<default path>
from /Makefile.
The sub-make logic for 'includes' in bsd.subdir.mk was attempted to be removed
in r289282 but turned out to be wrong. I have a working version now but
it is not yet ready for commit. So for now in Makefile.inc1 split out
'includes' to 'buildincludes' and 'installincludes' which will avoid the
problem.
MFC after: 2 weeks
X-MFC-With: r289438
Sponsored by: EMC / Isilon Storage Division
sub-makes.
Some of the world phases that used plain '${MAKE} -f Makefile.inc1' were not
passing this variable along which caused them to look it up again. By
using bmake's .export we can remove it from all of the other environment
lines.
Add a comment about the usage for VERSION for ctfmerge.
Sponsored by: EMC / Isilon Storage Division
We pass BOOTSTRAPPING=${OSRELDATE} to some of the sub-makes. Rather than
chase every ${MAKE} invokation, just export it as bmake lets us.
Sponsored by: EMC / Isilon Storage Division
Back in r30113, the 'par-*' targets were added to parallelize portions of
the build in a very similar fashion as the SUBDIR_PARALLEL feature used in
r263778. Calling a target without 'par-' (for 'parallel') resulted in the
standard bsd.subdir.mk handling without parallelization. Given we have
SUBDIR_PARALLEL now there is no reason to duplicate the handling here.
In build logs this will result in the ${dir}.${target}__D targets now showing
as the normal ${target}_subdir_${dir} targets.
I audited all of the uses of Makefile.inc1 and Makefile's targets that use
bsd.subdir.mk and found that all but 'all' and 'install' were fine to use
as always parallel.
- For 'install' (from installworld -j) the ordering of lib/ and libexec/
before the rest of the system (described in r289433), and etc/ being last
(described in r289435), is all that matters. So now a .WAIT is added in
the proper places when invoking any 'install*' target. A parallel
installworld does work and took 46% of the time a non-parallel
install would take on my system with -j15 to ZFS.
- For 'all' I left the default handling for this to not run in parallel. A
'par-all' target is still used by the 'everything' stage of buildworld
to continue building in parallel as it already has been. This works
because most of the dependencies are handled by the early bootstrap
phases as well as 'libraries' and 'includes' phases. This lets
all of the SUBDIR build in parallel fine, such as bin/ and lib/. This
will not work if the user invokes 'all' though as we have dependencies
spread all over the system with no way to depend between them (except
for the dirdeps feature in the META_MODE build). Calling 'make all'
from the top-level is still useful at least when using SUBDIR_OVERRIDE.
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
The ordering of 'etc' in the install has a long history dating back to the
first time it was realized it needed to be "last" in r4486. That commit
still left it before LOCAL_DIRS though. By having it before LOCAL_DIRS
any manpages they install were not being added to the whatis database in the
install image. They would likely show up in the file after a periodic
rebuild of the file though.
Currently the whatis file is built by an 'afterinstall' hook in etc/Makefile
that calls share/man's 'makedb' target.
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
It was added in r152006 to handle serializing access of info/dir when
installing INFO files. We no longer support INFO files since r276551
though.
Sponsored by: EMC / Isilon Storage Division
The case of make(1) using a new /bin/sh issue was fixed in r173219 when ITOOLS
was introduced.
There are still issues with mid-install errors leaving a system unusable that
are currently non-trivial to solve. The safest ordering requires installing
rtld, libc and libthr (in that order) before anything else. We don't do that
now though. Much improvement is needed here still.
Discussed with: kip and kan (rtld/library ordering)
MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division
This was handled for libraries in r256842 but for some reason was missed
for files (bsd.prog.mk).
MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division
Relnotes: yes
This was causing files to be removed from the objdir when -n was used.
_worldtmp makes no sub-make calls.
A more comprehensive solution is coming involving fine-grained '+' where
appropriate.
Sponsored by: EMC / Isilon Storage Division
MFC after: 1 week
with the current behaviour of calling "distribution" in the etc target.
This allows mergemaster/etcupdate to still work when some configuration will be
moved to be handled in the same directories their source code lives in.
when running the build-tools stage.
The requirement is due to the -P flag used when running m4 from usr.bin/lex
Makefile to generate skel.c. With the old m4 that fails and the failure is
ignored, resulting in an empty(-ish) skel.c, which leads to later build
failures when the misconfigured new lex tool is run.
This enables building -current (and 10-stable after MFC) on a stable-8
system again.
MFC after: 3 days
Bootstrap tools exist for backwards compatibility support. DTrace tools
tools are also needed for cross builds, so belong in cross-tools.
Reviewed by: imp (earlier), markj
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D2144
Stale CTF tools are a frequent source of DTrace issues, and they compile
quickly enough that the increase in build time is negligible.
Reviewed by: emaste, imp
Differential Revision: https://reviews.freebsd.org/D3670
a one word variable, which is quite unexpected from documentation.
So, to avoid double installation of a single kernel, protect the extra
kernels loop with ${BUILDKERNELS:[#]} > 1 conditional.
Sponsored by: Netflix
Sponsored by: Nginx, Inc.
from BUILDKERNELS list. This is more strict, since INSTALLKERNEL by
definition is the first word of BUILDKERNELS list. The previous
code failed if INSTALLKERNEL is a substring of additional kernel name.
Reviewed by: gjb
Sponsored by: Netflix
Sponsored by: Nginx, Inc.
For most cases they are equivalent, but BINUTILS_BOOTSTRAP is a
BROKEN_OPTION on arm64 as the in-tree GNU binutils do not support it,
so we need a separate internal flag for ELF Tool Chain.
Reviewed by: andrew, brooks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D3381
The option was added only to ease the transition from GNU Binutils to
ELF Tool Chain tools, and that process is now complete (for the viable
replacements). Noting the removal in UPDATING is sufficient as we have
not shipped a release with the option.
Reviewed by: brooks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D3240
changes to prevent the 'rescue: not found' errors from happening.
Bump FreeBSD_version to 1100078 since there's been no version bumps
since this change was made. Only people that installed since r284356
really need to do this bootstrapping, but since crunchgen needs to
bootstrap for other reasons, bumping the number was the simplest.
They need to be built and installed (including headers) prior to the
DTrace CTF tools.
Reviewed by: imp (as part of a larger change)
Sponsored by: The FreeBSD Foundation
