ccache is mostly beneficial for frequent builds where -DNO_CLEAN is not
used to achieve a safe pseudo-incremental build. This is explained in
more detail upstream [1] [2]. It incurs about a 20%-28% hit to populate the
cache, but with a full cache saves 30-50% in build times. When combined with
the WITH_FAST_DEPEND feature it saves up to 65% since ccache does cache the
resulting dependency file, which it does not do when using mkdep(1)/'CC
-E'. Stats are provided at the end of this message.
This removes the need to modify /etc/make.conf with the CC:= and CXX:=
lines which conflicted with external compiler support [3] (causing the
bootstrap compiler to not be built which lead to obscure failures [4]),
incorrectly invoked ccache in various stages, required CCACHE_CPP2 to avoid
Clang errors with parenthesis, and did not work with META_MODE.
The option name was picked to match the existing option in ports. This
feature is available for both in-src and out-of-src builds that use
/usr/share/mk.
Linking, assembly compiles, and pre-processing avoid using ccache since it is
only overhead. ccache does nothing special in these modes, although there is
no harm in calling it for them.
CCACHE_COMPILERCHECK is set to 'content' when using the in-tree bootstrap
compiler to hash the content of the compiler binary to determine if it
should be a cache miss. For external compilers the 'mtime' option is used
as it is more efficient and likely to be correct. Future work may optimize the
'content' check using the same checks as whether a bootstrap compiler is needed
to be built.
The CCACHE_CPP2 pessimization is currently default in our devel/ccache
port due to Clang requiring it. Clang's -Wparentheses-equality,
-Wtautological-compare, and -Wself-assign warnings do not mix well with
compiling already-pre-processed code that may have expanded macros that
trigger the warnings. GCC has so far not had this issue so it is allowed to
disable the CCACHE_CPP2 default in our port.
Sharing a cache between multiple checkouts, or systems, is explained in
the ccache manual. Sharing a cache over NFS would likely not be worth
it, but syncing cache directories between systems may be useful for an
organization. There is also a memcached backend available [5]. Due to using
an object directory outside of the source directory though you will need to
ensure that both are in the same prefix and all users use the same layout. A
possible working layout is as follows:
Source: /some/prefix/src1
Source: /some/prefix/src2
Source: /some/prefix/src3
Objdir: /some/prefix/obj
Environment: CCACHE_BASEDIR='${SRCTOP:H}' MAKEOBJDIRPREFIX='${SRCTOP:H}/obj'
This will use src*/../obj as the MAKEOBJDIRPREFIX and tells ccache to replace
all absolute paths to be relative. Using something like this is required due
to -I and -o flags containing both SRC and OBJDIR absolute paths that ccache
adds into its hash for the object without CCACHE_BASEDIR.
distcc can be hooked into by setting CCACHE_PREFIX=/usr/local/bin/distcc.
I have not personally tested this and assume it will not mix well with
using the bootstrap compiler.
The cache from buildworld can be reused in a subdir by first running
'make buildenv' (from r290424).
Note that the cache is currently different depending on whether -j is
used or not due to ccache enabling -fdiagnostics-color automatically if
stderr is a TTY, which bmake only does if not using -j.
The system I used for testing was:
WITNESS
Build options: -j20 WITH_LLDB=yes WITH_DEBUG_FILES=yes WITH_CCACHE_BUILD=yes
DISK: ZFS 3-way mirror with very slow disks using SSD l2arc/log.
The arc was fully populated with src tree files and ccache objects.
RAM: 76GiB
CPU: Intel(R) Xeon(R) CPU L5520 @2.27GHz
2 package(s) x 4 core(s) x 2 SMT threads = hw.ncpu=16
The WITH_FAST_DEPEND feature was used for comparison here as well to show
the dramatic time savings with a full cache.
buildworld:
x buildworld-before
+ buildworld-ccache-empty
* buildworld-ccache-full
% buildworld-ccache-full-fastdep
# buildworld-fastdep
+-------------------------------------------------------------------------------+
|% * # +|
|% * # +|
|% * # xxx +|
| |A |
| A|
| A |
|A |
| A |
+-------------------------------------------------------------------------------+
N Min Max Median Avg Stddev
x 3 3744.13 3794.31 3752.25 3763.5633 26.935139
+ 3 4519 4525.04 4520.73 4521.59 3.1104823
Difference at 95.0% confidence
758.027 +/- 43.4565
20.1412% +/- 1.15466%
(Student's t, pooled s = 19.1726)
* 3 1823.08 1827.2 1825.62 1825.3 2.0785572
Difference at 95.0% confidence
-1938.26 +/- 43.298
-51.5007% +/- 1.15045%
(Student's t, pooled s = 19.1026)
% 3 1266.96 1279.37 1270.47 1272.2667 6.3971113
Difference at 95.0% confidence
-2491.3 +/- 44.3704
-66.1952% +/- 1.17895%
(Student's t, pooled s = 19.5758)
# 3 3153.34 3155.16 3154.2 3154.2333 0.91045776
Difference at 95.0% confidence
-609.33 +/- 43.1943
-16.1902% +/- 1.1477%
(Student's t, pooled s = 19.0569)
buildkernel:
x buildkernel-before
+ buildkernel-ccache-empty
* buildkernel-ccache-empty-fastdep
% buildkernel-ccache-full
# buildkernel-ccache-full-fastdep
@ buildkernel-fastdep
+-------------------------------------------------------------------------------+
|# @ % * |
|# @ % * x + |
|# @ % * xx ++|
| MA |
| MA|
| A |
| A |
|A |
| A |
+-------------------------------------------------------------------------------+
N Min Max Median Avg Stddev
x 3 571.57 573.94 571.79 572.43333 1.3094401
+ 3 727.97 731.91 728.06 729.31333 2.2492295
Difference at 95.0% confidence
156.88 +/- 4.17129
27.4058% +/- 0.728695%
(Student's t, pooled s = 1.84034)
* 3 527.1 528.29 528.08 527.82333 0.63516402
Difference at 95.0% confidence
-44.61 +/- 2.33254
-7.79305% +/- 0.407478%
(Student's t, pooled s = 1.02909)
% 3 400.4 401.05 400.62 400.69 0.3306055
Difference at 95.0% confidence
-171.743 +/- 2.16453
-30.0023% +/- 0.378128%
(Student's t, pooled s = 0.954969)
# 3 201.94 203.34 202.28 202.52 0.73020545
Difference at 95.0% confidence
-369.913 +/- 2.40293
-64.6212% +/- 0.419774%
(Student's t, pooled s = 1.06015)
@ 3 369.12 370.57 369.3 369.66333 0.79033748
Difference at 95.0% confidence
-202.77 +/- 2.45131
-35.4225% +/- 0.428227%
(Student's t, pooled s = 1.0815)
[1] https://ccache.samba.org/performance.html
[2] http://www.mail-archive.com/ccache@lists.samba.org/msg00576.html
[3] https://reviews.freebsd.org/D3484
[5] https://github.com/jrosdahl/ccache/pull/30
PR: 182944 [4]
MFC after: 3 weeks
Sponsored by: EMC / Isilon Storage Division
Relnotes: yes
Bmake has a documented feature of '-N' to skip executing commands which is
specifically intended for debugging top-level builds and not recursing into
sub-directories. This matches the older 'make -n' behavior we added which made
'-n -n' the recursing target and '-n' a non-recursing target.
Removing the '-n -n' feature allows the build to work as documented in
the bmake manpage with '-n' and '-N'. The older '-n -n' feature was also
not documented anywhere that I could see.
Note that the ${_+_} var is still needed as currently bmake incorrectly
executes '+' commands when '-N' is specified.
The '-n' and '-n -n' features were broken for several reasons prior to this.
r251748 made '_+_' never expand with '-n -n' which resulted in many
sub-directories not being visited until fixed 2 years later in r288391, and
many targets were given .MAKE over the past few years which resulted in
non-sub-make commands, such as rm and ln and mtree, to be executed.
This should also allow removing some indirection hacks in bsd.subdir.mk and
other cases of .USE that have a .MAKE by using '+'.
Sponsored by: EMC / Isilon Storage Division
Discussed on: arch@ (mostly silence)
The condition used matches the condition in sys.mk for setting _+_ to blank
or +.
With this -n will continue to not descend into Makefile.inc1, while -n -n will
and cause Makefile.inc1's target to run with -n.
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
This is relevant for makeman using the 'make targets' output in src.conf(5).
This makes a _UNIVERSE_TARGETS that removes arm64 if the build
requirements are not met.
bmake specific constructs not needed for make bootstrap so fmake
doesn't see them. This works with fmake just well enough for us to
build bmake to build the rest of the tree without fatal errors. Tested
only with fmake package.
arm64 relies on an external binutils port or package right now, because
the in-tree linker from binutils 2.17.50 does not support arm64. Add
arm64 to universe if the linker is available. If not output a message
that arm64 is skipped.
buildworld and buildkernel use the external binutils automatically, so
it's sufficient to run 'pkg install aarch64-binutils' to build
FreeBSD/arm64.
Differential Revision: https://reviews.freebsd.org/D2302
Reviewed by: andrew, imp
Sponsored by: The FreeBSD Foundation
only adds support for kernel-toolchain, however it is expected further
changes to add kernel and userland support will be committed as they are
reviewed.
As our copy of binutils is too old the devel/aarch64-binutils port needs
to be installed to pull in a linker.
To build either TARGET needs to be set to arm64, or TARGET_ARCH set to
aarch64. The latter is set so uname -p will return aarch64 as existing
third party software expects this.
Differential Revision: https://reviews.freebsd.org/D2005
Relnotes: Yes
Sponsored by: The FreeBSD Foundation
Dynamically figure out the list of targets based on tags passed on the mtrees
First sanity check that all packages have existing manifests
Generate the packages
Please note that for now the mtree needs more work as it has duplicate entries,
everything is not yet tagged
The packages now have generic entries and needs to be customize
building toolchain for the host computer. This toolchain produces
TARGET_ARCH and assumes the rest of the system contains libraries for
the target. It is intended to be used in a "qemu-user jail" where all
the binaries would otherwise be the target architecture's to build
ports. However, emulation of the compilers is too slow, so we build
native binaries for that. Rather than use the xdev produced binaries,
with all their weird links and paths, these binaries use the native
paths. They will not work unless installed into the qemu-user jail.
Differential Revision: https://phabric.freebsd.org/D518
Reviewed by: sbruno@
During "make buildworld", building bmake is (one of) the very first steps
and we should not be building any of its tests. Conceptually, this is the
right thing to do 1) for build simplicity reasons and 2) because there is
no need to build any tests this early on.
In practice, this fixes tinderbox builds of CURRENT from 9.x when MK_TESTS
is enabled. This is because bsd.test.mk needs some modern bmake features
not present in 9.x (:tW) and tinderbox is forcing the build to use the
CURRENT share/mk files from the very beginning (see r266617). By skipping
the build of the tests when still using the host make, we omit the problem.
Arguably, what tinderbox is doing is wrong and needs to be addressed, but
that is a separate issue.
UPDATING. This is the first step towards the removal of ia64 from
head. A buildworld for ia64 will now yield:
% make buildworld
make[1]: "/usr/src/Makefile.inc1" line 151: Unknown target ia64:ia64.
While here, trim the ia64-specific additions from ObsoleteFiles.inc
Discussed at: BSDcan
in them. This is often the case, so just ignore the return
code. Actual errors that are found will also be detected downstream in
the rare cases where the return code is 2 instead of 1.
environments (that I can't reproduce locally, but that others have
reported) seem to get tripped up by this man page install. There's
really no need to do it, so turn off the man pages using the most
portable method. We can't just directly set MK_MAN=no here because
we're bootstrapping in the host environment and such a setting was
forbidden until very recently. NO_MAN= can produce a warning, but for
now the warning is benign.
build world, so it is the only make we build or install. fmake is
still in the tree, but disconnected, and upgrades from older systems
that still have bmake has not been removed, but its state has not been
tested (it should work given how minimal the work to upgrade to bmake
is).
#NO_UNIVERSE. Many of these config files are important examples, but
add little to no regresive value to the intended purpose of
UNIVERSE. We now build over 120 kernels during universe. There's
really little to no value to this over building say 60 or even 30 of
them (either is still a way too big number). This is especially true
for kernels that are nothing more than including a common base and
adding a static DTB file. Start by pruning 1/3 of the arm kernels that
add little regresion value.
commit 1d1b908107255ffdff4d17f015d8f057d73cc6cb
Author: Brooks Davis <brooks@one-eyed-alien.net>
Date: Fri Mar 28 16:24:45 2014 +0000
Add a long needed seatbelt.
Exit with an error when make is called without a target at the top level
rather than poluting the source tree and causing use confusion in future
builds.
commit a9d9aa341b2f4308a227ab460ba85f1f287ad028
Author: Brooks Davis <brooks@one-eyed-alien.net>
Date: Tue Apr 29 16:06:12 2014 +0000
Simplify seatbelt added in 1d1b908 based in feedback.
Discussed with: imp@FreeBSD.org
Reviewed by: imp
Sponsored by: DARPA, AFRL
This targets the existing ARMv6 and ARMv7 SoCs that contain a VFP unit.
This is an optional coprocessors may not be present in all devices, however
it appears to be in all current SoCs we support.
armv6hf targets the VFP variant of the ARM EABI and our copy of gcc is too
old to support this. Because of this there are a number of WITH/WITHOUT
options that are unsupported and must be left as the default value. The
options and their required value are:
* WITH_ARM_EABI
* WITHOUT_GCC
* WITHOUT_GNUCXX
In addition, without an external toolchain, the following need to be left
as their default:
* WITH_CLANG
* WITH_CLANG_IS_CC
As there is a different method of passing float and double values to
functions the ABI is incompatible with existing armv6 binaries. To use
this a full rebuild of world is required. Because no floating point values
are passed into the kernel an armv6 kernel with VFP enabled will work with
an armv6hf userland and vice versa.
broken. None of our kernels can boot armv6eb. The little-endian kernels do
not have the required code to be able to switch endian when running a
big-endian executable.
Approved by: re (gjb)
Include PROGNAME and DESTDIR in ${MMAKE} so that it doesn't need to be
passed to each make invocation.
Suggested by: hrs
Reviewed by: hrs
Approved by: re (gjb)
A HEAD buildworld on 9.x first bootstraps bmake, but this failed when
building with standalone debug. Pass in the PROGNAME override to the
'make all' stage as well as 'make install' so that the .debug file is
created with the correct name.
Reviewed by: sjg
Sponsored by: The FreeBSD Foundation
Approved by: re (rodrigc)
it gets built 16 times in parallel in the same location.
While we are at it, until we finish getting rid of fmake,
be explicit about the make we want to use, thus avoid the problem
of the temp make being the wrong version.
Reviewed by: obrien
so that job token pipe is passed to them.
To avoid surprising anyone, only add .MAKE to ${TGTS} when -n
has not been specified (at least for Makefile).
Reviewed by: obrien
make before starting the universe targets themselves. Otherwise, all of
the targets would attempt to build make simultaneously, overwriting each
other's copies of the make object files and executable. This could lead
to strange errors, for example when partially-written make executables
are invoked.
Also amend r216620, to make the rest of universe wait properly until the
upgrade_checks target is finished, by adding universe_${target}_prologue
to the .ORDER target. Otherwise, make will be too smart for its own
good, and start building the universe targets simultaneously with the
prologues anyway.
MFC after: 1 week
starting the kernels. Before this the kernels would be built as part of the
last architecture universe target. There can cause problems when this world
finishes before the other worlds as the host compiler may be picked up
rather than the target compiler.
The solution is to add a target to build the universe kernels that depends
on all the world targets finishing. As we may not be building a world only
depend on it when MAKE_JUST_KERNELS is undefined.
The automation can set TARGET_ARCH and TARGET and then make various
top-level targets, including buildLINT and buildkernel (with
KERNCONF=LINT). Previously there was no way to generate the LINT
kernel configuration without having to do something exceptionally
painful.
1. Don't do upgrade_checks when using bmake. As long as we have WITH_BMAKE,
there's a bootstrap complication in ths respect. Avoid it. Make the
necessary changes to have upgrade_checks work wth bmake anyway.
2. Remove the use of -E. It's not needed in our build because we use ?= for
the respective variables, which means that we'll take the environment
value (if any) anyway.
3. Properly declare phony targets as phony as bmake is a lot smarter (and
thus agressive) about build avoidance.
4. Make sure CLEANFILES is complete and use it on .NOPATH. bmake is a lot
smarter about build avoidance and should not find files we generate in
the source tree. We should not have files in the repository we want to
generate, but this is an easier way to cross this hurdle.
5. Have behavior under bmake the same as it is under make with respect to
halting when sub-commands fail. Add "set -e" to compound commands so
that bmake is informed when sub-commands fail.
6. Make sure crunchgen uses the same make as the rest of the build. This
is important when the make utility isn't called make (but bmake for
example).
7. While here, add support for using MAKEOBJDIR to set the object tree
location. It's the second alternative bmake looks for when determining
the actual object directory (= .OBJDIR).
Submitted by: Simon Gerraty <sjg@juniper.net>
Submitted by: John Van Horne <jvanhorne@juniper.net>
is something for make(1) to consume. Bmake gives output such as:
"warning: Couldn't read shell's output for "/bin/sh -c true"
Note we parted from traditional Pmake behavior in r18864 / r18255.
r238211:
Support TARGET_ARCH=armv6 and TARGET_ARCH=armv6eb
This adds a new TARGET_ARCH for building on ARM
processors that support the ARMv6K multiprocessor
extensions. In particular, these processors have
better support for TLS and mutex operations.
This mostly touches a lot of Makefiles to extend
existing patterns for inferring CPUARCH from ARCH.
It also configures:
* GCC to default to arm1176jz-s
* GCC to predefine __FreeBSD_ARCH_armv6__
* gas to default to ARM_ARCH_V6K
* uname -p to return 'armv6'
* make so that MACHINE_ARCH defaults to 'armv6'
It also changes a number of headers to use
the compiler __ARM_ARCH_XXX__ macros to configure
processor-specific support routines.
Submitted by: Tim Kientzle <kientzle@freebsd.org>
This makes our naming scheme more closely match other systems and the
expectations of much third-party software. MIPS builds which are little-endian
should require and exhibit no changes. Big-endian TARGET_ARCHes must be
changed:
From: To:
mipseb mips
mipsn32eb mipsn32
mips64eb mips64
An entry has been added to UPDATING and some foot-shooting protection (complete
with warnings which should become errors in the near future) to the top-level
base system Makefile.
second invocation only needs to operate on files with the immutable flag
set.
Submitted by: arundel (via private email) (original version)
Discussed on: -toolchain@
MFC after: 3 days
doc/, and now www/ trees, but only using the "cvsup" transport.
When "make update" is run using a tree's makefile, it can also use
"cvs" (except for www/) and "svn" (only src/).
Clean up documentation and code regarding "make update":
- Increase oddness by adding support for WWWSUPFILE and NO_WWWUPDATE to
Makefile.inc1 (analogous to PORTSSUPFILE/NO_PORTSUPDATE and
DOCSUPFILE/NO_DOCUPDATE; WWWSUPFILE already supported by www/Makefile).
- Document all trees that support CVS_UPDATE.
- Document all trees that support SUP_UPDATE.
- Document SVN_UPDATE.
- Document NO_WWWUPDATE.
- make.conf(5) mistakenly said that *SUPFILE* had defaults.
- Add an example entry for WWWSUPFILE.
Some files keep the SUN4V tags as a code reference, for the future,
if any rewamped sun4v support wants to be added again.
Reviewed by: marius
Tested by: sbruno
Approved by: re
this happens just before the build is started (within the same second)
CHECK_TIME actually triggers thinking param.h is in the future (see f_Xtime,
c_Xtime logi in find(1) sources for the details in !F_EXACTTIME case).
Using the -mtime -0s (seconds, rather than no unit) avoids this 1s race.
Submitted by: ed (2009-05-03)
Reviewed by: cperciva (2009-05-03), emaste
Tested by: bz (for almost two years)
MFC after: 4 days
infrastructure to use it. make distributeworld can now be used without
preparing its environment first and installs games into its distribution
using the regular make distribute logic instead of post-processing with
a script.
Also add two new targets, packageworld and packagekernel, that tar up the
results of distributeworld and distributekernel (also new), respectively.
then that target is invoked for each architecture rather than the
default action of building world and kernels for each architecture.
- Add a 'make toolchains' wrapper which uses UNIVERSE_TARGET to build
toolchains for all architectures.
- Document JFLAG, MAKE_JUST_KERNELS, and MAKE_JUST_WORLDS variables for
'make universe'.
Reviewed by: bz
MFC after: 1 week
need to do this because variables specified on the command line
override those specified in the Makefile. This is why we also moved
from TARGET to _TARGET in Makefile, and then set TARGET on the command
line when we fork a submake with Makefile.inc1.
This makes mips/mips work again, even without the workaround committed to
lib/libc/Makefile.
Properly document what `make targets` is supposed to list to avoid
further confusion given the place the target sits. Should have happened
with r217125.
Requested by: imp [1]
Reviewed by: rwatson
combinations) by forcing FAILFILE into .CURDIR as we do for all other
universe output files. [1] Similarly make FAILFILE start with "_." as well.
Reviewed by: silence-on-src [1]
MFC after: 6 days
This produces a list of currently supported targets. Here "supported"
means "built in make universe" on the theory that those targets are
more supported than any that might work in 'make buildworld TARGET=x
TARGET_ARCH=y' since the latter are less tested.
Suggested by: rwatson
Implement MACHINE_ARCH=mips64e[lb] to build N64 images. This replaces
MACHINE_ARCH=mipse[lb] TARGET_ABI=n64.
MACHINE_ARCH=mipsn32e[lb] has been added, but currently requires
WITHOUT_CDDL due to atomic issues in libzfs. I've not investigated
this much, but implemented this to preserve as much of the TARGET_ABI
functionality that I could. Since its presence doesn't affect the
working cases, I've kept it in for now.
Added mips64e[lb] to make universe, so more kernels build.
And I think this (finally) closes the curtain on the tbemd tree.
not the one we build as part of make world. This means that make
universe will fail if building on a too-old current or any stable
system prior to a few days ago in weird ways (parse errors from
shell). This copes with these old systems in two ways:
(1) Works around the WARNING: issue by filtering all warnings that
sneak onto stdout.
(2) if TARGET_ARCH_${kernel} winds up being empty, then we error out
immediately with a semi-useful error message. This usually comes from
config not groking -m.
Ideally, we'd use a buildworld's config here, but that's tricky, so
I'll leave that detail to others to fix (it has to be done post make
world for the arch rather than at the top level makefile). This
should make 'make universe' usable from recent 8-stable systems
(recent == last few months or so) for building -current. They have
-m, but spewed warnings out stdout. Older systems will now at least
get a firm error early rather than a confusing error late.
and powerpc64 to universe for the first time. In general, provide
(slightly hacky) knowledge of multi-architecture TARGETs to universe as
well as the ability to distinguish the correct toolchain for a given
kernel using config -m.
semantics for JFLAG with tinderbox as for universe. Previously doing
'make JFLAG=-j4 tinderbox' was equivalent to 'make -j4 universe'
(i.e. 4 worlds in parallel) rather than 'make JFLAG=-j4 universe'
(i.e. worlds in sequence, each built with -j4).
MFC after: 1 month
TARGET_BIG_ENDIAN is now completely dead, except where it was
originally supposed to be used (internally in the toolchain building).
TARGET_ARCH has changed in three cases:
(1) Little endian mips has changed to mipsel.
(2) Big endian mips has changed to mipseb.
(3) Big endian arm has changed to armeb.
Some additional changes are needed to make 'make universe' work on arm
and mips after this change, so those are commented out for now.
UPDATING information will be forthcoming. Any remaining rough edges
will be hammered out in -current.
Remove __gnu89_inline.
Now that we use C99 almost everywhere, just use C99-style in the pmap
code. Since the pmap code is the only consumer of __gnu89_inline, remove
it from cdefs.h as well. Because the flag was only introduced 17 months
ago, I don't expect any problems.
Reviewed by: alc
It was backed out, because it prevented us from building kernels using a
7.x compiler. Now that most people use 8.x, there is nothing that holds
us back. Even if people run 7.x, they should be able to build a kernel
if they run `make kernel-toolchain' or `make buildworld' first.
make universe, split the logic into two parts:
- 1st to build worlds and generate kernel configs like LINT.
- 2nd to build kernels for a given TARGET architecture correctly
finding all newly generated configs, not knowing anything about
LINT anymore. (*)
(*) If you know better/cleaner/... ways to do this, let me know.
Discussed on/with: arch, jhb, rwatson
MFC after: 1 month
you can build the cross development tools and install them as
$XDEV-freebsd-xxx for each tool. This allows one to use autoconf to
find the tools for cross building scenarios.
`make universe'. This catches a few more arm and, once enabled, mips
configs and permits having local configs named like NOINET6.
Reviewed by: phk
MFC after: 4 weeks
a developer can rest reasonably assured that the tinderbox will not
be broken. This target leverages most of 'universe' but will exit
non-zero and output a summary at the end.
"make tinderbox"
no active development on it for over a year now and it isn't
reliable under a simple buildworld. Developers can't be expected to
test code targeted for it.
only matters in the early stages of bootstrapping, of course, but gnu make can't
handle bsd make Makefiles at all if they use any of the 'dot' directives, which
src/Makefile has in abudnance.
and that controls which platforms are being built as part of a "make
universe". By default TARGETS is set to the 8 platforms currently
being built. This variable is useful for running or re-running a
"make universe" with only a selected set of platforms. This makes the
universe target slightly more useful in cases the universe is limited
to a developer's scope or objectives. For example, when a universe
failed for a particular platform and fixes need to be tested for that
particular platform then a developer can restart the universe for
only that platform, even if the initial universe is still building
other platforms.
HISTORICAL_MAKE_WORLD from the text that's output. This was committed
against the previous consensus. Leave the documentation in this file
as a compromose. The HISTORICAL_MAKE_WORLD knob is intentionally
obfuscated and we only trust people smart enough to read the Makefile
to use it. All others have no business using it due to its danger,
unless DESTDIR is set.
Dissentors: grog, obrien, trhodes
undocumented HISTORICAL_MAKE_WORLD variable and set it. Note it
here so the blow up will not really be a surprise to people who
read.
Link the buildingworld chapter of our handbook in the README
while I'm here.
environment for cross building (the same one you'd get interactively
in make buildenv). This cannot be a simple
make -f Makefile.inc1 -V WMAKEENV
because in PATH is not set correctly unless one takes a trip through
the Makefile/Makefile.inc1 indirection, the logic of which is too
large to reproduce outside of Makefiles.
- removes obsolete files/dirs or libraries.
- works in interactive (default) and batch mode
- respects DISTDIR
- documented in UPDATING and build(7)
The head of the file ObsoleteFiles.inc contains instructions how to add
obsolete files/dirs/libs to the list. Obviously one should add obsolete
files to this list, when he removes a file/dir/lib from the basesystem.
Additionally add check-old target:
- allows re@ to check if a file on the obsolete list resurfaces
Design goals:
- allows full control by the user (default interactive mode)
- possibility of scripted removal of obsolete files (batch mode)
- opt-in removal of files (explicit list of files)
- seperate removal of libs (2 delete targets)
Important design decissions:
- structured list of files to remove instead of a plain text file:
* allows to remove additional files if a NO_foo knob is specified
without the need to change the targets (no NO_foo knob is respected
yet)
- not using mtree like NetBSD does:
* mtree doesn't has an interactive mode
Discussed on: arch (long ago), current (this year)
Additional input from: re (hrs)
Approved by: mentor (joerg)
the "make -n universe" output looks more builder (human) friendly.
- Wrap the "universe" target into a ".if make(universe)"; it's only
intended to be called directly so it should be safe to do it.
and adjust the path in the Makefile for the upgrade_checks target.
These checks are really feature upgrade checks that should be fast
and just find out whether we need to build a new make before
proceeding with other targets like buildworld. This makes the
place free for a real regression test suite in the old place.
testing for variables that are always defined (e.g.,
"make -V CC") would still print a false warning. Fix
this by only passing a submake the MAKEOBJDIRPREFIX=*
argument if it was present. As a result, we loose
the check for -DMAKEOBJDIRPREFIX, or an esoteric
"MAKEOBJDIRPREFIX = foo" (with embedded spaces), but
these are unorthodox enough to not care about them.
The make(1) bug mentioned in the previous revision
was just fixed in make/main.c,v 1.109.
When make flags are passed to make in the environment, the string is
chopped up in an (argc,argv) vector. This happens in brk_string() and
the chopped up string is stored in static buffer. When this includes
something like "-V BINMAKE", then a pointer into the static buffer is
put on the variables list for evaluation later. However, brk_string()
is used for more than just chopping up the MAKEFLAGS env. variable, so
it's very likely that the static buffer is clobbered. In fact, this is
exactly what happens.
The result is that _MAKEOBJDIRPREFIX gets assigned whatever garbage
the child make happens to emit, causing the test to fail. Like this:
pluto2% cd /usr/src
pluto2% make -V BINMAKE
"/q/6.x/src/Makefile", line 94: MAKEOBJDIRPREFIX can only be set in environment, not as a global (in /etc/make.conf) or command-line variable.
pluto2% make -dv -V BINMAKE | & grep _MAKEOBJDIRPREFIX
Global:_MAKEOBJDIRPREFIX = }
The fix is to not use MAKEFLAGS for this, but simple pass the flags
as arguments. Ideally make(1) should be fixed but that's beyond the
scope of my attention span.
This fixes release.
by forcing the creation of an object directory for the make regression
tests. Let make handle the tracking of the dependency and installation
of test_shell script.
Submitted by: ru
rates pretty high on the "hack!" scale, but it works for me. Adding
-DWANT_LIB32 to the world build command line, or 'WANT_LIB32=yes' to
/etc/make.conf will include the 32 bit libraries with the build.
I have not made this default behavior. Cross compiling this stuff is an
adventure I have not investigated.
This is still a WIP. We needed this at work so that we could install from
a readonly obj tree - lib32/build.sh wasn't up to that.
built in parallel. Examples:
make universe
Build worlds sequentially, each world sequentially.
make universe JFLAG=-j4
Build worlds sequentially, each world in parallel.
make -j4 universe
make -j4 universe JFLAG=-j2
Build four worlds in parallel, each world will be
built in parallel too. World parallelization is
set to four in the first synopsis, and to two in
the second.
make -j4 universe JFLAG=-B
Build worlds in parallel, each world sequentially.
("world" == buildworld followed by buildkernels.)
Prayers: obrien, phk
two -n flags. If only one -n flag is given the old behaviour
is retained (POLA). In order to make this working for installworld
change the IMAKEENV in this case so that the tools are found
(we have no temporary installation environment in this case).
Submitted by: ru (IMAKEENV part)
trying to upgrade their system with make world instead of following
the preferred and suggested sequence of commands. The fact remains
that make world does not upgrade the kernel.
Allow make world when DESTDIR has been specified, including when
DESTDIR specifies the root file system. Otherwise, print a useful
warning and fail.
Reviewed, tested and scrutinized by: gad@
but the biggest issue is that there are situatons when
${.OBJDIR} == ${.SRCDIR}, and in those situations the previous version
would happily remove all your /usr/src while it was cleaning out the objects.
Not that *you* would be happy about it... Thanks to bde for immediately
noticing this serious possibility.
More improvements will be made to this target, but I wanted to commit this
safer version right now, before anyone lost their /usr/src due to it.
Reviewed by: bde
MFC after: 10 days
that was built in previous 'make buildworld' and 'make buildkernel'. The
target knows enough to run a 'chflags -R 0' and a second 'rm' if the first
'rm' ran into any errors while removing files.
Suggested by: email with Richard Coleman Re: upcoming 64b-time_t changes.
Reviewed by: imp, marcel, and others on -hackers
MFC after: 1 week
Makefile.inc1 underscore targets with a big warning that they are
intentionally undocumented, subject to change without notice and
might poison your dog etc. If you don't know what they are, then they
are not meant for you to use.
I've added these by hand to so many many trees that I've lost count. I
find them rather useful.
buildworld targets by default, but allow it to be done for all user
targets by introducing a boolean option, named ALWAYS_CHECK_MAKE.
This change is by no means perfect and I don't even want to claim
this to be a solution. It does however address the fact that not
everybody likes to see make(1) rebuilt simply because the regression
test failed for some reason or other, including pilot error. It
therefore serves the purpose of keeping the crowd happy until we
have something better or simply reached a compromise.
The reasons for changing the default behaviour are:
o It avoids a negative, possibly non-intuitive option,
o It's according to POLA and fond of feet,
o Only buildworld is documented to do its best to be
successful at reasonably cost.
Reviewed by: gad, imp, obrien, peter
