This takes longer but should reliably produce working binaries.
The old version linked against system libraries and headers
which would be a problem if building a native-xtools against
a newer source than the host system. With a proper sysroot made
first this is not a problem.
This also allows:
- NXBDIRS to be built in parallel
- NXBDIRS to be installed to NXBDESTDIR in parallel
- SYSTEM_COMPILER logic to work again. Note that because this change
is adding a sysroot phase the compiler may be built up to twice now.
The first is the "cross-compiler" even though it is for the native
architecture, but it is still built to ensure the latest compiler
is used to generate the binaries, unless SYSTEM_COMPILER allows
/usr/bin/cc to be used. Then the target compiler is built
which is actually a cross-compiler since it runs on native host
but generates TARGET.TARGET_ARCH binaries.
Note this also changes the path used for the OBJDIR. It used to use
/usr/obj/target.target_arch/nxb/<srcdir> for objects and
/usr/obj/target.target_arch/nxb-bin for installed files, but now uses
/usr/obj/nxb/target.target_arch/<srcdir> for objects and
/usr/obj/nxb/target.target_arch/<srcdir>/nxb-bin for installed files.
- NXBDESTDIR can be specified for where to install or queried with
`make -f Makefile.inc1 TARGET=... TARGET_ARCH=... -V NXBDESTDIR`
This could potentially be improved to reuse an existing sysroot. The
problem is with building the SUBDIR_OVERRIDE list it needs to use a
different OBJDIR since it is building all statically. We don't want to
pollute the existing 'buildworld' OBJDIR and cause confusion on the next
build. Using a separate OBJDIR for the 'everything' phase mostly works
except for some things like linking in INTERNALLIBS that exist in the
other OBJDIR.
MFC after: 1 month
Sponsored by: Dell EMC Isilon
Make armv7 as a new MACHINE_ARCH.
Copy all the places we do armv6 and add armv7 as basically an
alias. clang appears to generate code for armv7 by default. armv7 hard
float isn't supported by the the in-tree gcc, so it hasn't been
updated to have a new default.
Support armv7 as a new valid MACHINE_ARCH (and by extension
TARGET_ARCH).
Add armv7 to the universe build.
Differential Revision: https://reviews.freebsd.org/D12010
files will produce an error and buildkernel will fail. Previously missing
KERNCONF files silently failed giving no indication as to why, only to
subsequently discover during installkernel that the desired kernel was
never built in the first place.
This is a rework of r302865.
This is the correct patch.
Reviewed by: ngie (previous version, r302865)
MFC after: 2 months
Differential Revision: D7167
files will produce an error and buildkernel will fail. Previously missing
KERNCONF files silently failed giving no indication as to why, only to
subsequently discover during installkernel that the desired kernel was
never built in the first place.
This is a rework of r302865.
Reviewed by: ngie (previous version, r302865)
MFC after: 2 months
Differential Revision: D7167
This causes builds to use the in-tree libdwarf with the bug fix instead
of the host library.
Reviewed by: bdrewery, emaste, markj
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D12239
This helps reduce the WORLDTMP footprint slightly.
Based on a patch I submitted 5 years ago to GNATS.
PR: 174051
MFC after: 2 weeks
Relnotes: yes (anyone who cross-builds with MK_TCSH=yes will run into
build failures if the host doesn't have tcsh(1))
Reminded by: Fabian Keil <fk@fabiankeil.de>
LLD_BOOTSTRAP is intended to control the linker used to link world and
kernel, while LLD_IS_LD is intended to control the linker installed in
that world.
Force LLD_IS_LD equal to LLD_BOOTSTRAP for the cross-tools build and
install phase, so that lld will be installed as the ld to run on the
host, when LLD_BOOTSTRAP is set.
PR: 221543
Reviewed by: dim
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D12072
Disconnect the dependency on the kernel package from the runtime
package. There are a number of problems here:
1) The runtime package installed into a chroot or a jail would
include the kernel package, changing the behavior of how jails
work now [1];
2) As result of (1), it is possible a binary may incorrectly
resolve kernel symbols [2]; in addition, it is possible there
will be unexpected fallout with 32-bit jails on a 64-bit host
kernel [2].
Noticed by: brd [1]
Discussed with: kib [2]
MFC after: 3 days
MFC note: record-only to wipe from the merge tracker
Sponsored by: The FreeBSD Foundation
svnversion metadata to the runtime and kernel packages.
Instead of traversing src/sys, as is done by newvers.sh for uname(1),
a full tree walk is done to prevent userland and/or modifications
from not being reflected in a modified tree (M).
MFC after: 5 days
Sponsored by: The FreeBSD Foundation
The idea here is that, provided upstream pkg(8) maintainers accept
the proposed change, the kernel.ucl will contain a post-install
script causing pkg(8) to emit a message informing to reboot the
system after the kernel is upgraded using 'pkg upgrade', so the
new userland is installed on the running new kernel. At present,
this functionality does not exist in pkg(8), but will help ensure
the upgrade path follows that from UPDATING. To work around this
for now, evaluate ASSUME_ALWAYS_YES, and prompt the user if they
wish to proceed if not set to true.
Since there is a kernel dependency, and a non-GENERIC kernel may
be in use, update Makefile.inc1 to replace '%KERNCONF%' in the
runtime.ucl with the first-built kernel set either via command line
or in make.conf(5).
MFC after: 5 days
Sponsored by: The FreeBSD Foundation
arches), and handle two more cases where libc++ includes could be
incorrectly enabled, in case the host compiler is clang 5.0.0, and the
target (cross) compiler is gcc 4.2.1.
Noted by: bdrewery
MFC after: 3 days
X-MFC-With: 321684
- bsd.compiler.mk: Must ensure that the CCACHE_WRAPPER_PATH comes first
in PATH.
- Makefile.inc1: Must prepend the CCACHE_WRAPPER_PATH into BPATH as it
overrides the PATH set in bsd.compiler.mk in sub-makes. The PATH
set in bsd.compiler.mk is not exported and doing so would cause it to
then override the BPATH set from environment. The only sane solution
is to prepend into BPATH as needed.
CCACHE_PATH could possibly be used for some of this as well.
Sponsored by: Dell EMC Isilon
Since we imported clang 5.0.0, the version check in Makefile.inc1 which
checks whether to use libc++ fires even when the compiler for the target
architecture is gcc 4.2.1. This is because only X_COMPILER_VERSION is
checked. Also check X_COMPILER_TYPE, so it will only use libc++ when an
external gcc toolchain is used.
Reviewed by: emaste, rpokala
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D11776
It is full of distracting noise about UPDATING and may confuse
the user about what is actually being deleted. It is also
possible to have directories removed on every run with
use of WITHOUT_ knobs that the mtree files do not
account for and for which the directories are incorrectly
in OLD_DIRS currently.
X-MFC-With: r321443
MFC after: 1 month
Sponsored by: Dell EMC Isilon
This dependency does nothing since cddl/lib/libzfs is never
added into the 'make libraries' dependency chain
directly.
MFC after: 2 weeks
Sponsored by: Dell EMC Isilon
This prevents situations with -DNO_CLEAN from finding stale headers or
libraries in places that no longer exist or have moved. It avoids
the need to remove all of WORLDTMP by reusing what we already know
is obsolete.
MFC after: 1 month
Sponsored by: Dell EMC Isilon
The files are only ever generated to .OBJDIR, not to WORLDTMP (as a
sysroot) and are only ever included from a compilation. So using
a beforebuild target here removes the file before the compilation
tries to include it.
MFC after: 2 months
X-MFC-With: r321369
clang 4.0.0. Otherwise, these can get included before the two newly
generated ones (which are different) for clang 5.0.0.
Reported by: Mark Millard
MFC after: 2 months
X-MFC-With: r321369
In a scenario of cross-building it is possible that an OBJDIR's WORLDTMP
contains an older compiler in WORLDTMP/usr/bin/cc that is not rebuilt
if SYSTEM_COMPILER logic is triggered. This compiler was still
incorrectly used. Address this by removing WORLDTMP/usr/bin/cc and all
of the hardlinked files associated with it. Also do this for c++ for
GCC builds.
Sponsored by: Dell EMC Isilon
MFC after: 1 week
Our current approach to dependency tracking cannot cope with switching
generated asm syscall stubs into C wrappers. Perpetuate the hack in
Makefile.inc1 to paper over the problem until we can take a holistic
approach to fixing dependency problems.
Differential Revision: https://reviews.freebsd.org/D11344
The -B was originally added in projects/release-pkg r289381 as a copy
of what 'make world' did at the time. The -B was removed from
the 'installworld' call in 'world' in r303844 though. The staging
of files is safe to run in parallel.
MFC after: 2 weeks
Sponsored by: Dell EMC Isilon
This fixes LD errors during 'make packages' but also for the unlikely case of
'buildworld' on 1 system and 'packages' on another [1].
PR: 212877 [1]
MFC after: 2 weeks
Sponsored by: Dell EMC Isilon
This is useful for having directories behave differently depending
on the phase - such as enabling SUBDIR_PARALLEL or disabling
redundant building of library directories already done by
earlier 'make _libraries'.
Sponsored by: Dell EMC Isilon
This is to allow downstream Makefiles to know for sure they are building
against a sysroot rather than only depending on ${DESTDIR} or other
assumptions.
This also exports it into buildenv.
MFC after: 2 weeks
Sponsored by: Dell EMC Isilon
Use a similar approach to r318957 (which was for ptrace dependencies):
grep the .depend file for the old source file and delete the stale
dependency if found.
Reviewed by: bdrewery
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D11091
When makeman is generating src.conf(5) it tries to test all variation of options
including WITH_DIRDEPS_BUILD. it results in an error when filemon(4) is not
loaded.
Export variables that are needed to prevent this behaviour.
Helped by: sjg
Normally META_MODE ignores host files for "meta mode" decisions on whether a
file should be rebuilt or not. This is because a simple installworld can
update timestamps and cause the next build to rebuild all host tools, when the
previous ones may not have any changes in the source tree. These tools are
normally still ABI compatible. They are only rebuilt if NO_META_IGNORE_HOST is
set from the workaround/hack in r301467.
One of the major problems with this is when a host tool has objects spread
across many revisions that have mixed-ABI. For example, if struct stat were to
change on the host, some objects for a tool may have different ideas of that
struct's definition. If just 1 source file were modified and rebuilt and
linked into the tool, then that toll will have mixed-ABI objects and crash.
This exact thing happened with the ino64 commit in r301467 followed by a
trivial update to libbfd in r318750. The resulting binary would crash in
buildworld.
Sponsored by: Dell EMC Isilon
This will ensure that aarch64 gets a working native /usr/bin/ld rather
than requiring the aarch64-binutils hack in Poudriere, or emulating
the aarch64 lld.
PR: 217189
Reported by: swills, jbeich
FreeBSD does not guarantee kernel forward compatibility (that is,
running a newer userland on an older kernel). The documented upgrade
procedure specifies that installkernel should be performed, followed by
a reboot and then installworld. As a sanity check when installing onto
the running system (DESTDIR is / or unset), attempt to run "sh echo OK"
using rescue from the objdir. If rescue fails (e.g., because the system
has not been rebooted and the "old" kernel lacks a system call required
by the to-be-installed world), abort the installation.
This should avoid ino64 foot-shooting when the proper upgrade procedure
is not followed.
Reviewed by: allanjude, gjb, kib
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D10987
This is similar to r318912, except that ptrace.[sS] was previously a
file in the source tree, not a generated assembly wrapper.
Check for the existence of ptrace.[sS] in the .depend file to determine
if we have to clean it up. This is a bit hackish and will not be left
in place indefinitely, but provides a useful example case when
investigating a better solution in bmake.
Reviewed by: bdrewery
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D10930
This is an attempt to help -DNO_CLEAN builds after r302092 (which
removed the pipe libc syscall wrapper) and r318736 (which removed
getdents, lstat, mknod, and stat).
Dependencies cannot cope with certain source tree changes,
particularly with respect to removing source files and replacing
generated files. Handle these cases from _worldtmp in an ad-hoc
fashion.
Reviewed by: bdrewery, cem
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D10876
This allows for building the world against the already-created
host/sysroot environment. It is not overly useful outside of cases of
large-impact changes such as a testing a new compiler. It will
allow quickly getting back to an error in the target-phases of the
build where a new compiler is being used.
Sponsored by: Dell EMC Isilon
The closing paren for the list of architectures that should enable LIB32
by default was in the wrong place resulting in LIB32 always be enabled on
mips64 regardless of WITH_LIB32/WITHOUT_LIB32.
Submitted by: Alex Richardson <Alexander.Richardson@cl.cam.ac.uk>
Obtained from: CheriBSD
Sponsored by: DARPA / AFRL
Right after cross-tools, a compiler-metadata.mk file is created that
stores all of the bsd.compiler.mk metadata. It is then read in
with a fail-safe during installworld time.
The file is explicitly removed when invoking cross-tools to ensure that
a stale file is not left around from odd manual 'make _cross-tools' ->
'make installworld' invocations.
This fixes several issues:
- With WITH_SYSTEM_COMPILER (default yes on head and no on releng/11.0):
If you build on a system where the bootstrap compiler does not
build due to the host compiler matching the in-tree one, but then
installworld on another system where that logic fails (a
bootstrap compiler is needed), the installworld immediately fails
with:
sh: cc: not found
Note that fixing this logic may then hit a case where a rebuild is
attempted in installworld. Normally cc would be ran with
'CFLAGS+=ERROR-tried-to-rebuild-during-make-install' to cause an
error such as:
cc: error: no such file or directory: 'ERROR-tried-to-rebuild-during-make-install'
However, now it will just fail with the 'cc: not found' error.
Inspection of the compile line will show
'ERROR-tried-to-rebuild-during-make-install'; It's not useful to
set CC to anything other than 'cc' during install as it is more
helpful to see the attempted compile rather than some other bogus
error.
- This now avoids running bsd.compiler.mk (cc executions) even more
during installworld. There are compiler-dependent SUBDIR in the
tree which required having a compiler during install.
There is at least 1 case where CC is still executed in the install,
such as from a LOOKUP!= in secure/lib/libcrypto/Makefile.inc checking
for 'vzeroall' support. This is not significant for installworld
as the lookup has a fallback (and hides its error) and only modifies CFLAGS,
thus it's not worth fixing.
PR: 212877
MFC after: 2 weeks
Sponsored by: Dell EMC Isilon
Add WITH_LLD_BOOTSTRAP and WITHOUT_LLD_BOOTSTRAP knobs, similar to the
Clang bootstrap knobs.
Reviewed by: dim
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D10249
r279908 added logic to Makefile.inc1 to automatically set
CROSS_BINUTILS_PREFIX for architectures not supported by the in-tree
binutils: arm64 when first introduced, and later riscv64 as well.
LLVM's LLD linker is now included in the base system, and is enabled by
default for arm64 and capable of linking world and kernel. Thus, avoid
automatically setting CROSS_BINUTILS_PREFIX and requiring the binutils
port if WITH_LLD_IS_LD is true.
Reviewed by: kan
Relnotes: Yes
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D10310
In a cross-build, the build-tools are native host binaries. We do not
want to rebuild them when building for the target. Bmake previously
did not support checking .NOMETA on an existing target, so .NOMETA_CMP
was used here. However, .NOMETA_CMP still triggers meta mode conditions
if the number of commands or the command changes. In r312467 the paths
to build ncurses files were modified and thus triggered meta mode to
rebuild the build tools (make_keys, make_hash) in ncurses during the
target build. Bmake 20160604 committed in r301462 changed .NOMETA to
also skip meta mode logic for an existing .meta file as well, thus it
is now the proper fix here.
I explored moving the build-tools output to WORLDTMP/tools with
relatively good success, but have concerns that doing so would be
problematic for downstream vendors who use LOCAL_TOOL_DIRS and
expect the tools to be in current OBJDIR for the target. It also
adds more complexity into finding the tools during target build
and handling of where they are for rescue/rescue and
mkcsmapper_static/mkesdb_static which should really not be connected in
build-tools anyway.
MFC after: 2 weeks
Reported by: many
Sponsored by: Dell EMC Isilon
r313404 made libnetbsd require sha384.h from libmd. Libmd added it in
r292782. Update BOOTSTRAPPING to account for this.
Reported by: bde
Reviewed by: ngie
Some of the tests in devel/atf // devel/kyua rely on the tools being in $PATH,
which means that the tests fail when run via "make checkworld" because $PATH
is restricted to exclude directory elements like "${LOCALBASE}/bin".
MFC after: 1 week
Sponsored by: Dell EMC Isilon
The xdev build needed the same fixes as libcompat and external toolchain
support needed for handling of --sysroot, -L, -B, etc.
MFC after: 2 weeks
Sponsored by: Dell EMC Isilon
The case for which this was added, r274807, causes this warning to
always show. LOCAL_DIRS=foo LOCAL_LIB_DIRS=foo/lib. The only case in
which r274807 is a problem is if foo/Makefile does not contain
SUBDIR+=lib, which is a normal convention. LOCAL_LIB_DIRS is a special
hack only to get a library into the _generic_libs list for the
'make libraries' bootstrapping phase. The old behavior changed in
r274807 was only in head during the 10.0 cycle, so the warning was
only ever needed until release anyhow.
Reported by: ngie
MFC after: 1 week
Sponsored by: Dell EMC Isilon
The switch to elftoolchain's readelf in r280859 caused native-xtools
to no longer build readelf. This fixes poudriere builds not using
a native readelf when expected.
Reported by: strejda on freenode
MFC after: 2 weeks
Sponsored by: Dell EMC Isilon
Previously WITH_LLD_AS_LD installed LLD as /usr/bin/ld in the target
system, but still used the GNU BFD ld to link the binaries in that
target. LLD 4.0.0 can link the FreeBSD/amd64 world and kernel so use
LLD as the build-time linker as well when the knob is set.
Reviewed by: dim
Relnotes: Yes
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D9226
Note that mandoc does not use anymore sqlite3 but a home made database format
An important improvement has been made as well in makewhatis performance:
Tests on my laptop shows makewhatis on the entire system goes from 26s to 12s
Build and install an o32 set of libraries on mips64 suitable for
running o32 binaries via COMPAT_FREEBSD32. Enable COMPAT_FREEBSD32 in
MALTA64.
Reviewed by: jmallett, imp
Sponsored by: DARPA / AFRL
Differential Revision: https://reviews.freebsd.org/D9032
MK_KERBEROS_SUPPORT != no
This fixes the odd case where someone specified MK_GSSAPI=no and
MK_KERBEROS_SUPPORT=yes (which admittedly, probably doesn't make sense,
but the build system doesn't prevent this case today, and it didn't when
I filed the bug back in 2011 either).
MFC after: 2 weeks
PR: 159745
During the clang/llvm 3.9.0 import, the build structure for it was
completely revamped. This broke the native-xtools target.
It first attempts to build libllvmminimal, then the llvm-tblgen and
clang-tblgen executables, but these fail to link because they are linked
to the 'full' libllvm by default, as they normally are during the
'world' stage.
To make these link against libllvmminimal instead, define TOOLS_PREFIX,
similarly as during the bootstrap-tools phase. The value itself is
empty, as we don't really want to use a prefix.
Reviewed by: imp
PR: 215684
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D9026
being a bootstrap tool. However, for reproducible build output,
FreeBSD added dd status=none because it was otherwise difficult to
suppress the status information, but retain any errors that might
happen. There's no real reason that dd has to be a build tool, other
than we use status=none unconditional. Remove dd from a bootstrap tool
entirely by only using status=none when available. This may also help
efforts to build the system on non-FreeBSD hosts as well.
Differential Revision: https://reviews.freebsd.org/D8605
Any .debug or .symbols files under /usr/lib/debug which correspond to
OLD_FILES entries in ObsoleteFiles.inc are also automatically cleaned up
by the delete-old target. Make this also apply to any OLD_DIRS entries.
Reviewed by: emaste
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D8683
to create a repo during 'make packages'
This would have been useful for a situation I found myself in where
pkg(8) had been upgraded to a version that wanted the FBSD_1.5 ABI
version but libc.so.7 had not been upgraded, and only provided
FBSD_1.4. I found I needed to update libc in order to run pkg, and I
also needed to use pkg to update libc... Which is why pkg-static
exists, but there's currently no way to tell the build system to use
pkg-static instead of pkg.
This creates a variable PKG_CMD, default value 'pkg', that can be
overridden from the command line.
Reviewed by: gjb
Approved by: gjb
Differential Revision: https://reviews.freebsd.org/D8120
If set it installs LLD as /usr/bin/ld. LLD (as of version 3.9) is not
capable of linking the world and kernel, but can self-host and link many
substantial applications. GNU ld continues to be used for the world and
kernel build, regardless of how this knob is set.
It is on by default for arm64, and off for all other CPU architectures.
Sponsored by: The FreeBSD Foundation
