Since 16fbf01912 PATH is no longer set
to a hardcoded value on non-FreeBSD build hosts, so we can end up with
spaces in $PATH. Instead of only escaping PATH I updated all `env PATH=`
uses in the toplevel makefile. While many of these currently can't
contain any special characters (since the build would have failed
already), in theory this gets us closer to allowing build/source
directory to contain e.g. spaces.
Allow the build process to find host binaries during the host-symlinks target when
cross-building on non-FreeBSD systems. Whilst most non-FreeBSD systems have all
the needed tools in /sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin (the final
path added by host-symlinks itself), Homebrew for macOS on Arm defaults to
/opt/homebrew/bin, other more niche systems may also deviate and users may
expect tools in a customised PATH to be picked up, unlike on FreeBSD where we
want to ensure everything comes from base. In particular, (un)xz are needed
from Homebrew on macOS, and thus cannot be found on Arm without this.
Note that non-FreeBSD builds enforce BUILD_WITH_STRICT_TMPPATH, and so the
actual main build steps will still use a sanitised PATH.
Reviewed by: jrtc27, arichardson
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D37991
Similar to UPDATING and handbook changes, treat Makefile the same.
PR: 260822
Approved by: jhb, philip
Differential Revision: https://reviews.freebsd.org/D36877
With the meta-build, it's always a NO_CLEAN build. Provide a way to
remove so one can rebuild from scratch. 'cleankernel' will delete the
kernel and modules object directories. Document this in build(7).
Sponsored by: Netflix
Reviewed by: debdrup, markj
Differential Revision: https://reviews.freebsd.org/D32978
The text after .error et al is emitted verbatim.
Reviewed by: sjg
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D33904
These targets generate a raw list of the candidate old files roughly
corresponding to the values of OLD_DIRS, OLD_FILES, and OLD_LIBS.
Currently list-old-files also includes uncompressed manpages in
addition to compressed manpages.
Use these targets in the implementation of check-old-* and
delete-old-* to replace duplicated logic.
Reviewed by: imp, emaste
Sponsored by: The University of Cambridge, Google Inc.
Differential Revision: https://reviews.freebsd.org/D33327
Summary: It's currently just as stable as powerpc64, with more ports working.
Reviewers: alfredo, bdragon, luporl, jhibbits, #manpages
Subscribers: imp
Differential Revision: https://reviews.freebsd.org/D33610
A number of header files in sys/* have, going back to 7th Edition Unix
in 1979, reqiured other files (like sys/types.h) to compile. Likewise
the 4BSD networking code has had prerequisites. However, going back to
around the turn of the 21st century, other systems have made them be
independently include-able (wide-spread header include protection
post-dates 7th edition Unix by maybe 3 or so years judging from USENET
source postings). Start down the path of making them all independently
include-able by creating this test that fails buildworld when they are
not.
The file 'badfiles.inc' contains a list of the currently broken files
that cannot be included w/o any prerequisites. As files are fixed, 'make
badfiles.inc' should be re-run to remove them from the list. Note: All
files that start with an underscore are considered internal and not
tested.
Please note: once a file is removed from badfiles.inc, it must pass on
all architectures. Buildworld through at least the _includes target is
needed to ensure its working (though a buildkernel should also be done
on all architectures as well).
Sponsored by: Netflix
Reviewed by: brooks, markj
Differential Revision: https://reviews.freebsd.org/D32498
Unlike the old fmake, running make FOO=bar when using bmake doesn't put
FOO=bar in .MAKEFLAGS at the top level, it instead just puts FOO in
.MAKEOVERRIDES and the full MAKEFLAGS will be formed for sub-makes.
Moreover, this only applies for sub-makes in rules, so this doesn't
apply to those in shell assignments. This means that the current check
does not catch make MAKEOBJDIRPREFIX=..., only those defined in config
files. Thus we must also check .MAKEOVERRIDES explicitly.
Reviewed by: sjg
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D31015
In the CVS days this used be a wrapper around either CVS or CVSup and
used to support updating src, doc, and ports checkouts. With the move
to subversion this only supported updating src and was itself a
wrapper around 'svn update'. With Git, users are probably better off
using appropriate Git commands directly to update without needing an
explicit make target as a wrapper.
Reviewed by: bcr, imp, emaste
Differential Revision: https://reviews.freebsd.org/D30736
As suggested in D27598. This also supports MK_WERROR.clang=no and
MK_WERROR.gcc=no to support the existing NO_WERROR.<compiler> uses.
Reviewed By: brooks
Differential Revision: https://reviews.freebsd.org/D27601
The new version only includes a specific version once, and uses the one
that's currently advised by tinderbox: -gcc6.
It also advises just installing the pkg, but mentions in a side-note at the
end where to find the source in the ports tree.
Reviewed by: jrtc27
Suggested by: jhb (use default from tinderbox)
Differential Revision: https://reviews.freebsd.org/D26820
This adds a new target update-packages which will create the new packages
compared to the last run.
This is how to use it:
At this point we cut a release
$ make buildworld ...
$ make buildkernel
$ make packages
There is now a PKG_VERSION directory with latest link pointing to it
Distribute the packages to server
$ something something that update the source tree
$ make buildworld ...
$ make buildkernel
$ make update-packages
You know have a PKG_VERSION directory in the REPODIR and latest link pointing to it.
In PKG_VERSION dir only the packages which differs from the latest run are
named PKG_VERSION, otherwise the old packages are there.
The process is :
Build the new packages in the PKG_VERSION directory
Compare the internal data with the PKG_VERSION_FROM version. The comparison is done
by checking the internal hash of the packages.
By default PKG_VERSION_FROM is set to what the latest link points to.
If the old and new version matches, we rm the new package and cp the old one.
Differential Revision: https://reviews.freebsd.org/D25984
LINT config files are about to be checked in directly. Eliminate
building them by hand here from NOTES files.
Reviewed by: kevans
Differential Revision: https://reviews.freebsd.org/D26540
This is the initial set up for PowerPC64LE.
The current plan is for this arch to remain experimental for FreeBSD 13.
This started as a weekend learning project for me and kinda snowballed from
there.
(More to follow momentarily.)
Reviewed by: imp (earlier version), emaste
Sponsored by: Tag1 Consulting, Inc.
Differential Revision: https://reviews.freebsd.org/D26399
${TARGET_ARCH} is empty here which results in empy MAKE_PARAMS being
passed to the buildkernel phase. This breaks the build when using the
strict TMPPATH since cc will not be included in $PATH.
Reviewed By: jhb
This uses GCC toolchains instead of LLVM on architectures supported by
GCC. Currently this uses GCC 6 on aarch64, amd64, i386, and mips.
Although this does also try to use GCC 6 on powerpc, it is always
skipped for now since a powerpc-gcc6 package is not available and the
structure of make universe makes it hard to skip a subset of arches
for a target. This should be short-lived as freebsd-gcc9 does include
a powerpc-gcc9 package so powerpc should work once this switches to
GCC 9.
Discussed with: emaste
Differential Revision: https://reviews.freebsd.org/D25732
This is a temporary hack to aid with config(8) changing in r360443.
It will not work for all cases.
env PATH is used because universe-toolchain is only built when worlds
are built, and then only if clang is needed, so it may not exist.
universe-toolchain needs to be expanded to always be built, inspected to
remove non-cross-build-safe tools, used for buildworld/buildkernel,
and potentially incremental build support.
Sponsored by: Dell EMC
Currently, powerpcspe is broken with clang. Add it to the EXTRA_TARGETS until
that's fixed.
Reviewed by: brooks, bdrewery, emaste (LGTM)
MFC After: 3 days
Differential Revision: https://reviews.freebsd.org/D24212
make kernels was originally documented (in commit r295099) as the same as make
universe -DJUST_BUILD_KERNELS. However, it used the UNIVERSE_TARGET=buildkernel
which is subtly different in that it also builds the toolchains and doesn't
build the LINT modules unless they happened to already exist in the tree. This
unbreaks POLA and just builds the kernels, including LINT, now rather than all
that other stuff as well. When the commit originally happened, the two just
differed by the LINT bug. r335711 introduced the universe dependency on the
toolchain that wasn't present before, which diverged the two further. This
restores the original intent of r295099.
Reviewed by: brooks, bdrewery, emaste (LGTM)
MFC After: 3 days
Differential Revision: https://reviews.freebsd.org/D24212
Add 'WITHOUT_WORLDS' and 'WITHOUT_KERNELS' as aliases for the inconsistently
named MAKE_JUST_KERNELS and MAKE_JUST_WORLDS respectively. I always forget the
MAKE_ part (or is it BUILD_), and it's inconsistent with everything
else. Document the new things, but leave speculation of any eventual MAKE_JUST_*
deprecation out of the manuals and comments.
Reviewed by: brooks, bdrewery, emaste (LGTM)
MFC After: 3 days
Differential Revision: https://reviews.freebsd.org/D24212
If EXTRA_TARGETS is defined, build all supported architecture
variants. By default, build architecture variants needed to provide
code coverage or that are commonly used.
Use this to disable building of all the hard-float and little-endian
MIPS architecture variants along with n32 by default.
Reviewed by: rpokala
Discussed with: imp, emaste
Differential Revision: https://reviews.freebsd.org/D24178
Now that we have updated the in-tree version of LLVM to 10.0, we have all the
necessary LLVM changes to use Clang+LLD as the default toolchain for MIPS.
Relnotes: yes
Reviewed By: emaste, jhb, brooks, kevans
Differential Revision: https://reviews.freebsd.org/D23204
The sparc64 architecture is being removed from FreeBSD 13, starting
now. This removes it from the top level only. It is the only
architecture that didn't see substantial work after the call to get
things working with the external toolchain.
- Enable clang and lld as system toolchains.
- Don't use external GCC for universe by default.
- Re-enable riscv64sf since it builds fine with clang + lld.
Reviewed by: emaste, mhorne
Relnotes: yes
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D23089
This fixes a regression in r356418 where the entire universe would
fail early due to an undefined make target when a given TARGET_ARCH
had no associated kernel configs. This is true for all of the
hard-float mips TARGET_ARCHes currently.
Pointy hat to: me
Reviewed by: emaste
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D23071
powerpcspe is disabled for now until clang/llvm issues with spe
have been fixed.
Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D23031
Previously, all of the kernels for a given TARGET were built if that
target was enabled. This was implemented by having each kernel built
via a universe_kernconf_<KERNEL> target that was depended on by a
universe_kernconfs target. However, this meant that if one did a
build with a limited set of TARGET_ARCH values for a given TARGET,
kernels could be built for which we hadn't built a world or toolchain.
For example, 'make TARGETS=mips TARGET_ARCHES_mips=mips64' would build
mips32 kernels.
Fix this by adding an extra layer of indirection in the kernel make
targets. universe_kernconf_<KERNEL> is now a dependency of a new
universe_kernconfs_<TARGET_ARCH>. universe_kernconfs in turn depends
on a list of universe_kernconfs_<target_arch> values, but only the
values enabled in TARGET_ARCHES_<TARGET>.
Reviewed by: imp
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D23031
TARGET=arm now defaults to TARGET_ARCH=armv7
TARGET_ARCH=arm is no longer valid.
Bump __FreeBSD_version to 1300073
Tested with make universe. Any stale LINT-V5 config files remaining in the tree
will fail the universe build. However, LINT-V5 was removed in r355119.
This retirement has been planned since last summer. The armv5 port is fragile:
it works OK for some peeople, and fails badly for others. There's a number of
subtle bugs in busdma, pmap and other MD parts of thee system that present
themselves under load or in unusual circumstances (like fsck after a
crash). stable/8, branched 10 years ago, was the last reliable release. Since
the support burden is larger then the benefit, the consensus view is armv5
should be removed from the tree.
Discussed with: arm@ mailing list and arm developer community.
It's on the chopping block in two months, the CI tinderbox doesn't bother with
it anymore either, and buildworld fails today due to an issue linking clang.
It's not worth investigating and it just eats up CPU cycles running universe
builds.
The wrong toolchain was set in MAKE_PARAMS_powerpc, however, there are
some other issues preventing powerpc from building in tinderbox:
1) There is no powerpc-gcc and powerpc-xtoolchain-gcc port that
provides an external 32-bit powerpc GCC toolchain.
2) On other targets, the same toolchain can build all of the
architectures for a given target. MIPS achieves this by always
setting -mabi and -EB/-EL explicitly instead of relying on the
compiler's default architecture. PowerPC might be able to do the
same thing, but as of today, powerpc-gcc would be required for
powerpc and powerpcspe and powerpc64-gcc would be required for
powerpc64. Our existing logic for make universe does not permit
per-MACHINE_ARCH toolchains.
I tried hacking TARGETS_powerpc to only include powerpc64 when
powerpc64-gcc was present, and while that skipped the 32-bit worlds,
it tried to build all the kernels.
Reported by: jeff
Discussed with: imp
Simplify expressions as suggested by jhb. The extra indirection made
sense in earlier versions of this patch, but not the final one.
While here, apply suggestion from emaste for wording of universe.
Also wordsmith awkwardly worded comment about when we effectively
neuter the universe build for an architecture.
Once llvm 9.0 has been vetted for mips and powerpc, I'll take them out
of these lists.
Only compile clang supporting architectures of amd64, arm, arm64,
i386, and riscv as part of universe. Compile the other architectures
if MAKE_OBSOLETE_GCC is defined. In all cases, explicit lists of
architectures in TARGETS= on the command line override.
For mips, powerpc and sparc64, do the same thing we do for risvc when
MAKE_OBSOLETE_GCC isn't defined and short-circuit their universe build
with an echo saying to install the xtoolchain port or pkg.
PR: 241134
Discussed on: arch@ (https://lists.freebsd.org/pipermail/freebsd-arch/2019-August/019674.html)
Differential Revision: https://reviews.freebsd.org/D21942
It doesn't make sense to limit to -j12 anymore, build scalability
is better than it used to be. Fold the hint into the description
of the universe target.
Reviewed by: imp
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D20342
The path is incorrect for the linker in the error message. It should have been
/usr/bin/ld, not /usr/bin/cc .
Reviewed by: emaste
Approved by: emaste (mentor)
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D19852
The sysent target is useful when changing makesyscalls.sh, when
making paired changes to syscalls.master files, or in a future where
freebsd32 sysent entries are built from the default syscalls.master.
Reviewed by: bdrewery
Obtained from: CheriBSD
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D17899
Inheriting $PATH during the build phase can cause the build to fail when
compiling on a different system due to missing build tools or incompatible
versions somewhere in $PATH. This has cause build failures for us before
due to the jenkins slaves still running FreeBSD 10.
Listing the tools we depend on explicitly instead of just using whatever
happens to be in $PATH allows us to check that we don't accidentally add a
new build dependency.
All tools that do no need to be bootstrapped will now be symlinked to
${WORLDTMP}/legacy/bin and during the build phase $PATH will only contain
${WORLDTMP}. There is also a new variable "BOOTSTRAP_ALL_TOOLS" which can
be set to force compiling almost all bootstrap tools instead of symlinking
them. This will not bootstrap tools such as cp,mv, etc. since they may be
used during the build and for those we should really only be using POSIX
compatible options.
Furthermore, this change is required in order to be able to build on
non-FreeBSD hosts. While the same binaries may exist on Linux/MacOS they
often accept different flags or produce incompatible output.
Approved By: brooks (mentor)
Differential Revision: https://reviews.freebsd.org/D16815
