Commit Graph

641 Commits

Author SHA1 Message Date
Bryan Drewery
2482ea42fb Revert "Fix native-xtools build"
This reverts commit 36269b8231.

This had an unintended change included.
2021-08-03 10:00:28 -07:00
Bryan Drewery
36269b8231 Fix native-xtools build
Fixes		https://github.com/freebsd/poudriere/issues/894
Fixes:		d0c737e18 ("Makefile: Fix MAKEOBJDIRPREFIX command-line")
X-MFC-With:	d0c737e18
2021-08-03 08:22:14 -07:00
Jessica Clarke
d0c737e184 Makefile: Fix MAKEOBJDIRPREFIX command-line variable check for bmake
Unlike the old fmake, running make FOO=bar when using bmake doesn't put
FOO=bar in .MAKEFLAGS at the top level, it instead just puts FOO in
.MAKEOVERRIDES and the full MAKEFLAGS will be formed for sub-makes.
Moreover, this only applies for sub-makes in rules, so this doesn't
apply to those in shell assignments. This means that the current check
does not catch make MAKEOBJDIRPREFIX=..., only those defined in config
files. Thus we must also check .MAKEOVERRIDES explicitly.

Reviewed by:	sjg
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D31015
2021-07-21 02:51:26 +01:00
John Baldwin
e290182bcf Remove 'make update'.
In the CVS days this used be a wrapper around either CVS or CVSup and
used to support updating src, doc, and ports checkouts.  With the move
to subversion this only supported updating src and was itself a
wrapper around 'svn update'.  With Git, users are probably better off
using appropriate Git commands directly to update without needing an
explicit make target as a wrapper.

Reviewed by:	bcr, imp, emaste
Differential Revision:	https://reviews.freebsd.org/D30736
2021-06-11 14:56:28 -07:00
Bryan Drewery
eb8bf6bb42 Fix 'make bmake' top-level bootstrapping.
Fixes:	ee10666327
2021-03-06 09:45:08 -08:00
Alex Richardson
7fa2f2a62f Rename NO_WERROR -> MK_WERROR=no
As suggested in D27598. This also supports MK_WERROR.clang=no and
MK_WERROR.gcc=no to support the existing NO_WERROR.<compiler> uses.

Reviewed By:	brooks
Differential Revision: https://reviews.freebsd.org/D27601
2021-01-07 09:31:03 +00:00
Kyle Evans
cf82304d7d Makefile: re-wordsmith the blurb about xtoolchain ports
The new version only includes a specific version once, and uses the one
that's currently advised by tinderbox: -gcc6.

It also advises just installing the pkg, but mentions in a side-note at the
end where to find the source in the ports tree.

Reviewed by:	jrtc27
Suggested by:	jhb (use default from tinderbox)
Differential Revision:	https://reviews.freebsd.org/D26820
2020-11-14 18:06:35 +00:00
Emmanuel Vadot
0ac8aa55da pkgbase: Add incremental packages
This adds a new target update-packages which will create the new packages
compared to the last run.

This is how to use it:
At this point we cut a release
$ make buildworld ...
$ make buildkernel
$ make packages

    There is now a PKG_VERSION directory with latest link pointing to it
    Distribute the packages to server

$ something something that update the source tree
$ make buildworld ...
$ make buildkernel
$ make update-packages
You know have a PKG_VERSION directory in the REPODIR and latest link pointing to it.
In PKG_VERSION dir only the packages which differs from the latest run are
named PKG_VERSION, otherwise the old packages are there.

The process is :
Build the new packages in the PKG_VERSION directory
Compare the internal data with the PKG_VERSION_FROM version. The comparison is done
by checking the internal hash of the packages.
By default PKG_VERSION_FROM is set to what the latest link points to.
If the old and new version matches, we rm the new package and cp the old one.

Differential Revision:	https://reviews.freebsd.org/D25984
2020-11-02 18:23:50 +00:00
Kyle Evans
5f2aaba453 Makefile: add a small blurb about building with gcc xtoolchain
The key details are to install the appropriate flavor of devel/freebsd-gcc9
and pass CROSS_TOOLCHAIN while building.

Suggested by:	kib
2020-10-16 15:16:23 +00:00
Warner Losh
bf0cf8dfba Eliminate building LINT makefiles
LINT config files are about to be checked in directly. Eliminate
building them by hand here from NOTES files.

Reviewed by: kevans
Differential Revision: https://reviews.freebsd.org/D26540
2020-10-09 00:27:40 +00:00
Brandon Bergren
b75abea4d0 [PowerPC64LE] Set up powerpc.powerpc64le architecture
This is the initial set up for PowerPC64LE.

The current plan is for this arch to remain experimental for FreeBSD 13.

This started as a weekend learning project for me and kinda snowballed from
there.

(More to follow momentarily.)

Reviewed by:	imp (earlier version), emaste
Sponsored by:	Tag1 Consulting, Inc.
Differential Revision:	https://reviews.freebsd.org/D26399
2020-09-22 23:49:30 +00:00
Alex Richardson
dce3fcd4ff Fix typo in r364325 that broke tinderbox with -DBUILD_WITH_STRICT_TMPPATH
${TARGET_ARCH} is empty here which results in empy MAKE_PARAMS being
passed to the buildkernel phase. This breaks the build when using the
strict TMPPATH since cc will not be included in $PATH.

Reviewed By:	jhb
2020-08-25 13:30:19 +00:00
John Baldwin
7d0c2b1131 Add a USE_GCC_TOOLCHAINS knob to make universe.
This uses GCC toolchains instead of LLVM on architectures supported by
GCC.  Currently this uses GCC 6 on aarch64, amd64, i386, and mips.

Although this does also try to use GCC 6 on powerpc, it is always
skipped for now since a powerpc-gcc6 package is not available and the
structure of make universe makes it hard to skip a subset of arches
for a target.  This should be short-lived as freebsd-gcc9 does include
a powerpc-gcc9 package so powerpc should work once this switches to
GCC 9.

Discussed with:	emaste
Differential Revision:	https://reviews.freebsd.org/D25732
2020-08-17 20:11:43 +00:00
Bryan Drewery
b19042cb01 Use universe-toolchain config(8)
This is a temporary hack to aid with config(8) changing in r360443.
It will not work for all cases.

env PATH is used because universe-toolchain is only built when worlds
are built, and then only if clang is needed, so it may not exist.

universe-toolchain needs to be expanded to always be built, inspected to
remove non-cross-build-safe tools, used for buildworld/buildkernel,
and potentially incremental build support.

Sponsored by:	Dell EMC
2020-04-29 02:18:39 +00:00
Warner Losh
be68b4b323 Add powerpcspe to the EXTRA_TARGETS
Currently, powerpcspe is broken with clang. Add it to the EXTRA_TARGETS until
that's fixed.

Reviewed by: brooks, bdrewery, emaste (LGTM)
MFC After: 3 days
Differential Revision: https://reviews.freebsd.org/D24212
2020-03-31 22:41:57 +00:00
Warner Losh
c9870c1542 Fix make kernels to match original commit message
make kernels was originally documented (in commit r295099) as the same as make
universe -DJUST_BUILD_KERNELS. However, it used the UNIVERSE_TARGET=buildkernel
which is subtly different in that it also builds the toolchains and doesn't
build the LINT modules unless they happened to already exist in the tree. This
unbreaks POLA and just builds the kernels, including LINT, now rather than all
that other stuff as well. When the commit originally happened, the two just
differed by the LINT bug. r335711 introduced the universe dependency on the
toolchain that wasn't present before, which diverged the two further. This
restores the original intent of r295099.

Reviewed by: brooks, bdrewery, emaste (LGTM)
MFC After: 3 days
Differential Revision: https://reviews.freebsd.org/D24212
2020-03-31 22:41:48 +00:00
Warner Losh
a7783b5a72 Make universe configuration more consistent with rest of system
Add 'WITHOUT_WORLDS' and 'WITHOUT_KERNELS' as aliases for the inconsistently
named MAKE_JUST_KERNELS and MAKE_JUST_WORLDS respectively. I always forget the
MAKE_ part (or is it BUILD_), and it's inconsistent with everything
else. Document the new things, but leave speculation of any eventual MAKE_JUST_*
deprecation out of the manuals and comments.

Reviewed by: brooks, bdrewery, emaste (LGTM)
MFC After: 3 days
Differential Revision: https://reviews.freebsd.org/D24212
2020-03-31 22:41:43 +00:00
John Baldwin
ff4c0e6f83 Disable rarely used architecture variants in make universe by default.
If EXTRA_TARGETS is defined, build all supported architecture
variants.  By default, build architecture variants needed to provide
code coverage or that are commonly used.

Use this to disable building of all the hard-float and little-endian
MIPS architecture variants along with n32 by default.

Reviewed by:	rpokala
Discussed with:	imp, emaste
Differential Revision:	https://reviews.freebsd.org/D24178
2020-03-24 18:16:02 +00:00
Alex Richardson
4a4c9a0f9e Use Clang and LLD as the default toolchain for MIPS
Now that we have updated the in-tree version of LLVM to 10.0, we have all the
necessary LLVM changes to use Clang+LLD as the default toolchain for MIPS.

Relnotes:	yes
Reviewed By:	emaste, jhb, brooks, kevans
Differential Revision: https://reviews.freebsd.org/D23204
2020-03-23 10:36:32 +00:00
Warner Losh
5672c76477 Per the firm plan, start to remove sparc64
The sparc64 architecture is being removed from FreeBSD 13, starting
now. This removes it from the top level only. It is the only
architecture that didn't see substantial work after the call to get
things working with the external toolchain.
2020-02-02 11:37:27 +00:00
John Baldwin
e48503fffd Use clang and lld as the default toolchain for RISCV.
- Enable clang and lld as system toolchains.
- Don't use external GCC for universe by default.
- Re-enable riscv64sf since it builds fine with clang + lld.

Reviewed by:	emaste, mhorne
Relnotes:	yes
Sponsored by:	DARPA
Differential Revision:	https://reviews.freebsd.org/D23089
2020-01-08 17:25:59 +00:00
John Baldwin
0840c1db16 Don't fail universe kernel stage for TARGET_ARCHes without a kernel config.
This fixes a regression in r356418 where the entire universe would
fail early due to an undefined make target when a given TARGET_ARCH
had no associated kernel configs.  This is true for all of the
hard-float mips TARGET_ARCHes currently.

Pointy hat to:	me
Reviewed by:	emaste
MFC after:	1 month
Differential Revision:	https://reviews.freebsd.org/D23071
2020-01-07 21:56:28 +00:00
John Baldwin
57496d6c2d Remove support for using GCC 4.2.1 from make tinderbox.
Reviewed by:	emaste
Differential Revision:	https://reviews.freebsd.org/D23054
2020-01-07 16:53:51 +00:00
John Baldwin
20723f186a Enable powerpc in make tinderbox using clang instead of GCC.
powerpcspe is disabled for now until clang/llvm issues with spe
have been fixed.

Reviewed by:	imp
Differential Revision:	https://reviews.freebsd.org/D23031
2020-01-06 17:36:28 +00:00
John Baldwin
0b2b53a2ea Only build kernels for enabled TARGET_ARCHes in make universe/tinderbox.
Previously, all of the kernels for a given TARGET were built if that
target was enabled.  This was implemented by having each kernel built
via a universe_kernconf_<KERNEL> target that was depended on by a
universe_kernconfs target.  However, this meant that if one did a
build with a limited set of TARGET_ARCH values for a given TARGET,
kernels could be built for which we hadn't built a world or toolchain.
For example, 'make TARGETS=mips TARGET_ARCHES_mips=mips64' would build
mips32 kernels.

Fix this by adding an extra layer of indirection in the kernel make
targets.  universe_kernconf_<KERNEL> is now a dependency of a new
universe_kernconfs_<TARGET_ARCH>.  universe_kernconfs in turn depends
on a list of universe_kernconfs_<target_arch> values, but only the
values enabled in TARGET_ARCHES_<TARGET>.

Reviewed by:	imp
MFC after:	1 month
Differential Revision:	https://reviews.freebsd.org/D23031
2020-01-06 17:34:17 +00:00
John Baldwin
0865e2bb05 Use the freebsd-gcc6 toolchains instead of powerpc64-xtoolchain-gcc.
Reviewed by:	brooks
Differential Revision:	https://reviews.freebsd.org/D23029
2020-01-04 00:59:47 +00:00
Andrew Turner
d0d8c0977c Add the missing trailing '/' when setting TARGET_ARCH from TARGET
This should fix the build when setting TARGET but not TARGET_ARCH.

Sponsored by:	DARPA, AFRL
2020-01-02 11:02:09 +00:00
Warner Losh
eb4977bd0f Remove arm/arm as a valid target.
TARGET=arm now defaults to TARGET_ARCH=armv7
TARGET_ARCH=arm is no longer valid.

Bump __FreeBSD_version to 1300073

Tested with make universe. Any stale LINT-V5 config files remaining in the tree
will fail the universe build. However, LINT-V5 was removed in r355119.

This retirement has been planned since last summer. The armv5 port is fragile:
it works OK for some peeople, and fails badly for others. There's a number of
subtle bugs in busdma, pmap and other MD parts of thee system that present
themselves under load or in unusual circumstances (like fsck after a
crash). stable/8, branched 10 years ago, was the last reliable release.  Since
the support burden is larger then the benefit, the consensus view is armv5
should be removed from the tree.

Discussed with: arm@ mailing list and arm developer community.
2020-01-02 03:25:26 +00:00
Conrad Meyer
e55890a9b8 Take arm.arm (armv5) out of universe
It's on the chopping block in two months, the CI tinderbox doesn't bother with
it anymore either, and buildworld fails today due to an issue linking clang.
It's not worth investigating and it just eats up CPU cycles running universe
builds.
2019-11-03 19:36:34 +00:00
John Baldwin
b411a285d9 Disconnect powerpc from the default tinderbox for now.
The wrong toolchain was set in MAKE_PARAMS_powerpc, however, there are
some other issues preventing powerpc from building in tinderbox:

1) There is no powerpc-gcc and powerpc-xtoolchain-gcc port that
   provides an external 32-bit powerpc GCC toolchain.

2) On other targets, the same toolchain can build all of the
   architectures for a given target.  MIPS achieves this by always
   setting -mabi and -EB/-EL explicitly instead of relying on the
   compiler's default architecture.  PowerPC might be able to do the
   same thing, but as of today, powerpc-gcc would be required for
   powerpc and powerpcspe and powerpc64-gcc would be required for
   powerpc64.  Our existing logic for make universe does not permit
   per-MACHINE_ARCH toolchains.

I tried hacking TARGETS_powerpc to only include powerpc64 when
powerpc64-gcc was present, and while that skipped the 32-bit worlds,
it tried to build all the kernels.

Reported by:	jeff
Discussed with:	imp
2019-10-14 21:22:42 +00:00
Warner Losh
2f9520d5fe Wordsmith and simplify
Simplify expressions as suggested by jhb. The extra indirection made
sense in earlier versions of this patch, but not the final one.

While here, apply suggestion from emaste for wording of universe.
Also wordsmith awkwardly worded comment about when we effectively
neuter the universe build for an architecture.

Once llvm 9.0 has been vetted for mips and powerpc, I'll take them out
of these lists.
2019-10-09 21:18:46 +00:00
Warner Losh
3f376e4b54 Don't compile old gcc 4.2.1 archs by default in universe/tinderbox.
Only compile clang supporting architectures of amd64, arm, arm64,
i386, and riscv as part of universe. Compile the other architectures
if MAKE_OBSOLETE_GCC is defined. In all cases, explicit lists of
architectures in TARGETS= on the command line override.

For mips, powerpc and sparc64, do the same thing we do for risvc when
MAKE_OBSOLETE_GCC isn't defined and short-circuit their universe build
with an echo saying to install the xtoolchain port or pkg.

PR: 241134
Discussed on: arch@ (https://lists.freebsd.org/pipermail/freebsd-arch/2019-August/019674.html)
Differential Revision: https://reviews.freebsd.org/D21942
2019-10-09 20:59:10 +00:00
Kyle Evans
c1d4532062 Add a top-level makeman target
Abstracting away the details of how src.conf(5) is generated is arguably a
good thing; do so with a top-level makeman target.
2019-10-01 15:10:35 +00:00
Emmanuel Vadot
df2bb80bf0 pkgbase: Add the sub stage-packages targets to TGTS
This helps when you don't want to run the dependencies targets.

Reviewed by:	bapt
Differential Revision:	https://reviews.freebsd.org/D20955
2019-07-24 08:00:00 +00:00
Mark Johnston
707b2d6586 Modernize the MAKE_JUST_KERNELS hint in the top-level makefile.
It doesn't make sense to limit to -j12 anymore, build scalability
is better than it used to be.  Fold the hint into the description
of the universe target.

Reviewed by:	imp
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D20342
2019-05-24 15:45:43 +00:00
Enji Cooper
78b78a6d3b Fix a typo when sanity checking in the bootstrap-tools target
The path is incorrect for the linker in the error message. It should have been
/usr/bin/ld, not /usr/bin/cc .

Reviewed by:	emaste
Approved by:	emaste (mentor)
MFC after:	2 weeks
Differential Revision: https://reviews.freebsd.org/D19852
2019-04-09 16:17:31 +00:00
Eric van Gyzen
7c5f370d3b Update comment about 'universe' disk usage
It's 167 GB now (or 81 GB with ZFS lz4).

MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2018-11-10 19:09:48 +00:00
Brooks Davis
0a9fcf3237 Add a top-level make target to rebuild all sysent files.
The sysent target is useful when changing makesyscalls.sh, when
making paired changes to syscalls.master files, or in a future where
freebsd32 sysent entries are built from the default syscalls.master.

Reviewed by:	bdrewery
Obtained from:	CheriBSD
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D17899
2018-11-08 00:35:00 +00:00
Alex Richardson
8f62bca488 Allow building world without inheriting $PATH
Inheriting $PATH during the build phase can cause the build to fail when
compiling on a different system due to missing build tools or incompatible
versions somewhere in $PATH. This has cause build failures for us before
due to the jenkins slaves still running FreeBSD 10.
Listing the tools we depend on explicitly instead of just using whatever
happens to be in $PATH allows us to check that we don't accidentally add a
new build dependency.

All tools that do no need to be bootstrapped will now be symlinked to
${WORLDTMP}/legacy/bin and during the build phase $PATH will only contain
${WORLDTMP}. There is also a new variable "BOOTSTRAP_ALL_TOOLS" which can
be set to force compiling almost all bootstrap tools instead of symlinking
them. This will not bootstrap tools such as cp,mv, etc. since they may be
used during the build and for those we should really only be using POSIX
compatible options.

Furthermore, this change is required in order to be able to build on
non-FreeBSD hosts. While the same binaries may exist on Linux/MacOS they
often accept different flags or produce incompatible output.

Approved By:	brooks (mentor)
Differential Revision: https://reviews.freebsd.org/D16815
2018-11-05 19:51:10 +00:00
Brooks Davis
e6d4316a45 Don't include the broken riscv64sf TARGET_ARCH in universe.
riscv64sf has been broken due to duplicate symbols for months and
degrades the quality of universe builds.  Remove it until this is
resolved leaving a comment to it is not re-added.

PR:		232085
Reviewed by:	emaste
Approved by:	re (gjb, kib)
Sponsored by:	DARPA, AFRL
2018-10-09 22:22:15 +00:00
Warner Losh
8b6b96c93d As discussed several times on freebsd-arch, start to decommission armeb.
Remove armeb/arm from KNOWN_ARCHS.
Remove armeb from arm universe targets.

Differential Revision:	https://reviews.freebsd.org/D16257
2018-07-17 23:23:34 +00:00
Bryan Drewery
b38b45af2f tinderbox: If the clang lookup fails fallback to the old default behavior.
This fixes errors from the MK_CLANG_BOOTSTRAP/MK_LLD_BOOTSTRAP lookups
to not force using XCC/XLD but to rather just build them as normal by
allowing their own bootstrap logic to work.

MFC after:	3 weeks
X-MFC-with:	r335711 r335769
Sponsored by:	Dell EMC
2018-06-28 18:22:20 +00:00
John Baldwin
32cec12c75 Include existing MAKE_PARAMS_<target> when determining the universe toolchain.
This fixes a warning for each RISCV target during universe by passing in
the required CROSS_TOOLCHAIN setting which will in turn set
CROSS_BINUTILS_PREFIX correctly.  It also ensures that a tinderbox build
uses the correct compiler for riscv.  Previously it was using the shared
clang compiler instead of riscv64-gcc.

Reviewed by:	bdrewery
Sponsored by:	DARPA / AFRL
Differential Revision:	https://reviews.freebsd.org/D16049
2018-06-28 18:19:46 +00:00
Devin Teske
37e29448ea Fix typo in top-level Makefile
Submitted by:	Ben Widawsky <ben.widawsky@intel.com>
MFC after:	3 days
X-MFC-to:	stable/11 stable/10
Sponsored by:	Smule, Inc.
Differential Revision:	https://reviews.freebsd.org/P186
2018-06-27 22:15:50 +00:00
Bryan Drewery
01f124b46f tinderbox: Only build clang/lld once if needed.
Need to handle LLD_BOOTSTRAP separately (for archs like i386).
This would be much better off with an off-by-default option like
SHARED_TOOLCHAIN that universe force-enabled.  Then a normal buildworld
would store the toolchain there if enabled and otherwise in WORLDTMP
with only the 1 arch selected.

MFC after:	3 weeks
Sponsored by:	Dell EMC
2018-06-27 16:58:10 +00:00
Bryan Drewery
581672388f tinderbox: Give details about kernel builds.
This is a bit noisy now but it was silent before leading to
wondering if it was doing anything.

MFC after:	1 week
Suggested by:	rpokala
Sponsored by:	Dell EMC
2018-06-27 16:57:59 +00:00
Bryan Drewery
d406018534 Only look for NOTES as needed.
Sponsored by:	Dell EMC
2018-06-20 19:45:04 +00:00
Bryan Drewery
a7d84af890 Add WITH_SYSTEM_LINKER, on by default, that avoids building lld when possible.
This works similar to WITH_SYSTEM_COMPILER added in r300354.  It only
supports lld via WITH_LLD_BOOTSTRAP.

When both SYSTEM_COMPILER and SYSTEM_LINKER logic passes then libclang
will not build in cross-tools.  If either check fails though then
libclang is built.

The .info is reworked to notify when libclang will be built since if
either clang or lld needs to be rebuilt, but not the other, the
notification can lead to confusion on why "clang is building".

-fuse-ld= is not used with this method so some combinations of compiler
and linker are expected to fail.

A new 'make test-system-linker' target is added to see the logic results.

Makefile.inc1:
  CROSS_BINUTILS_PREFIX support had to be moved higher up so that XLD
  could be set and MK_LLD_BOOTSTRAP disabled before checking SYSTEM_LINKER
  logic as done with SYSTEM_COMPILER.  This also required moving where
  bsd.linker.mk was read since XLD needs to be set before parsing it.  This
  creates a situation where src.opts.mk can not test LINKER_FEATURES or
  add LLD_BOOTSTAP to BROKEN_OPTIONS.

Reviewed by:	emaste (earlier version)
Sponsored by:	Dell EMC
Differential Revision:	https://reviews.freebsd.org/D15894
2018-06-20 16:10:14 +00:00
Ed Maste
ff3a8b3427 Restore arm, riscv, sparc64, and mips to UNIVERSE after r334128 2018-05-24 14:01:22 +00:00
Matt Macy
e98bbcf9ca libpmcstat: compile in events based on json description 2018-05-24 04:30:06 +00:00