Commit Graph

628 Commits

Author SHA1 Message Date
Bryan Drewery
b19042cb01 Use universe-toolchain config(8)
This is a temporary hack to aid with config(8) changing in r360443.
It will not work for all cases.

env PATH is used because universe-toolchain is only built when worlds
are built, and then only if clang is needed, so it may not exist.

universe-toolchain needs to be expanded to always be built, inspected to
remove non-cross-build-safe tools, used for buildworld/buildkernel,
and potentially incremental build support.

Sponsored by:	Dell EMC
2020-04-29 02:18:39 +00:00
Warner Losh
be68b4b323 Add powerpcspe to the EXTRA_TARGETS
Currently, powerpcspe is broken with clang. Add it to the EXTRA_TARGETS until
that's fixed.

Reviewed by: brooks, bdrewery, emaste (LGTM)
MFC After: 3 days
Differential Revision: https://reviews.freebsd.org/D24212
2020-03-31 22:41:57 +00:00
Warner Losh
c9870c1542 Fix make kernels to match original commit message
make kernels was originally documented (in commit r295099) as the same as make
universe -DJUST_BUILD_KERNELS. However, it used the UNIVERSE_TARGET=buildkernel
which is subtly different in that it also builds the toolchains and doesn't
build the LINT modules unless they happened to already exist in the tree. This
unbreaks POLA and just builds the kernels, including LINT, now rather than all
that other stuff as well. When the commit originally happened, the two just
differed by the LINT bug. r335711 introduced the universe dependency on the
toolchain that wasn't present before, which diverged the two further. This
restores the original intent of r295099.

Reviewed by: brooks, bdrewery, emaste (LGTM)
MFC After: 3 days
Differential Revision: https://reviews.freebsd.org/D24212
2020-03-31 22:41:48 +00:00
Warner Losh
a7783b5a72 Make universe configuration more consistent with rest of system
Add 'WITHOUT_WORLDS' and 'WITHOUT_KERNELS' as aliases for the inconsistently
named MAKE_JUST_KERNELS and MAKE_JUST_WORLDS respectively. I always forget the
MAKE_ part (or is it BUILD_), and it's inconsistent with everything
else. Document the new things, but leave speculation of any eventual MAKE_JUST_*
deprecation out of the manuals and comments.

Reviewed by: brooks, bdrewery, emaste (LGTM)
MFC After: 3 days
Differential Revision: https://reviews.freebsd.org/D24212
2020-03-31 22:41:43 +00:00
John Baldwin
ff4c0e6f83 Disable rarely used architecture variants in make universe by default.
If EXTRA_TARGETS is defined, build all supported architecture
variants.  By default, build architecture variants needed to provide
code coverage or that are commonly used.

Use this to disable building of all the hard-float and little-endian
MIPS architecture variants along with n32 by default.

Reviewed by:	rpokala
Discussed with:	imp, emaste
Differential Revision:	https://reviews.freebsd.org/D24178
2020-03-24 18:16:02 +00:00
Alex Richardson
4a4c9a0f9e Use Clang and LLD as the default toolchain for MIPS
Now that we have updated the in-tree version of LLVM to 10.0, we have all the
necessary LLVM changes to use Clang+LLD as the default toolchain for MIPS.

Relnotes:	yes
Reviewed By:	emaste, jhb, brooks, kevans
Differential Revision: https://reviews.freebsd.org/D23204
2020-03-23 10:36:32 +00:00
Warner Losh
5672c76477 Per the firm plan, start to remove sparc64
The sparc64 architecture is being removed from FreeBSD 13, starting
now. This removes it from the top level only. It is the only
architecture that didn't see substantial work after the call to get
things working with the external toolchain.
2020-02-02 11:37:27 +00:00
John Baldwin
e48503fffd Use clang and lld as the default toolchain for RISCV.
- Enable clang and lld as system toolchains.
- Don't use external GCC for universe by default.
- Re-enable riscv64sf since it builds fine with clang + lld.

Reviewed by:	emaste, mhorne
Relnotes:	yes
Sponsored by:	DARPA
Differential Revision:	https://reviews.freebsd.org/D23089
2020-01-08 17:25:59 +00:00
John Baldwin
0840c1db16 Don't fail universe kernel stage for TARGET_ARCHes without a kernel config.
This fixes a regression in r356418 where the entire universe would
fail early due to an undefined make target when a given TARGET_ARCH
had no associated kernel configs.  This is true for all of the
hard-float mips TARGET_ARCHes currently.

Pointy hat to:	me
Reviewed by:	emaste
MFC after:	1 month
Differential Revision:	https://reviews.freebsd.org/D23071
2020-01-07 21:56:28 +00:00
John Baldwin
57496d6c2d Remove support for using GCC 4.2.1 from make tinderbox.
Reviewed by:	emaste
Differential Revision:	https://reviews.freebsd.org/D23054
2020-01-07 16:53:51 +00:00
John Baldwin
20723f186a Enable powerpc in make tinderbox using clang instead of GCC.
powerpcspe is disabled for now until clang/llvm issues with spe
have been fixed.

Reviewed by:	imp
Differential Revision:	https://reviews.freebsd.org/D23031
2020-01-06 17:36:28 +00:00
John Baldwin
0b2b53a2ea Only build kernels for enabled TARGET_ARCHes in make universe/tinderbox.
Previously, all of the kernels for a given TARGET were built if that
target was enabled.  This was implemented by having each kernel built
via a universe_kernconf_<KERNEL> target that was depended on by a
universe_kernconfs target.  However, this meant that if one did a
build with a limited set of TARGET_ARCH values for a given TARGET,
kernels could be built for which we hadn't built a world or toolchain.
For example, 'make TARGETS=mips TARGET_ARCHES_mips=mips64' would build
mips32 kernels.

Fix this by adding an extra layer of indirection in the kernel make
targets.  universe_kernconf_<KERNEL> is now a dependency of a new
universe_kernconfs_<TARGET_ARCH>.  universe_kernconfs in turn depends
on a list of universe_kernconfs_<target_arch> values, but only the
values enabled in TARGET_ARCHES_<TARGET>.

Reviewed by:	imp
MFC after:	1 month
Differential Revision:	https://reviews.freebsd.org/D23031
2020-01-06 17:34:17 +00:00
John Baldwin
0865e2bb05 Use the freebsd-gcc6 toolchains instead of powerpc64-xtoolchain-gcc.
Reviewed by:	brooks
Differential Revision:	https://reviews.freebsd.org/D23029
2020-01-04 00:59:47 +00:00
Andrew Turner
d0d8c0977c Add the missing trailing '/' when setting TARGET_ARCH from TARGET
This should fix the build when setting TARGET but not TARGET_ARCH.

Sponsored by:	DARPA, AFRL
2020-01-02 11:02:09 +00:00
Warner Losh
eb4977bd0f Remove arm/arm as a valid target.
TARGET=arm now defaults to TARGET_ARCH=armv7
TARGET_ARCH=arm is no longer valid.

Bump __FreeBSD_version to 1300073

Tested with make universe. Any stale LINT-V5 config files remaining in the tree
will fail the universe build. However, LINT-V5 was removed in r355119.

This retirement has been planned since last summer. The armv5 port is fragile:
it works OK for some peeople, and fails badly for others. There's a number of
subtle bugs in busdma, pmap and other MD parts of thee system that present
themselves under load or in unusual circumstances (like fsck after a
crash). stable/8, branched 10 years ago, was the last reliable release.  Since
the support burden is larger then the benefit, the consensus view is armv5
should be removed from the tree.

Discussed with: arm@ mailing list and arm developer community.
2020-01-02 03:25:26 +00:00
Conrad Meyer
e55890a9b8 Take arm.arm (armv5) out of universe
It's on the chopping block in two months, the CI tinderbox doesn't bother with
it anymore either, and buildworld fails today due to an issue linking clang.
It's not worth investigating and it just eats up CPU cycles running universe
builds.
2019-11-03 19:36:34 +00:00
John Baldwin
b411a285d9 Disconnect powerpc from the default tinderbox for now.
The wrong toolchain was set in MAKE_PARAMS_powerpc, however, there are
some other issues preventing powerpc from building in tinderbox:

1) There is no powerpc-gcc and powerpc-xtoolchain-gcc port that
   provides an external 32-bit powerpc GCC toolchain.

2) On other targets, the same toolchain can build all of the
   architectures for a given target.  MIPS achieves this by always
   setting -mabi and -EB/-EL explicitly instead of relying on the
   compiler's default architecture.  PowerPC might be able to do the
   same thing, but as of today, powerpc-gcc would be required for
   powerpc and powerpcspe and powerpc64-gcc would be required for
   powerpc64.  Our existing logic for make universe does not permit
   per-MACHINE_ARCH toolchains.

I tried hacking TARGETS_powerpc to only include powerpc64 when
powerpc64-gcc was present, and while that skipped the 32-bit worlds,
it tried to build all the kernels.

Reported by:	jeff
Discussed with:	imp
2019-10-14 21:22:42 +00:00
Warner Losh
2f9520d5fe Wordsmith and simplify
Simplify expressions as suggested by jhb. The extra indirection made
sense in earlier versions of this patch, but not the final one.

While here, apply suggestion from emaste for wording of universe.
Also wordsmith awkwardly worded comment about when we effectively
neuter the universe build for an architecture.

Once llvm 9.0 has been vetted for mips and powerpc, I'll take them out
of these lists.
2019-10-09 21:18:46 +00:00
Warner Losh
3f376e4b54 Don't compile old gcc 4.2.1 archs by default in universe/tinderbox.
Only compile clang supporting architectures of amd64, arm, arm64,
i386, and riscv as part of universe. Compile the other architectures
if MAKE_OBSOLETE_GCC is defined. In all cases, explicit lists of
architectures in TARGETS= on the command line override.

For mips, powerpc and sparc64, do the same thing we do for risvc when
MAKE_OBSOLETE_GCC isn't defined and short-circuit their universe build
with an echo saying to install the xtoolchain port or pkg.

PR: 241134
Discussed on: arch@ (https://lists.freebsd.org/pipermail/freebsd-arch/2019-August/019674.html)
Differential Revision: https://reviews.freebsd.org/D21942
2019-10-09 20:59:10 +00:00
Kyle Evans
c1d4532062 Add a top-level makeman target
Abstracting away the details of how src.conf(5) is generated is arguably a
good thing; do so with a top-level makeman target.
2019-10-01 15:10:35 +00:00
Emmanuel Vadot
df2bb80bf0 pkgbase: Add the sub stage-packages targets to TGTS
This helps when you don't want to run the dependencies targets.

Reviewed by:	bapt
Differential Revision:	https://reviews.freebsd.org/D20955
2019-07-24 08:00:00 +00:00
Mark Johnston
707b2d6586 Modernize the MAKE_JUST_KERNELS hint in the top-level makefile.
It doesn't make sense to limit to -j12 anymore, build scalability
is better than it used to be.  Fold the hint into the description
of the universe target.

Reviewed by:	imp
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D20342
2019-05-24 15:45:43 +00:00
Enji Cooper
78b78a6d3b Fix a typo when sanity checking in the bootstrap-tools target
The path is incorrect for the linker in the error message. It should have been
/usr/bin/ld, not /usr/bin/cc .

Reviewed by:	emaste
Approved by:	emaste (mentor)
MFC after:	2 weeks
Differential Revision: https://reviews.freebsd.org/D19852
2019-04-09 16:17:31 +00:00
Eric van Gyzen
7c5f370d3b Update comment about 'universe' disk usage
It's 167 GB now (or 81 GB with ZFS lz4).

MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2018-11-10 19:09:48 +00:00
Brooks Davis
0a9fcf3237 Add a top-level make target to rebuild all sysent files.
The sysent target is useful when changing makesyscalls.sh, when
making paired changes to syscalls.master files, or in a future where
freebsd32 sysent entries are built from the default syscalls.master.

Reviewed by:	bdrewery
Obtained from:	CheriBSD
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D17899
2018-11-08 00:35:00 +00:00
Alex Richardson
8f62bca488 Allow building world without inheriting $PATH
Inheriting $PATH during the build phase can cause the build to fail when
compiling on a different system due to missing build tools or incompatible
versions somewhere in $PATH. This has cause build failures for us before
due to the jenkins slaves still running FreeBSD 10.
Listing the tools we depend on explicitly instead of just using whatever
happens to be in $PATH allows us to check that we don't accidentally add a
new build dependency.

All tools that do no need to be bootstrapped will now be symlinked to
${WORLDTMP}/legacy/bin and during the build phase $PATH will only contain
${WORLDTMP}. There is also a new variable "BOOTSTRAP_ALL_TOOLS" which can
be set to force compiling almost all bootstrap tools instead of symlinking
them. This will not bootstrap tools such as cp,mv, etc. since they may be
used during the build and for those we should really only be using POSIX
compatible options.

Furthermore, this change is required in order to be able to build on
non-FreeBSD hosts. While the same binaries may exist on Linux/MacOS they
often accept different flags or produce incompatible output.

Approved By:	brooks (mentor)
Differential Revision: https://reviews.freebsd.org/D16815
2018-11-05 19:51:10 +00:00
Brooks Davis
e6d4316a45 Don't include the broken riscv64sf TARGET_ARCH in universe.
riscv64sf has been broken due to duplicate symbols for months and
degrades the quality of universe builds.  Remove it until this is
resolved leaving a comment to it is not re-added.

PR:		232085
Reviewed by:	emaste
Approved by:	re (gjb, kib)
Sponsored by:	DARPA, AFRL
2018-10-09 22:22:15 +00:00
Warner Losh
8b6b96c93d As discussed several times on freebsd-arch, start to decommission armeb.
Remove armeb/arm from KNOWN_ARCHS.
Remove armeb from arm universe targets.

Differential Revision:	https://reviews.freebsd.org/D16257
2018-07-17 23:23:34 +00:00
Bryan Drewery
b38b45af2f tinderbox: If the clang lookup fails fallback to the old default behavior.
This fixes errors from the MK_CLANG_BOOTSTRAP/MK_LLD_BOOTSTRAP lookups
to not force using XCC/XLD but to rather just build them as normal by
allowing their own bootstrap logic to work.

MFC after:	3 weeks
X-MFC-with:	r335711 r335769
Sponsored by:	Dell EMC
2018-06-28 18:22:20 +00:00
John Baldwin
32cec12c75 Include existing MAKE_PARAMS_<target> when determining the universe toolchain.
This fixes a warning for each RISCV target during universe by passing in
the required CROSS_TOOLCHAIN setting which will in turn set
CROSS_BINUTILS_PREFIX correctly.  It also ensures that a tinderbox build
uses the correct compiler for riscv.  Previously it was using the shared
clang compiler instead of riscv64-gcc.

Reviewed by:	bdrewery
Sponsored by:	DARPA / AFRL
Differential Revision:	https://reviews.freebsd.org/D16049
2018-06-28 18:19:46 +00:00
Devin Teske
37e29448ea Fix typo in top-level Makefile
Submitted by:	Ben Widawsky <ben.widawsky@intel.com>
MFC after:	3 days
X-MFC-to:	stable/11 stable/10
Sponsored by:	Smule, Inc.
Differential Revision:	https://reviews.freebsd.org/P186
2018-06-27 22:15:50 +00:00
Bryan Drewery
01f124b46f tinderbox: Only build clang/lld once if needed.
Need to handle LLD_BOOTSTRAP separately (for archs like i386).
This would be much better off with an off-by-default option like
SHARED_TOOLCHAIN that universe force-enabled.  Then a normal buildworld
would store the toolchain there if enabled and otherwise in WORLDTMP
with only the 1 arch selected.

MFC after:	3 weeks
Sponsored by:	Dell EMC
2018-06-27 16:58:10 +00:00
Bryan Drewery
581672388f tinderbox: Give details about kernel builds.
This is a bit noisy now but it was silent before leading to
wondering if it was doing anything.

MFC after:	1 week
Suggested by:	rpokala
Sponsored by:	Dell EMC
2018-06-27 16:57:59 +00:00
Bryan Drewery
d406018534 Only look for NOTES as needed.
Sponsored by:	Dell EMC
2018-06-20 19:45:04 +00:00
Bryan Drewery
a7d84af890 Add WITH_SYSTEM_LINKER, on by default, that avoids building lld when possible.
This works similar to WITH_SYSTEM_COMPILER added in r300354.  It only
supports lld via WITH_LLD_BOOTSTRAP.

When both SYSTEM_COMPILER and SYSTEM_LINKER logic passes then libclang
will not build in cross-tools.  If either check fails though then
libclang is built.

The .info is reworked to notify when libclang will be built since if
either clang or lld needs to be rebuilt, but not the other, the
notification can lead to confusion on why "clang is building".

-fuse-ld= is not used with this method so some combinations of compiler
and linker are expected to fail.

A new 'make test-system-linker' target is added to see the logic results.

Makefile.inc1:
  CROSS_BINUTILS_PREFIX support had to be moved higher up so that XLD
  could be set and MK_LLD_BOOTSTRAP disabled before checking SYSTEM_LINKER
  logic as done with SYSTEM_COMPILER.  This also required moving where
  bsd.linker.mk was read since XLD needs to be set before parsing it.  This
  creates a situation where src.opts.mk can not test LINKER_FEATURES or
  add LLD_BOOTSTAP to BROKEN_OPTIONS.

Reviewed by:	emaste (earlier version)
Sponsored by:	Dell EMC
Differential Revision:	https://reviews.freebsd.org/D15894
2018-06-20 16:10:14 +00:00
Ed Maste
ff3a8b3427 Restore arm, riscv, sparc64, and mips to UNIVERSE after r334128 2018-05-24 14:01:22 +00:00
Matt Macy
e98bbcf9ca libpmcstat: compile in events based on json description 2018-05-24 04:30:06 +00:00
Bryan Drewery
9d5beec1b9 Don't read SRC_ENV_CONF for MAKEOBJDIRPREFIX guard.
This is mostly to allow using MAKEOBJDIRPREFIX in src-env.conf on stable where
src.sys.obj.mk is not going to be MFC'd.  It is still valid on head but
effectively a NOP due to MAKEOBJDIRPREFIX being handled differently in
src.sys.obj.mk.

Reported by:	eadler
MFC after:	1 week
Sponsored by:	Dell EMC
2018-03-03 23:23:23 +00:00
Warner Losh
207efdb345 Add a note about why we have the conditional before including
bsd.compiler.mk. It's so fmake from older 9.x systems still
works (still a supported build config, and having the note here
will let us know when we can cull it more easily).

Also pull in a related change from include to sinclude from
arichardson@'s cross building work, as well as it's companion in
Makefile.inc1 with a note about why we do the odd thing there.

Submitted by: archardson
Differential Revision: https://reviews.freebsd.org/D14241
2018-02-07 16:28:26 +00:00
Alex Richardson
95eff7c0c3 crossbuild: Make the CHECK_TIME variable work on Linux
Linux /usr/bin/find doesn't understand the -mtime -0s flag.
Instead create a temporary file and compare that file's mtime to
sys/sys/param.h to check whether the clock is correct.

Reviewed By:	jhb, imp
Approved By:	jhb (mentor)
Differential Revision: https://reviews.freebsd.org/D14157
2018-02-06 15:41:45 +00:00
Bryan Drewery
1ccacaed5f tinderbox/universe: Don't execute KERNCONFS lookup if not needed.
Sponsored by:	Dell EMC
2017-12-07 00:31:28 +00:00
Bryan Drewery
9ef3e71cac Revert r325529: No longer needed after r325699.
Sponsored by:	Dell EMC
2017-12-07 00:31:10 +00:00
Warner Losh
573a3ef083 Use TARGET_ARCH=riscv64 when TARGET=riscv
The supported targets are riscv64 and riscv64sf. Use the former when
building with a bare TARGET=riscv and it is the more common one.

Sponsored by: Netflix
2017-11-21 19:23:12 +00:00
Bryan Drewery
a7fa261e37 Move top-level AUTO_OBJ logic to Makefile.sys.inc.
Sponsored by:	Dell
2017-11-18 20:01:12 +00:00
Warner Losh
1cbb58886a Remove build system support for lint.
Differential Revision: https://reviews.freebsd.org/D13124
2017-11-17 18:16:46 +00:00
Bryan Drewery
d7a699d3d8 Rework r325568 so all 'make LINT' targets work.
Reported by:	ngie
Sponsored by:	Dell EMC Isilon
2017-11-13 20:49:08 +00:00
Bryan Drewery
d8d24cec9f META_MODE: Avoid some logic if disabled and allow enabling with make argument.
Sponsored by:	Dell EMC Isilon
2017-11-11 01:10:50 +00:00
Bryan Drewery
58d86ed342 AUTO_OBJ: Disable in non-build targets.
There's no reason to create object directories for targets like 'installworld'
or 'distributeworld', and the others in this list.  Specifying MK_AUTO_OBJ as a
make argument allows circumventing this if needed for some reason.

This fixes mergemaster creating a full object tree due to doing a 'make
installconfig' tree walk.

Reported by:	Mark Millard
Sponsored by:	Dell EMC Isilon
2017-11-11 01:10:46 +00:00
Bryan Drewery
6555da9c5b Remove rerelease target that was added in r38978.
I'm not sure what this target is but it doesn't exist now and doesn't seem
to have existed before.

Sponsored by:	Dell EMC Isilon
2017-11-10 21:52:33 +00:00
Bryan Drewery
9cde684d49 native-xtools: Fix for GCC archs.
- This also adds in a _cleanobj step as needed.
- This redirects TARGET/TARGET_ARCH to NXB_TARGET/NXB_TARGET_ARCH in
  Makefile.inc1 as the main build needs to be for MACHINE rather
  than TARGET.

  First build the toolchain and then use that as an external toolchain
  to build the needed directories and NXB_TARGET-toolchain, all as
  MACHINE files though via TARGET_TRIPLE=MACHINE_TRIPLE.

  The NXBDIRS is evaluated in the 'everything' submake as it needs to be
  based on TARGET's src.opts.mk values, such as MK_GCC=yes when building
  on a MK_CLANG=yes MACHINE.  This can likely be changed to a specific
  _native-xtools-everything target later and the funky late evaluation
  of SUBDIR_OVERRIDE removed.

X-MFC-With:	r325001
Pointyhat to:	bdrewery
MFC after:	2 months
Sponsored by:	Dell EMC Isilon
2017-11-10 19:53:07 +00:00