Commit Graph

1771 Commits

Author SHA1 Message Date
Alex Richardson
5bb9250e0a Add necessary Makefile.inc1 infrastructure for building on non-FreeBSD
The most awkward bit in this patch is the bootstrapping of m4:
We can't simply use the host version of m4 since that is not compatible
with the flags passed by lex (at least on macOS, possibly also on Linux).
Therefore we need to bootstrap m4, but lex needs m4 to build and m4 also
depends on lex (which needs m4 to generate any files). To work around this
cyclic dependency we can build a bootstrap version of m4 (with pre-generated
files) then use that to build the real m4.

This patch also changes the xz/unxz/dd tools to always use the host version
since the version in the source tree cannot easily be bootstrapped on macOS
or Linux.

Reviewed By:	brooks, imp (earlier version)
Differential Revision: https://reviews.freebsd.org/D25992
2020-08-25 13:29:57 +00:00
Matt Macy
9e5787d228 Merge OpenZFS support in to HEAD.
The primary benefit is maintaining a completely shared
code base with the community allowing FreeBSD to receive
new features sooner and with less effort.

I would advise against doing 'zpool upgrade'
or creating indispensable pools using new
features until this change has had a month+
to soak.

Work on merging FreeBSD support in to what was
at the time "ZFS on Linux" began in August 2018.
I first publicly proposed transitioning FreeBSD
to (new) OpenZFS on December 18th, 2018. FreeBSD
support in OpenZFS was finally completed in December
2019. A CFT for downstreaming OpenZFS support in
to FreeBSD was first issued on July 8th. All issues
that were reported have been addressed or, for
a couple of less critical matters there are
pull requests in progress with OpenZFS. iXsystems
has tested and dogfooded extensively internally.
The TrueNAS 12 release is based on OpenZFS with
some additional features that have not yet made
it upstream.

Improvements include:
  project quotas, encrypted datasets,
  allocation classes, vectorized raidz,
  vectorized checksums, various command line
  improvements, zstd compression.

Thanks to those who have helped along the way:
Ryan Moeller, Allan Jude, Zack Welch, and many
others.

Sponsored by:	iXsystems, Inc.
Differential Revision:	https://reviews.freebsd.org/D25872
2020-08-25 02:21:27 +00:00
Warner Losh
0e533c72bc When copying over the binaries, use '-p' to preserve date/time
Although I can't reproduce it, others are seeing different lex/yacc
programs always regenerated after my change to copy rather than
symlink the files. The reported fix is to add '-p' to the copies.
Since it doesn't hurt, go head and add it, though the reasons for
this mattering remain at best obscure and poorly articulated.
2020-08-24 16:06:11 +00:00
Alex Richardson
ebae797c82 Also print number of available CPUs on Linux
Without this change the buildworld/buildkernel epilogue looks like this:
>>> World built in 249 seconds, sysctl: cannot stat /proc/sys/hw/ncpu: No such file or directory
ncpu: , make -j72.

Reviewed By:	emaste, bdrewery
Differential Revision: https://reviews.freebsd.org/D26056
2020-08-24 09:20:38 +00:00
Alex Richardson
0f31fdf253 Pass the installworld install(1) flags to make buildenv
This ensure that running make install inside buildenv correctly includes
the METALOG flags when building with -DNO_ROOT.

Reviewed By:	brooks
Differential Revision: https://reviews.freebsd.org/D26038
2020-08-24 09:20:13 +00:00
Bjoern A. Zeeb
530134d291 After the clang/llvm version 11 import LLD_VERSION is no longer used
upstream so Version.inc now only defines LLD_VERSION_STRING.

This breaks the WANT_LINKER_VERSION magic and might lead to us building
more than needed (e.g., for croos-tools).

Change the awk script to parse LLD_VERSION_STRING instead of LLD_VERSION,
which not only unbreaks the current situation but should also be backwards
compatible as dim points out.

PR:		248818
Reviewed by:	emaste, dim (seems right and the way to go)
MFC after:	4 weeks
X-MFC before:	364284
2020-08-21 22:31:45 +00:00
Glen Barber
440cec3faf MFH
Sponsored by:	Rubicon Communications, LLC (netgate.com)
2020-08-12 17:16:26 +00:00
Alex Richardson
f57b27e3f8 Fix -DBUILD_WITH_STRICT_TMPPATH dtrace builds
Some of the scripts used for libdtrace invoke nawk instead of awk
(for example cddl/contrib/opensolaris/lib/libdtrace/common/mknames.sh).
When bootstrapping all tools, we get the nawk -> awk link while building
usr.bin/awk, but when linking/copying the dependencies from the host we
were only adding awk but not nawk.

This was silently generating invalid files when building libdtrace with
BUILD_WITH_STRICT_TMPPATH=1 since those scripts invoke nawk instead of
awk. In addition to adding the missing link this commit also adds
set -e to those scripts to catch errors like this in the future.

Reviewed By:	markj, emaste
Differential Revision: https://reviews.freebsd.org/D26025
2020-08-11 16:46:48 +00:00
Alex Richardson
1a18ab420b Allow overriding the tool used for stripping binaries
Since the make variable STRIP is already used for other purposes, this
uses STRIPBIN (which is also used for the same purpose by install(1).
This allows using LLVM objcopy to strip binaries instead of the in-tree
elftoolchain objcopy. We make use of this in CheriBSD since passing
binaries generated by our toolchain to elftoolchain strip sometimes results
in assertion failures.

This allows working around https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=248516
by specifying STRIPBIN=/path/to/llvm-strip

Obtained from:	CheriBSD
Reviewed By:	emaste, brooks
Differential Revision: https://reviews.freebsd.org/D25988
2020-08-11 16:46:27 +00:00
Emmanuel Vadot
a90022d4d1 pkgbase: Replace / with | for PKG_WWW
PKG_WWW contain / char, replace the delimiter by a '|'.

Reported by:	0mp
2020-08-11 10:07:59 +00:00
Emmanuel Vadot
82087d4b68 pkgbase: Add PKG_NAME_PREFIX, PKG_MAINTAINER and PKG_WWW
This is useful for downstream users to customize the packages.
Reviewed by:	emaste
Differential Revision:	https://reviews.freebsd.org/D26019
2020-08-11 08:42:24 +00:00
Warner Losh
33176cdc87 The practice of creating symbolic links is somewhat fragile. Always
make copies instead.

There's too many times that we can't run the new binaries with old
libraries. Making the links when things are known to be 'safe' is a
nice optimization, but a copy of all the binaries is only 30MB, so
saving the copies at the cost of increased support when new symbols
are added and used as part of the bootstrap seems to be unwise.

There may be additional optimizations possible here, especially for
!FreeBSD hosts. However, that's beyond the scope of the problem I'm
trying to fix with make failing mid-way through an installworld across
change r363679. This optimization there caused us to run a new binary
with an old library once a new make was installed due to the symbolic
link. One could just copy make, but then other binaries fail as well,
so rather than play whack-a-mole, I opted to take us back to the old
way.  Before r340157 or so we did copies (thogh of a lot fewer
artifacts), and we didn't have issues like this.

Reviewed by: arichards@
Differential Revision: https://reviews.freebsd.org/D25967
2020-08-07 16:26:56 +00:00
Alex Richardson
0d834e0f4d Fix duplicate assignment of _localedef in Makefile.inc1
The same .if exists a few lines below.
2020-08-07 16:04:15 +00:00
Emmanuel Vadot
70fc0cc387 pkgbase: Add the possibility to choose the output dir
The output dir is set to ${REPODIR}/${PKG_ABI}/${PKG_VERSION} now.
Add the possibility to specify the last componant and set it by default
to ${PKG_VERSION} as before.
This is useful for tests and also for building packages with the same
PKG_VERSION provided to check differences.
2020-08-06 16:14:43 +00:00
Emmanuel Vadot
d0327929ab pkgbase: Add a new variable PKG_TIMESTAMP
libarchive uses the SOURCE_DATE_EPOCH env variable to set the date of file
in an archive, this is useful for reproducibility.
Add a variable name PKG_TIMESTAMP that take a epoch time and set SOURCE_DATE_EPOCH
to this. By default it is the current time so no changes here.
2020-08-06 16:13:54 +00:00
Kyle Evans
eb578fec7f Ensure libregex is built in time for googletest
In lib/Makefile, we document the dependency with SUBDIR_DEPEND

For buildworld orchestration, just prebuild libregex if GOOGLETEST is
enabled. googletest will get built in a later pass.
2020-08-04 03:43:28 +00:00
Glen Barber
40056899f6 Move GIT_CMD a few lines down, so it is not in between SVN_*
lines.

Sponsored by:	Rubicon Communications, LLC (netgate.com)
2020-07-16 20:44:04 +00:00
Glen Barber
ff1031c1b7 Add GIT_CMD, similar to SVN_CMD.
Sponsored by:	Rubicon Communications, LLC (netgate.com)
2020-07-16 20:36:22 +00:00
Conrad Meyer
0a9ab9f549 Add WITH_CLANG_FORMAT option
clang-format is enabled conditional on either WITH_CLANG_EXTRAS or
WITH_CLANG_FORMAT.  Some sources in libclang are build conditional on
either rule, and obviously the clang-format binary itself depends on the
rule.

clang-format could still use a manual page.

Reviewed by:	emaste
Differential Revision:	https://reviews.freebsd.org/D25427
2020-06-24 17:03:42 +00:00
Kyle Evans
a19aa4f704 Squash liblzma build race
As of r362452, liblzma depends on libmd but the buildworld build order
hadn't been amended to document the new dependency.

Reported by:	jenkins via freqlabs
X-MFC-With:	r362452
2020-06-22 03:44:01 +00:00
Ryan Moeller
94a8266684 flua: add ucl library
libucl comes with a Lua library binding.  Build it into flua.

This lets us parse/generate config files in the various formats supported by
libucl with flua.  For example, the following script will detect the format of
an object written to stdin as one of UCL config, JSON, or YAML and write it to
stdout as pretty-printed JSON:

local ucl = require('ucl')
local parser = ucl.parser()
parser:parse_string(io.read('*a'))
local obj = parser:get_object()
print(ucl.to_format(obj, 'json'))

Reviewed by:	kevans, pstef
Approved by:	mmacy (mentor)
Relnotes:	yes
Differential Revision:	https://reviews.freebsd.org/D25009
2020-06-22 03:14:43 +00:00
Ed Maste
74e8d41e0a Retire BINUTILS and BINUTILS_BOOTSTRAP options
As of r361857 all BINUTILS options are disabled by default - ports
have been changed to depend on binutils if they require GNU as, and
all base system assembly files have been switched to use Clang's
integrated assembler.

Relnotes:	Yes
Sponsored by:	The FreeBSD Foundation
2020-06-07 00:07:21 +00:00
Ed Maste
5843b6e7b2 Makefile.inc1: remove stale dependency cleanup from 2018 2020-06-05 20:18:55 +00:00
Ed Maste
694034e227 Makefile.inc1: remove BINUTILS_BOOTSTRAP linker support
As of r359347 BINUTILS_BOOTSTRAP does not include the bfd linker.
2020-05-30 16:20:18 +00:00
Ed Maste
697b271da9 pkgbase: use -dev,-dbg instead of -development,-debug
-development is long and awkward, and is also inconsistent with prior art
from the Linux world, which uses -dev (Debian) or -devel (Red Hat).  Follow
the Debian convention, and similarly for debug info packages.

Also remove redundant pkgbase development tag from includes.  We already tag
include files with package=runtime,dev; there is no need to separately tag
them as dev.

Discussed with:	bapt
Reviewed by:	manu
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D24139
2020-05-20 19:45:22 +00:00
Kyle Evans
20076f97fd Revert r360833, r360882: certctl rehash in installworld
This was solving the correct-ish problem in the wrong place. Noted by
brooks; while he didn't request a prompt revert, doing so now will
facilitate proper testing for the revised version of this.
2020-05-18 01:48:55 +00:00
Kyle Evans
a93817e04f buildworld: add back in missing semicolon
if foo; then blah else blah; fi has a drastically different meaning than
if foo; then blah; else blah; fi. Fix it.

Reported by:	0mp
X-MFC-With:	r360833
2020-05-10 20:28:38 +00:00
Kyle Evans
70868d48e8 installworld: attempt a certctl rehash at the tail end
This can be run as root or normal user with no problem; if they hadn't
twisted the WITHOUT_CAROOT knob, we'll attempt to use the host certctl to
rehash the DESTDIR. This would allow one to build systems WITHOUT_OPENSSL +
WITH_CAROOT with a populated /etc/ssl that they can then use with an
appropriate *ssl from somewhere else.

Cross-builds are fine because this will always use the host certctl, or just
nag if it's missing and it wasn't a WITHOUT_CAROOT build.

MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D24641
2020-05-09 02:01:29 +00:00
Dimitry Andric
61fbd61b59 Fix race between prebuilding libsbuf and libgeom
The latter needs the former, but with a multi-job build on a fast
machine, the race is sometimes lost. This leads to "ld: error: unable to
find library -lsbuf", when linking libgeom.so.

Submitted by:	kevans
MFC after:	3 days
2020-04-25 20:24:41 +00:00
Kyle Evans
6fe257c780 Move shm_open dependency cleanup into a new home
r359461 introduced this nifty script to centralize these things, so add
shm_open.c there to remove a total of one (1) bad example from
Makefile.inc1.

Looked over by:	emaste
2020-04-13 19:50:47 +00:00
Maxim Sobolev
a8b8edb25e Normalize deployment tools usage and definitions by putting into one place
instead of sprinkling them out over many disjoint files. This is a follow-up
to achieve the same goal in an incomplete rev.348521.

Approved by:	imp
MFC after:	1 month
Differential Revision:	https://reviews.freebsd.org/D20520
2020-04-07 02:46:22 +00:00
Brooks Davis
f382bac49b Fix compilation with upstream clang builtin headers.
By using -nobuiltininc and adding the clang builtin headers resource dir
to the end of the compiler header search path, we can still find headers
such as immintrin.h but find the FreeBSD version of stddef.h/stdarg.h/..
first.

This is a workaround until we are able to settle on and complete a plan
to harmonize guard macros with LLVM.  We've mostly worked out this on
FreeBSD systems by removing select headers from the installed set of
devel/llvm*, but that isn't a good solution for cross build.

Submitted by:	arichardson
Obtained from:	CheriBSD
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D17002
2020-04-06 23:38:46 +00:00
Brooks Davis
e40fa28f1a Pass the real value of MK_TESTS_SUPPORT to distribution.
This allows kyua's config file to actually be installed.

Reported by:	arichardson
Reviewed by:	arichardson
Sponsored by:	DARPA
Differential Revision:	https://reviews.freebsd.org/D24268
2020-04-03 16:19:07 +00:00
Ed Maste
4dad87a498 add shell script for stale dependency hack
It's rather awkward to debug issues with the dependency cleanup hacks
when implemented via make.  Add a cleanup shell script and move the
libomp hack there as an initial example.

Reviewed by:	brooks
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D24228
2020-03-30 20:20:15 +00:00
Brooks Davis
c42a67671b Relax existence check of ${LOCALBASE}/bin/kyua
After the base kyua import the testsuite can utilize the in-base kyua
just fine.

Submitted by:	Dries Michiels
Differential Revision:	https://reviews.freebsd.org/D24230
2020-03-30 20:04:55 +00:00
Brooks Davis
be2af8fbf4 Use the real value of MK_TESTS_SUPPORT in _libraries.
We need to build kyua libraries for kyua.  Because we set MK_TESTS=no,
we can't not set MK_TESTS_SUPPORT=${MK_TESTS_SUPPORT} because the latter
defaults MK_TESTS_SUPPORT to no.

This fixes WITHOUT_TESTS + WITH_TESTS_SUPPORT builds.

Reported by:	Ruslan Garipov
Reviewed by:	emaste
Sponsored by:	DARPA
Differential Revision:	https://reviews.freebsd.org/D24206
2020-03-27 23:27:54 +00:00
Emmanuel Vadot
eb61f7bdf2 Stop building libl and liby
Those libs aren't needed anymore so stop building them for the bootstrap tools
or the compat libs.

Reviewed by:	bapt, emaste
X-Differential Revision:	D24171
2020-03-26 08:23:09 +00:00
Mitchell Horne
52de22cae7 Makefile.inc1: override MACHINE for native-xtools
For the final step of the native-xtools target, "everything" is built
with TARGET and TARGET_ARCH set to the architecture we wish to
cross-build for. However, CROSSENV overwrites the values of MACHINE and
MACHINE_ARCH, setting them to be identical to TARGET and TARGET_ARCH.
For native-xtools this is undesirable since we are building binaries to
run on the host architecture, that can cross compile for the target
architecture.

When building native-xtools for RISC-V, this issue manifests as an invalid
argument for "-march". The compiler is invoked with the target triple of the
host architecture, but the CFLAGS inherited from bsd.cpu.mk are that of the
target architecture.

Reviewed by:	imp, bdrewery
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D23838
2020-03-24 23:25:54 +00:00
Ed Maste
f1af40b54b Makefile.inc1: remove old (2018) stale dependency hacks
We handle a number of stale dependency issues on an ad-hoc basis, in
order to support ongoing NO_CLEAN builds.  These hacks do not need to
be maintained indefinitely; now remove those for issues that are more
than a year old.

Sponsored by:	The FreeBSD Foundation
2020-03-24 21:37:34 +00:00
Ed Maste
c260dbfb47 Makefile.inc1: set DB_FROM_SRC also for stageworld
stageworld, invoked from e.g. `make packages`, shouldn't rely on the
build host's users/groups.

Sponsored by:	The FreeBSD Foundation
2020-03-22 03:10:31 +00:00
Ed Maste
dadc342d07 Revert r359083, fixed properly by r359088
r359083 introduced a workaround for stale libomp dependencies during a
regular (no -DNO_CLEAN) buildworld.  r359088 addressed the reason the
clean step missed libomp, so revert the workaround.

Sponsored by:	The FreeBSD Foundation
2020-03-18 21:20:10 +00:00
Ed Maste
0d02682c96 invoke _cleanobj_fast_depend_hack unconditionally
Apparently make ${CLEANDIR} is leaving stale entries in .depend files;
for now invoke the hacky cleanup in both the -DNO_CLEAN and normal
(no -DNO_CLEAN) cases.

In collaboration with:	dim
Sponsored by:	The FreeBSD Foundation
2020-03-18 20:12:46 +00:00
Brooks Davis
564b9ff2a7 Add an internal liblua and use it in flua.
The new liblua will be used in a forthcoming import of kyua.

Reviewed by:	kevans
Obtained from:	CheriBSD
Sponsored by:	DARPA
Differential Revision:	https://reviews.freebsd.org/D24090
2020-03-17 17:28:12 +00:00
Ed Maste
dfcef1de85 Makefile.inc1: add a note when deleting stale dependencies
We have ad-hoc stale dependency handling in Makefile.inc1 to handle the
cases where file extensions change, but it appears that some cases are
not functional.  Add a note when about to clean stale deps to help
when investigating failure reports.

Sponsored by:	The FreeBSD Foundation
2020-03-17 14:12:19 +00:00
Ed Maste
450cc34172 Makefile.inc1: move dependency hack comment to the block it applies to 2020-03-12 14:01:17 +00:00
Ed Maste
3b1781b4fd Extend r358907 to explicitly remove stale lib32 dependency
After r325072 stale lib32 dependencies were not remooved.  A more
holistic approach is needed to address this but for the immediate issue
(-DNO_CLEAN builds across r358851) just readd the explicit lib32 path.

Reported by:	dim
Sponsored by:	The FreeBSD Foundation
2020-03-12 13:42:08 +00:00
Dimitry Andric
0efe25f9a5 Allow -DNO_CLEAN build across r358851.
The openmp 10.0.0 import renamed one .c file to .cpp, and this is
something our dependency system does not handle correctly.  Add another
ad-hoc cleanup to get rid of the stale dependency.

PR:		244251
MFC after:	6 weeks
X-MFC-With:	358851
2020-03-12 11:39:04 +00:00
Ed Maste
13f7dbe822 retire amd(8)
autofs was introduced with FreeBSD 10.1 and is the supported method for
automounting filesystems.  As of r296194 the amd man page claimed that it
is deprecated.  Remove it from base now; the sysutils/am-utils port is
still available if necessary.

Discussed with:	cy
Relnotes:	Yes
Sponsored by:	The FreeBSD Foundation
2020-03-09 20:46:43 +00:00
Ed Maste
134b378392 retire in-tree GPL dtc devicetree compiler
Now that we no longer have GCC 4.2.1 in the tree and can assume FreeBSD
is being built with a C++11 compiler available, we can use BSDL dtc
unconditionally and retire the GPL dtc.

GPL dtc now has FreeBSD CI support via Cirrus-CI to help ensure it
continues to build/work on FreeBSD and is available in the ports tree
if needed.

The copy of (copyfree licensed) libfdt that we actually use is in
sys/contrib/libfdt so the extra copy under contrib/dtc/libfdt can be
removed along with the rest of the GPL dtc.

Reviewed by:	kevans, ian, imp, manu, theraven
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D23192
2020-02-29 17:10:54 +00:00
Ed Maste
c45018041d retire the LLVM_LIBUNWIND option
LLVM's libunwind is used on all FreeBSD-supported CPU architectures and
is a required component.

Reviewed by:	brooks (earlier)
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D23123
2020-02-29 12:43:43 +00:00
Ed Maste
57f804675e remove GCC 4.2.1 build infrastructure
As described in Warner's email message[1] to the FreeBSD-arch mailing
list we have reached GCC 4.2.1's retirement date.  At this time all
supported architectures either use in-tree Clang, or rely on external
toolchain (i.e., a contemporary GCC version from ports).

GCC 4.2.1 was released July 18, 2007 and was imported into FreeBSD later
that year, in r171825.  GCC has served us well, but version 4.2.1 is
obsolete and not used by default on any architecture in FreeBSD.  It
does not support modern C and does not support arm64 or RISC-V.

Thanks to everyone responsible for maintaining, updating, and testing
GCC in the FreeBSD base system over the years.

So long, and thanks for all the fish.

[1] https://lists.freebsd.org/pipermail/freebsd-arch/2020-January/019823.html

PR:		228919
Reviewed by:	brooks, imp
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D23124
2020-02-29 03:25:51 +00:00
Dimitry Andric
eb834d9f1b Take LINKER_FREEBSD_VERSION from numerical field after dash
Summary:
With COMPILER_FREEBSD_VERSION, we use a numeric value that we bump each
time we make a change that requires re-bootstrapping, but with the
linker variant, we instead take the entire part after "FreeBSD", as in
this example version output:

LLD 9.0.1 (FreeBSD c1a0a213378a458fbea1a5c77b315c7dce08fd05-1300006) (compatible with GNU linkers)

E.g., LINKER_FREEBSD_VERSION is currently being set to
"c1a0a213378a458fbea1a5c77b315c7dce08fd05-1300006".  This means that
*any* new upstream lld version will cause re-bootstrapping.

We should only look at the numerical field we append after a dash
instead.  This review attempts to make it so.

The only thing I am not happy about is the post-processing of awk output
in Makefile.inc1.  I notice that our awk does not have gensub(), so it
can't substitute a numbered sub-regex with \1, \2, etc.  Suggestions
welcome. :)

MFC after:	1 week
Differential Revision: https://reviews.freebsd.org/D23691
2020-02-19 21:12:59 +00:00
Warner Losh
75ababf8b5 Restore missing comment
I was overly agressive about removing the entire comment. It was still
valid, except the part about being only for some architectures.

Reviewed by: emaste
Differenial Revision: https://reviews.freebsd.org/D23523
2020-02-06 17:51:48 +00:00
Warner Losh
5672c76477 Per the firm plan, start to remove sparc64
The sparc64 architecture is being removed from FreeBSD 13, starting
now. This removes it from the top level only. It is the only
architecture that didn't see substantial work after the call to get
things working with the external toolchain.
2020-02-02 11:37:27 +00:00
Ed Maste
43e8403953 retire BSD_CRTBEGIN option
BSD crt is currently used on all architectures (other than sparc64).
Remove the option and use BSD crt everywhere as part of the GCC 4.2.1
retirement plan.

https://lists.freebsd.org/pipermail/freebsd-arch/2020-January/019823.html

PR:		239851
Reviewed by:	andrew, brooks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D23122
2020-01-31 18:04:04 +00:00
John Baldwin
7ec5e1c4cd Remove support for auto-selecting an external binutils.
All of the in-tree architectures not supported by in-tree binutils are
supported by lld, so the condition is now always false.  It also
didn't fully work since the external binutils are installed into a
directory that uses the host's OS version, not the target OS version.

Reviewed by:	emaste, imp
Differential Revision:	https://reviews.freebsd.org/D23294
2020-01-22 21:21:24 +00:00
Ed Maste
9ccace3f7c pkgbase: allow the pkg format to be overridden
Compressing .txz packages can be rather slow, and speed is likely more
important than disk space during development.  Allow package format to
be set via PKG_FORMAT make variable.

Reviewed by:	bapt
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D23257
2020-01-19 14:46:28 +00:00
Ed Maste
c530c7915b pkgbase: cache pkg ABI for all world/kernel packages
Rather than invoking `pkg config ABI` repeatedly.

Reviewed by:	manu
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D23255
2020-01-18 22:58:32 +00:00
Kyle Evans
4f47920e9c Makefile.inc1: push /usr/libexec into the BPATH/TMPPATH
${WORLDTMP}/legacy/usr/libexec will only have libexec/ bits that we've
pushed as bootstrap tools, so this is generally safe to include prior to
PATH. The following are the ramifications of this change:

- BPATH addition gets us at least bootstrap flua in WMAKEENV path for
  buildenv, for those earlier systems where it's bootstrapped still

- Reworked the sysent target to just set PATH and let it get worked out in
  src.lua.mk or individual sysent makefiles -- this gives us back the
  ability to overwrite LUA_CMD and use a different/external lua for these
  targets.  sysent can also now work cleanly in buildenv.

- tools/build/Makefile will now symlink the host flua into build's host
  tools so that the above can work without needing to add the host's
  /usr/libexec explicitly into TMPPATH.

Reviewed by:	arichardson, brooks, imp (all slightly earlier version)
Differential Revision:	https://reviews.freebsd.org/D22464
2020-01-12 04:18:36 +00:00
Ed Maste
7c88377010 Remove prereq lib support for GNU unwinder
LLVM's libunwind is used for all archs, as of r356514.  As GCC 4.2.1
will soon be removed, and with it the GNU unwinder, start simplifying
the prereq/startup lib logic in Makefile.inc1.  From here if there are
any unwinder bugs (on any arch) the path foward is to fix LLVM's
libunwind.
2020-01-08 20:56:30 +00:00
Kyle Evans
cd0d51baaa Provide libssp based on libc
For libssp.so, rebuild stack_protector.c with FORTIFY_SOURCE stubs that just
abort built into it.

For libssp_nonshared.a, steal stack_protector_compat.c from
^/lib/libc/secure and massage it to maintain that __stack_chk_fail_local
is a hidden symbol.

libssp is now built unconditionally regardless of {WITH,WITHOUT}_SSP in the
build environment, and the gcclibs version has been disconnected from the
build in favor of this one.

PR:		242950 (exp-run)
Reviewed by:	kib, emaste, pfg, Oliver Pinter (earlier version)
Also discussed with:	kan
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D22943
2020-01-04 20:19:25 +00:00
Kyle Evans
36afd1d2cb Makefile.inc1: remove libcompiler_rt from _startup_libs
libcompiler_rt is already present in _prereq_libs, which will get built just
before _startup_libs and in advance of shared libc. Dedupe.
2020-01-03 21:40:32 +00:00
John Baldwin
39eb07f172 Look for cross toolchain makefiles in /usr/share/toolchains.
The freebsd-binutils and freebsd-gcc* packages install toolchain
makefiles to /usr/share/toolchains rather than LOCALBASE.

Reviewed by:	bapt
Differential Revision:	https://reviews.freebsd.org/D22985
2020-01-02 21:34:44 +00:00
Warner Losh
eb4977bd0f Remove arm/arm as a valid target.
TARGET=arm now defaults to TARGET_ARCH=armv7
TARGET_ARCH=arm is no longer valid.

Bump __FreeBSD_version to 1300073

Tested with make universe. Any stale LINT-V5 config files remaining in the tree
will fail the universe build. However, LINT-V5 was removed in r355119.

This retirement has been planned since last summer. The armv5 port is fragile:
it works OK for some peeople, and fails badly for others. There's a number of
subtle bugs in busdma, pmap and other MD parts of thee system that present
themselves under load or in unusual circumstances (like fsck after a
crash). stable/8, branched 10 years ago, was the last reliable release.  Since
the support burden is larger then the benefit, the consensus view is armv5
should be removed from the tree.

Discussed with: arm@ mailing list and arm developer community.
2020-01-02 03:25:26 +00:00
Ed Maste
f6e159c6fc revert r355609 2019-12-13 14:48:44 +00:00
Ed Maste
c039e9596f libpmc: build json event support also on arm64 2019-12-12 00:14:01 +00:00
Ed Maste
0ecf09af8c connect arm linuxulator to top-level make sysent
Reported by:	kevans
2019-12-11 17:48:34 +00:00
Ed Maste
c590fd27de Make NOCLEAN an error instead of a warning
The warning was added in r289728 (over four years ago) and at that time
NO_CLEAN was already the correct spelling for over a decade.

Make NOCLEAN an error as the next step to removing these backward
compatibility shims.
2019-12-11 14:54:29 +00:00
Ed Maste
3e10556208 Update Makefile.inc1 dtc comment
We use the BSDL dtc by default now (as long as we're using a C++11
compiler).
2019-11-26 14:25:50 +00:00
Bryan Drewery
97900b44f6 WITH_SYSTEM_LINKER: Fix rebuilding lld every time.
This is due to LLD_REVISION_STRING being renamed to LLD_REVISION in
r351442 and the value being moved to another location in r351965.

`make test-system-linker` can be used to see the values being used here.

Reported by:	ler
2019-11-19 16:40:46 +00:00
Kyle Evans
f22a592111 Convert in-tree sysent targets to use new makesyscalls.lua
flua is bootstrapped as part of the build for those on older
versions/revisions that don't yet have flua installed. Once upgraded past
r354833, "make sysent" will again naturally work as expected.

Reviewed by:	brooks
Differential Revision:	https://reviews.freebsd.org/D21894
2019-11-18 23:28:23 +00:00
John Baldwin
9d919726a0 Refine r354661 to unbreak the GCC_BOOTSTRAP case.
MK_CLANG_IS_CC controls installing links for GCC, not just clang.  Set
MK_CLANG_IS_CC to the value of MK_CLANG_BOOTSTRAP.  This will leave it
as "no" if no bootstrap compiler is being built or GCC 4.2.1 is being
used as the bootstrap compiler, and "yes" if clang is being used as
the bootstrap compiler.

Submitted by:	bdrewery (kind of, he suggested this on IRC while I was
	  		  testing the original patch)
Reviewed by:	kevans, imp
Sponsored by:	DARPA
Differential Revision:	https://reviews.freebsd.org/D22350
2019-11-13 21:49:46 +00:00
John Baldwin
a99d99e5db Force MK_CLANG_IS_CC on in XMAKE.
This ensures that a bootstrap clang compiler is always installed as cc
in WORLDTMP.  If it is only installed as 'clang' then /usr/bin/cc is
used during the build instead of the bootstrap compiler.

Reviewed by:	imp
MFC after:	1 month
Sponsored by:	DARPA
Differential Revision:	https://reviews.freebsd.org/D22332
2019-11-12 21:29:52 +00:00
Ed Maste
c723af1169 suggest xtoolchain package if binutils and GCC bootstraps are both broken
Previously we checked for only BINUTILS_BOOTSTRAP as a broken option
and suggested installing the binutils package.  This was originally done
for arm64 where we used the in-tree Clang with external binutils package.

Add a case to the warning to suggest instead the full xtoolchain package
if we have no in-tree compiler either.

Differential Revision:	https://reviews.freebsd.org/D21851
2019-11-08 14:06:48 +00:00
Brooks Davis
55c2583a16 Revert r354518 and commit the intented fix rather than the diagnostic
check.

This fixes the definition of MK_LIB32 in Makefile.inc1.
2019-11-07 23:54:40 +00:00
Brooks Davis
99519f2120 Fix the ARCH check for LIB32 from Makefile.inc1. 2019-11-07 23:50:33 +00:00
Brooks Davis
a4330302f2 libcompat: build 32-bit rtld and ldd as part of "everything"
Alter bsd.compat.mk to set MACHINE and MACHINE_ARCH when included
directly so MD paths in Makefiles work. In the process centralize
setting them in LIBCOMPATWMAKEENV.

Alter .PATH and CFLAGS settings in work when the Makefile is included.

While here only support LIB32 on supported platforms rather than always
enabling it and requiring users of MK_LIB32 to filter based
TARGET/MACHINE_ARCH.

The net effect of this change is to make Makefile.libcompat only build
compatability libraries.

Changes relative to r354449:

Correct detection of the compiler type when bsd.compat.mk is used
outside Makefile.libcompat.  Previously it always matched the clang
case.

Set LDFLAGS including the linker emulation for mips where -m32 seems to
be insufficent.

Reviewed by:	imp, kib (origional version in r354449)
Obtained from:	CheriBSD (conceptually)
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D22251
2019-11-07 22:58:10 +00:00
Brooks Davis
ccad77fe1e Revert r354449: libcompat: build 32-bit rtld and ldd as part of "everything"
Additional testing is required..
2019-11-07 19:22:51 +00:00
Brooks Davis
36712a9497 libcompat: build 32-bit rtld and ldd as part of "everything"
Alter bsd.compat.mk to set MACHINE and MACHINE_ARCH when included
directly so MD paths in Makefiles work. In the process centralize
setting them in LIBCOMPATWMAKEENV.

Alter .PATH and CFLAGS settings in work when the Makefile is included.

While here only support LIB32 on supported platforms rather than always
enabling it and requiring users of MK_LIB32 to filter based
TARGET/MACHINE_ARCH.

The net effect of this change is to make Makefile.libcompat only build
compatability libraries.

Reviewed by:	imp, kib
Obtained from:	CheriBSD (conceptually)
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D22251
2019-11-07 17:10:33 +00:00
Simon J. Gerraty
17adf17b30 Building head on stable/11 requires libzstd
Add lib/libzstd to _elftoolchain_libs

tools/build/Makefile needs to create the install dir for libzstd
Since this would make the line too long, rework to use a list
in one per line format (easier to add in future)
and dispense with the .for loop

Reviewed by:	emaste bapt
Differential Revision:	https://reviews.freebsd.org/D220134
2019-10-28 20:45:29 +00:00
John Baldwin
4196949c01 Strip "sf" suffix when generating a target triple.
This fixes the target triple used when compiling riscv64sf with clang.

Discussed with:	mhorne
MFC after:	2 weeks
Sponsored by:	DARPA
2019-10-23 16:43:51 +00:00
Ed Maste
dfe76e95ae Additional fix for -DNO_CLEAN build across r353340 and r353381
opensolaris_atomic.S is now only used on i386 with opensolaris_atomic.c
used on other platforms.  After r353381 it doesn't exist on those
platforms so the stale dependency would result in a build error.

r353408 addressed this issue for cddl/lib/libzpool, but it persisted
with the opensolaris and zfs modules.
2019-10-21 18:40:03 +00:00
Andriy Gapon
01a69565d8 buildkernel: always add standard kernel configuration include path
This should change nothing for kernel configurations at the standard
locations in the source tree.  However, if KERNCONFDIR is used to
specify a custom location for a kernel configuration file (e.g., out of
tree), then both the custom location and the standard location, in this
order, will be used as include paths for config(8).  This will allow the
kernel configuration to include files from both locations.

Reviewed by:	bdrewery
MFC after:	16 days
Differential Revision: https://reviews.freebsd.org/D22057
2019-10-19 07:16:20 +00:00
Brooks Davis
5dab131339 Build compat libraries before "everything".
This is required for us to link programs against compat versions of
libraries.

Reviewed by:	bdrewery, jhb
Sponsored by:	DARPA, AFRL
2019-10-15 21:22:13 +00:00
Brooks Davis
ad2dd70b44 Rename top-level LIBCOMPAT to _LIBCOMPAT.
This avoids a conflict with LIBCOMPAT defined in bsd.libnames.mk.

Reviewed by:	bdrewery
Sponsored by:	DARPA, AFRL
2019-10-15 21:11:22 +00:00
Brooks Davis
a1ff92a580 Fix -DNO_CLEAN build across r353340 and r353381
opensolaris_atomic.S is now only used on i386 with opensolaris_atomic.c
used on other platforms.  After r353381 it doesn't exist on those
platforms so the stale dependency would result in a build error.
2019-10-10 16:29:13 +00:00
Dimitry Andric
9122aeeaa6 Merge ^/head r353316 through r353350. 2019-10-09 16:40:22 +00:00
Brooks Davis
0ed89481e1 Allow -DNO_CLEAN build across r352689.
Split the LIBCOMPAT case because the usual egrep only matches in
LIBCOMPAT on amd64.
2019-10-08 20:26:51 +00:00
Dimitry Andric
8b3bc70a2b Merge ^/head r352764 through r353315. 2019-10-08 18:17:02 +00:00
Kyle Evans
c1d4532062 Add a top-level makeman target
Abstracting away the details of how src.conf(5) is generated is arguably a
good thing; do so with a top-level makeman target.
2019-10-01 15:10:35 +00:00
Brooks Davis
f9fd94f534 Update cloudabi(32|64) sysents with "make sysent".
MFC after:	1 week
2019-09-30 21:14:33 +00:00
John Baldwin
c6b00391b3 Disable build of LOCAL_MODULES for cross-builds by default.
WITHOUT_LOCAL_MODULES can be set to disable LOCAL_MODULES for native
builds.  WITH_LOCAL_MODULES can be set to leave it enabled for cross
builds.

This does not use a knob in kern.opts.mk because the options framework
does not currently support options whose default varies on the build
type.  I discussed a few options there with Warner (e.g. maybe having
a tri-state where the default value is "auto" and having Makefile.inc1
apply logic when MK_LOCAL_MODULES is set to "auto"), but Warner ok'd
this approach for now until a better solution is implemented.

Requested by:	many
Reviewed by:	imp (in person at EuroBSDCon)
Differential Revision:	https://reviews.freebsd.org/D21608
2019-09-28 14:20:28 +00:00
Dimitry Andric
668ee10168 Merge ^/head r352587 through r352763. 2019-09-26 18:25:54 +00:00
Kyle Evans
9ccd2fde4c Adjust Makefile.inc1 syscall sub commit 2019-09-25 18:04:09 +00:00
Kyle Evans
3e25d1fb61 Add linux-compatible memfd_create
memfd_create is effectively a SHM_ANON shm_open(2) mapping with optional
CLOEXEC and file sealing support. This is used by some mesa parts, some
linux libs, and qemu can also take advantage of it and uses the sealing to
prevent resizing the region.

This reimplements shm_open in terms of shm_open2(2) at the same time.

shm_open(2) will be moved to COMPAT12 shortly.

Reviewed by:	markj, kib
Differential Revision:	https://reviews.freebsd.org/D21393
2019-09-25 18:03:18 +00:00
Dimitry Andric
f05b9584fa Merge ^/head r352537 through r352586. 2019-09-21 21:02:57 +00:00
Baptiste Daroussin
af3ccc0740 Ensure libthr is always built before libprivatezstd when building the
startup libs

Reported by:	"Galazka, Krzysztof" <krzysztof.galazka@intel.com>
2019-09-20 09:45:38 +00:00
Dimitry Andric
0f80acb965 Merge ^/head r352436 through r352536. 2019-09-19 19:26:12 +00:00
Baptiste Daroussin
d567f909ef Add native support for zstd to libarchive
Note that old pkg will failed to build after this. A recent ports tree (one
providing pkg 1.12+) is required to build. Older already built pkg, should
continue working as expected

PR:		238797
Exp run by:	antoine
Reviewed by:	cem
Approved by:	cem
Differential Revision:	https://reviews.freebsd.org/D20752
2019-09-18 07:57:56 +00:00
Dimitry Andric
84ee6de9ea Always build lldb-tblgen, for now, and add a note. 2019-09-02 17:30:00 +00:00
Dimitry Andric
a32f82c2c1 Update build glue for a build of clang plus lldb (enabled via MK_LLDB). 2019-08-26 21:00:14 +00:00
Antoine Brodin
1321298390 Add head(1) to native-xtools so that it can be used in qemu-user jails 2019-06-20 13:24:58 +00:00
Rene Ladan
0026fec57d Differentiate package versions for ALPHA/BETA/PRERELEASE/RC phases.
Currently APLHA packages are treated as CURRENT or STABLE versions,
resulting in e.g. 13.0.s20190615125609. This version number is indeed
different from the next version number but ALPHA2 would be nicer IMO.

For the BETA, PRERELEASE and RC phases the packages are versioned the
same as for releases, so 11.3-BETA1 is 11.3 and so is 11.3-RC1, meaning
that pkg cannot easiliy upgrade from the former the next. This happened
on my Raspberry Pi which runs pkgbase.

Submitted by:	rene
Approved by:	manu
Event:		Berlin hackathon 2019
Differential Revision:	https://reviews.freebsd.org/D20651
2019-06-16 11:53:22 +00:00
Ed Maste
cbc493c662 Makefile.inc1: clean up stale dependency hacks
Carrying on from r339607, remove ad-hoc dependency cleanup for changes
prior to June 2018.

Sponsored by:	The FreeBSD Foundation
2019-06-04 12:56:20 +00:00
Maxim Sobolev
e1216db508 Rollback the rest of the botched r348521. Re-work would be posted to
reviews. Sorry.

Reported by:	Enji Cooper
2019-06-03 15:34:00 +00:00
Maxim Sobolev
5d280640cd Leave mtree hardcoded for now. Reverting partially 348521 and also
the followup stopgap change, because I don't think it's a correct. I still
need to figure out where to stick it in. In cannot be in Makefile.inc1
and it cannot be in etc/Makefile from the looks of it to avoid
chicken-and-egg problem.
2019-06-03 15:12:44 +00:00
Warner Losh
a88c8ae798 Stopgap fix to breakage from r348521.
MTREE is defined, while MTREE_CMD isn't, use it instead.
This may not be 'right' but it fixes the CI and other issues.

Submitted by: Andreas Nilsson
2019-06-03 14:14:53 +00:00
Maxim Sobolev
5ec57af4b2 Fix several places where tool name has been hardcoded:
install -> ${INSTALL}
    mtree -> ${MTREE_CMD}
    services_mkdb -> ${SERVICES_MKDB_CMD}
    cap_mkdb -> ${CAP_MKDB_CMD}
    pwd_mkdb -> ${PWD_MKDB_CMD}
    kldxref -> ${KLDXREF_CMD}

If you do custom FreeBSD builds you may want to override those
in some cases.

Sponsored by:	Sippy Software, Inc.
2019-06-02 23:38:19 +00:00
Justin Hibbits
78473c580b Update __FreeBSD_version and Makefile check for r348347
libdwarf needs forcibly rebuilt after r348347.
2019-05-29 02:26:15 +00:00
Baptiste Daroussin
7fc5693f49 Change package the MTREEFLAGS and INSTALLFLAGS missed in r348185
Reported by:	lwhsu
2019-05-23 18:55:48 +00:00
Brad Davis
871b6b10ce Move master.passwd and group to lib/libc/gen/
libc was picked as the destination location for these because of the syscalls
that use these files as the lowest level place they are referenced.

Approved by:	will (mentor), rgrimes, manu
Differential Revision:	https://reviews.freebsd.org/D16728
2019-05-16 01:09:13 +00:00
Antoine Brodin
9c8f66cb8b Add wc(1) to native-xtools so that it can be used in qemu-user jails 2019-05-08 15:17:36 +00:00
Emmanuel Vadot
8ae9b1fe96 pkgbase: Use uname as ABI_FILE
Add some more missed in r345159

MFC after:	1 week
2019-03-27 17:55:39 +00:00
Wolfram Schneider
656898ab98 Improve english grammar for kernel built time
PR:		224433
Approved by:	jhb
Differential Revision:	https://reviews.freebsd.org/D19636
2019-03-19 19:58:40 +00:00
Wolfram Schneider
1d3a067e85 `make buildkernel' should display the build time in seconds
PR:		224433
Approved by:	cem
Differential Revision:	https://reviews.freebsd.org/D13910
2019-03-16 20:02:57 +00:00
Emmanuel Vadot
08df8f82b8 pkgbase: Use uname as ABI_FILE
uname is always rebuild on FreeBSD so use this as ABI_FILE for pkg when
building pkg for pkgbase.
pkg uses uname too as default ABI_FILE as of commit d8bbf980b7f6f424fb7cc672c23ab2dfc82b6599
d8bbf980b7

Discussed with:	bapt
MFC after:	1 week
2019-03-14 22:08:09 +00:00
Wolfram Schneider
328dcc68a6 `make buildworld' should display the build time in seconds
PR:		224433
Reviewed by:	emaste, @bdrewery
Approved by:	cem
Differential Revision: https://reviews.freebsd.org/D13911
2019-03-12 07:40:38 +00:00
Mateusz Guzik
92dec625e3 Remove iBCS2: plug a wart missed in r342242
Sponsored by:	The FreeBSD Foundation
2018-12-19 22:08:51 +00:00
Ed Maste
466ed83b6c Makefile.inc1: update stale wpa dependency removal statement
Only stale .depend files are removed; do not mention object files.
2018-12-12 15:23:40 +00:00
Ed Maste
457e6311ad Clean stale wpa dependencies and objects after r341759
The wpa update added some source files with the same name as a file in
another directory (found via .PATH in the previous version).  Having a
stale entry in a .depend file means the new file won't be built, so test
for this case and if found remove all of wpa's dependency files.

MFC with:	r341759
Sponsored by:	The FreeBSD Foundation
2018-12-10 14:50:11 +00:00
Alex Richardson
c10d927c40 Fix newvers.sh with BUILD_WITH_STRICT_TMPPATH=1
newvers.sh runs mkfifo which did not exist before this change.
However, I didn't notice before because it is run from a function
where a missing command does cause a noticeable failure.

Reviewed By:	emaste, markj
Differential Revision: https://reviews.freebsd.org/D18377
2018-12-05 10:57:57 +00:00
Andrew Turner
7dc61ac11f Only gnu/lib/csu when MK_BSD_CRTBEGIN is off.
We were still building it from Makefile.inc1. Disable it there so we don't
try to build the GNU crtbegin/crtend when the BSD version was asked for.

PR:		233733
Reported by:	lwhsu
Reviewed by:	emaste
MFC with:	r339738
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D18428
2018-12-04 18:51:28 +00:00
Alex Richardson
6edf284675 Fix -DNO_CLEAN amd64 build after r340463
Without this change I got the following error:
clang-7: error: no such file or directory: '..../lib/libc/amd64/string/bzero.S'

Reviewed By:	mjg
Differential Revision: https://reviews.freebsd.org/D18031
2018-11-18 19:55:03 +00:00
Glen Barber
8643808a68 In followup to r340406, remove doc from the EXTRA_DISTRIBUTIONS
list and remaining references from the script used to create the
MANIFEST file used by bsdinstall(8).

No MFC is planned at this time.

Sponsored by:	The FreeBSD Foundation
2018-11-13 19:53:02 +00:00
Brooks Davis
0a9fcf3237 Add a top-level make target to rebuild all sysent files.
The sysent target is useful when changing makesyscalls.sh, when
making paired changes to syscalls.master files, or in a future where
freebsd32 sysent entries are built from the default syscalls.master.

Reviewed by:	bdrewery
Obtained from:	CheriBSD
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D17899
2018-11-08 00:35:00 +00:00
Alex Richardson
cd6b0f0ea5 Turn off BUILD_WITH_STRICT_TMPPATH by default
Building with a strict $PATH (without inheriting from the parent
environment) still causes build failures in some workflows/environemnts
that I have not yet tested.
I will try to bring this back once these issues have all been resolved
since it is actually extremely useful in tracking broken dependencies
and wrong assumptions about the build environemt.

Discussed With:	brooks
2018-11-06 18:06:52 +00:00
Alex Richardson
4805dd68d6 Remove btxld from symlinked host tools
It is only present on amd64/i386 systems which breaks buildworld on
other hosts. In fact there is no need to add it to the bootstrap tools
list since it is already included in the cross-tools phase.
However, for cross-tools it was only built if the host and target
architecture didn't match. After this change it is also built when we
are builtin with a strict $PATH.

Reported By:	mmel
2018-11-06 09:36:59 +00:00
Alex Richardson
d3384db33c Keep inheriting $PATH when using system linker/compiler
I missed this case when testing r340157. For now just keep
$PATH when we aren't bootstrapping a compiler so that the build
can find cc/c++/ld without an absolute path.

Reported by:	yuripv
2018-11-05 22:51:44 +00:00
Alex Richardson
6806504da4 Build the elftoolchain libraries as part of bootstrap-tools
It is not necessary to build libelf and libdwarf this early. Furthermore,
when building on Linux/MacOS, m4 will only be built during the bootstrap
tools phase and not be available in $PATH before.

Reviewed By:	emaste
Approved By:	brooks (mentor)
Differential Revision: https://reviews.freebsd.org/D17800
2018-11-05 19:51:16 +00:00
Alex Richardson
8f62bca488 Allow building world without inheriting $PATH
Inheriting $PATH during the build phase can cause the build to fail when
compiling on a different system due to missing build tools or incompatible
versions somewhere in $PATH. This has cause build failures for us before
due to the jenkins slaves still running FreeBSD 10.
Listing the tools we depend on explicitly instead of just using whatever
happens to be in $PATH allows us to check that we don't accidentally add a
new build dependency.

All tools that do no need to be bootstrapped will now be symlinked to
${WORLDTMP}/legacy/bin and during the build phase $PATH will only contain
${WORLDTMP}. There is also a new variable "BOOTSTRAP_ALL_TOOLS" which can
be set to force compiling almost all bootstrap tools instead of symlinking
them. This will not bootstrap tools such as cp,mv, etc. since they may be
used during the build and for those we should really only be using POSIX
compatible options.

Furthermore, this change is required in order to be able to build on
non-FreeBSD hosts. While the same binaries may exist on Linux/MacOS they
often accept different flags or produce incompatible output.

Approved By:	brooks (mentor)
Differential Revision: https://reviews.freebsd.org/D16815
2018-11-05 19:51:10 +00:00
Ed Maste
9117dffc3c Avoid binutils test in cleandir/obj stages
Further to r339946 skip the .error case for lack of in-tree binutils
support when we haven't set the toolchain feature variables.

Discussed with:	arichardson
Sponsored by:	The FreeBSD Foundation
2018-10-31 18:08:12 +00:00
Alex Richardson
d6be9fdde5 Don't run cc --version during cleandir/obj stages
This will no work when there is no cc in $PATH (which is the case before the
cross-tools stage once we no longer inherit $PATH in $WMAKE).
The variables set by bsd.compiler.mk/bsd.linker.mk are not needed in these
stages so this avoids a little bit of makefile parsing.

Reviewed By:	emaste
Approved By:	jhb (mentor)
Differential Revision: https://reviews.freebsd.org/D16814
2018-10-31 10:45:28 +00:00
Ed Maste
f4cbc81dd8 Update elftoolchain bootstrapping test after r339083
Sponsored by:	The FreeBSD Foundation
2018-10-26 01:13:20 +00:00
Ed Maste
5acedb55c0 sort {delete,check}-old* output
It is more convenient to review old libraries, files, and directories
in order.

Sort check-* after checking for existence of files for efficiency, and
because /usr/lib/debug entries are added while iterating over the list.

Reviewed by:	brooks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D17649
2018-10-23 13:00:11 +00:00
Alex Richardson
74f6548619 Only compute the X_COMPILER_*/X_LINKER_* variables when needed
When building CheriBSD we have to set XLD/XCC/XCFLAGS on the command line.
This triggers the $XCC != $CC case in bsd.compiler.mk (and the same for LD
in bsd.linker.mk) which causes it to call ${XCC} --version and
${XLD} --version (plus various awk+sed+echo calls) in every subdirectory.
For incremental builds and stages that only walk the source tree this is
often the majority of the time spent in that directory.

By only computing the value of the X_COMPILER_*/X_LINKER_* variables if
_WANT_TOOLCHAIN_CROSS_VARS is set we can reduce the number of cc/ld calls
to once per build stage instead of once per recursive make.

With this change (and no changes to the sources) the `make includes` stage
now takes 28 seconds at -j1 instead of 86 seconds.

Approved By:	brooks (mentor)
Differential Revision: https://reviews.freebsd.org/D17046
2018-10-23 06:31:25 +00:00
Ed Maste
730a7ce9d6 Makefile.inc1: clean up stale dependency hacks
Our dependency tracking cannot directly cope with certain source tree
changes, particularly with respect to removing or moving source files or
replacing generated files.  We have a collection of ad-hoc workarounds
to handle these cases.  As there is a (small) build-time cost inherent
in these workarounds, we do not want to keep them indefinitely.  Thus,
remove workarounds from 2017.

Sponsored by:	The FreeBSD Foundation
2018-10-22 18:40:21 +00:00
Ed Maste
fc191b1111 Introduce src.conf knob to build userland with retpoline
WITH_RETPOLINE enables -mretpoline vulnerability mitigation in userland
for CVE-2017-5715.

Reported by:	Peter Malcom
Reviewed by:	markj
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D17421
2018-10-21 00:27:59 +00:00
Glen Barber
b958317950 - Update head to 13.0-CURRENT.
- Bump MACHINE_TRIPLE, TARGET_TRIPLE, FBSD_MAJOR, FBSD_CC_VER,
  FREEBSD_CC_VERSION, OS_VERSION.
- Update comment in UPDATING regarding debugging options.
- Remove debug.witness.trace=0 from installation media.
- Bump __FreeBSD_version.

Approved by:	re (implicit)
Sponsored by:	The FreeBSD Foundation
2018-10-19 00:37:47 +00:00
Ed Maste
bfe5c1dcd4 Makefile.inc1: clean up dependencies after r339348
r339348 switched bcopy from .s to .c.  Add ad-hoc dependency cleanup
as done for similar cases.

Approved by:	re (kib)
Sponsored by:	The FreeBSD Foundation
2018-10-14 00:29:57 +00:00
Dag-Erling Smørgrav
e4a41be544 Replace libldns's bootstrap dependency on libcrypto with one on libssl
(which in turn has a bootstrap dependency on libcrypto).

Submitted by:	jkim
Approved by:	re (gjb)
2018-10-12 13:01:17 +00:00
Dag-Erling Smørgrav
d97bf51914 Move libssl up in the bootstrap order.
Submitted by:	jkim
Approved by:	re (gjb)
2018-10-12 05:42:38 +00:00
Ed Maste
539a27e105 Switch ntp's embedded libevent to 2.1.18
For OpenSSL 1.1.1 compatibility.

In Makefile.inc1 add (to the existing similar cases) a hack to handle
dependencies across the migration.

Reviewed by:	jhb
Approved by:	re (gjb)
Sponsored by:	The FreeBSD Foundation.
Differential Revision:	https://reviews.freebsd.org/D17481
2018-10-09 18:35:45 +00:00
John Baldwin
0c0c965a8f Re-enable kernel modules for the MALTA64EL kernel configuration.
Update the BOOTSTRAPPING check for libelf to require the fix for
mips64el object files committed in r338478 and re-enable kernel
modules in the MALTA64EL config file.

Reviewed by:	emaste
Approved by:	re (gjb)
Differential Revision:	https://reviews.freebsd.org/D17054
2018-09-06 19:21:31 +00:00
Li-Wen Hsu
26ffc1ab46 Fix 'install: symlink usr/src/sys -> /sys: File exists' in distributeworld
Follow r334617, specify ${DISTDIR} (by ${INSTALL_DDIR}), '/base' and add
${INSTALLFLAGS} while installing the '/sys' symbolic link.

Reviewed by:	bapt (earlier version), markj
Approved by:	re (gjb), markj (mentor)
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D16877
2018-08-28 23:56:52 +00:00
Alex Richardson
c97e51679b Only bootstrap localedef if ${MK_LOCALES} != "no"
During the build it is only used by share/ctypedef and share/colldef
which will not be built if ${MK_LOCALE} == "no". This saves a tiny bit
of time when building without locales.

Approved By:	jhb (mentor)
2018-08-23 18:19:21 +00:00
Alex Richardson
4acc8a67ba Don't create directories in ${WORLDTMP}/legacy with mtree
This has two advantages:
1) We no longer create lots of empty directories that are not needed
2) This is a requirement for building on non-FreeBSD hosts since mtree will
only exist after the bootstrap-tools phase there.

Aproved By:	jhb (mentor)
Differential Revision: https://reviews.freebsd.org/D16773
2018-08-20 10:39:53 +00:00
Kyle Evans
7920ad944b libbe(3): Move build goop back out of cddl/
Some background: in the GSoC project, libbe/Makefile lived in lib/libbe. I
created projects/bectl branch, maintained the above for all of five
minutes before I misread Makefile.inc1 and decided that it couldn't possibly
build outside of cddl/, so I kicked the Makefile out into the cddl/ build
and all was good. The misreading was of the bit where .WAIT is added to
SUBDIR after lib, libexec but prior to building bin and cddl *only during
the install targets*, which is the critical part.

Fast forward- buildworld was still broken in my branch unbeknownst to me
because I didn't nuke my OBJDIR. Combing through Makefile.inc1 eventually
revealed the necessary magic to make sure that libbe's dependencies are
specified well enough, and it becomes clear what needs done to make a
non-cddl/ build work. This is an interesting prospect, because the build
split is kind of annoying to work with.

IGNORE_PRAGMA is added to avoid dropping WARNS by one more. This was
previously pulled in via cddl/Makefile.inc.
2018-08-18 03:20:59 +00:00
Bjoern A. Zeeb
49f1692a3b METALOG, unless manually overwritten, is defined as ${DESTDIR}/${DISTDIR}/METALOG
In the create-world-packages target we manually piece this together (unless
it is undefined), without the DISTDIR.  Normally DISTDIR is empty (unset) and
no one notices.  Now DISTDIR is a well known long-standing PORTS environment
variable and if that is set in the local environment the path to METALOG
is wrong as it no longer is ${DESTDIR}/METALOG.

Long-term we should start to avoid "publicly well known" names for global
variables, for now just piece ${DISTDIR} in as well.  This allows
create-world-packages to continue if DISTDIR is set in the env.
2018-08-17 21:19:18 +00:00
Roger Pau Monné
2502c66bbd build: skip the database check when generating install media
There are several scripts and targets solely used to generate install
media, make sure DB_FROM_SRC is used in that case in order to prevent
checking the host database, which is irrelevant when generating
install binaries.

Sponsored by:		Citrix Systems R&D
PR:			230459
Reviewed by:		gjb
Differential revision:	https://reviews.freebsd.org/D16638
2018-08-17 07:27:15 +00:00
Kyle Evans
35d2028fb8 libbe(3)/bectl(8): More SYSROOT/GCC build fixes
- Missing include path
- Fully specify libzfs's dependencies (except for deps pulled in by other
  deps) in Makefile.inc1
- Drop WARNS back down to 2 for libbe(3). I do this with much hesitation,
  but the libzfs headers are apparently a hot warning-filled mess as far as
  GCC 4.2 is concerned.
2018-08-11 22:45:39 +00:00