Commit Graph

1689 Commits

Author SHA1 Message Date
Bryan Drewery
4d64cb780b Let CROSS_BINUTILS_PREFIX work without a trailing slash.
Reported by:	jhb
MFC after:	2 weeks
Sponsored by:	Dell EMC
2018-06-19 23:40:39 +00:00
Bryan Drewery
8a854b3683 SYSTEM_COMPILER: Should use X_ vars for comparing wanted version.
It is XCC used during the build of target binaries that replaces the
bootstrap compiler.

Also slightly tweak style.

MFC after:	2 weeks
Sponsored by:	Dell EMC
2018-06-19 23:40:36 +00:00
Bryan Drewery
aafc33396e Fix X_COMPILER_* and X_LINKER_* not being passed to installworld environment.
This could lead to 'sh: head: not found' warnings which were a symptom
of running 'ld --version' during installworld.  This was only happening
with XCC or XLD set.  It is intended that cc and ld do not run during
installworld.  The metadata for these are already stored in
compiler-metadata.mk added in r316794.

This also removes redundant CROSSENV additions that were for
WITH_SYSTEM_COMPILER, WITHOUT_CROSS_COMPILER, and WITHOUT_TOOLCHAIN
which all don't have a cc or ld in their PATH during install.

Reported by:	Mark Millard
MFC after:	2 weeks
Sponsored by:	Dell EMC
2018-06-19 23:39:55 +00:00
Bryan Drewery
cf43faaa9f Don't bootstrap lld if an external linker is used.
Sponsored by:	Dell EMC
2018-06-19 01:00:27 +00:00
Ian Lepore
2a821fd9d2 Build LOCAL_LIB_DIRS along with system lib dirs, rather than building them
in parallel with LOCAL_DIRS and all the other system post-libs stuff.
2018-06-17 14:13:45 +00:00
Konstantin Belousov
f39bffc62c Rework ofed build.
Aligns the build with the FreeBSD traditional approach to not build in
contrib/, and to track inter-dependencies between libraries.

With help from:	bdrewery
Reviewed by:	bdrewery, hselasky
Sponsored by:	Mellanox Technologies
MFC after:	2 weeks
Differential revision:	https://reviews.freebsd.org/D15648
2018-06-16 15:05:05 +00:00
Bryan Drewery
4e3db1e0af Assert that a build is done before an install.
This should also catch cases where the wrong MAKEOBJDIRPREFIX is used for
install.

MFC after:	2 weeks
Sponsored by:	Dell EMC
2018-06-16 00:35:19 +00:00
Bryan Drewery
6f1f2b1931 Only build jevents if MK_PMC is yes and only for amd64 in libcompat.
Sponsored by:	Dell EMC
2018-06-15 16:48:09 +00:00
Mark Johnston
8590505f48 Bump __FreeBSD_version after r334881 and force libdwarf to be rebuilt.
Reported by:	O. Hartmann <ohartmann@walstatt.org>
Reviewed by:	bdrewery
2018-06-09 20:01:03 +00:00
Ed Maste
521ef556a0 Handle -DNO_CLEAN builds across brk/sbrk rewrite in r334626
Sponsored by:	The FreeBSD Foundation
2018-06-05 18:27:09 +00:00
Brad Davis
8dc84f09e0 Move /sys symlink creating out of etc/Makefile.
This is prep for etc/Makefile going away.

Approved by:	bapt (mentor)
2018-06-04 15:17:24 +00:00
Matt Macy
7d1c2b74a0 libpmc/pmu: enable for i386 as well 2018-05-31 22:26:55 +00:00
Matt Macy
41e258c4b9 libpmc/jevents: fix cross-compile _to_ amd64
Reported by:	emaste
2018-05-31 01:01:35 +00:00
Matt Macy
959826ca1b pmc(3)/hwpmc(4): update supported Intel processors to rely fully on the
vendor provided pmu-events tables and sundry cleanups.

The vendor pmu-events tables provide counter descriptions, default
sample rates, event, umask, and flag values for all the counter
configuration permutations. Using this gives us:

- much simpler kernel code for the MD component
- helpful long and short event descriptions
- simpler user code
- sample rates that won't overload the system

Update man page with newer sample types and remove unused sample type.
2018-05-26 19:29:19 +00:00
Matt Macy
5506ceb87f Revert r334242 "pmc(3)/hwpmc(4): update supported Intel processors to rely fully on the"
because of squash commit messages
2018-05-26 19:26:19 +00:00
Matt Macy
4928135658 pmc(3)/hwpmc(4): update supported Intel processors to rely fully on the
vendor provided pmu-events tables and sundry cleanups.

The vendor pmu-events tables provide counter descriptions, default
sample rates, event, umask, and flag values for all the counter
configuration permutations. Using this gives us:

- much simpler kernel code for the MD component
- helpful long and short event descriptions
- simpler user code
- sample rates that won't overload the system

Update man page with newer sample types and remove unused sample type.

Squashed commit of the following:

commit 4459d43eff815bec08ccc5533dbe5de846f03128
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Sat May 26 00:06:31 2018 -0700

    libpmc: fix pmu function signatures for non amd64

commit a2cb8bbc586c65d41f9b291430a2261ec67b59fe
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 22:38:11 2018 -0700

    pmcstat: fix indentation of usage

commit f686954b15ff56a833ac80404898977cb80a265b
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 22:19:49 2018 -0700

    pmclog(3): add callchain and pmcallocatedyn, remove pcsample

commit 73e13a0d2e9498c81c150d14d022050cee7511bb
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 22:19:00 2018 -0700

    pmclog.h: GC pcsample field

commit 3e93ffd65da641fa657539dad3c48e281f8b5798
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 22:05:57 2018 -0700

    hwpmc: make Intel core CPUs use external event tables

commit 634f5fae1e1644ac324003136c66cd9c619d1c93
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 22:00:06 2018 -0700

    pmclog: update log record types, bump PMC_MAJOR
    - explicitly make log record types a multiple of 8 bytes
    - hook in pmu event types for pmc_allocate records
    - remove references to no longer PCSAMPLE record

commit 83d84fcd2d65bdf6ddcb2e155a22f0cfa2a9c225
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 21:52:10 2018 -0700

    libpmc: add support for having vendor table driven pmc_allocate

commit 9e6ad63c40c2fce8404847ace5078ca6cb33a736
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 19:11:33 2018 -0700

    hwpmc_core: add accessors for EVSEL & UMASK, make IAP_UMASK useful to user

commit 859dceb93daa6419a48c794db99b6758e5b041c9
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 19:09:45 2018 -0700

    pmcstat: update usage and man page as well as make -L consistent with pmccontrol

commit 79c7d8597e28c2eb13f5f9113e65ec2792ca57b1
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 18:07:03 2018 -0700

    pmu_util: add support for all current intel event keywords

commit d8089c7f6a6c8527f38324252b1ffb47004694c6
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 17:45:00 2018 -0700

    add description for new arguments

commit 058336740bab53c62ec88a3a026ea848cf3878c6
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 17:38:15 2018 -0700

    libpmc: move pmu_events table and pmu_utils out of libpmcstat so that they can be used by pmc_allocate

commit 049b66b382e2f833c3f47bc8df9e750cb265709f
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 16:12:41 2018 -0700

    pmcstat: hook pmu_events counter description utility routines in

commit f5e01e7b37a691dc045e1aa16b3ebdd162515de8
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 16:11:59 2018 -0700

    pmu_events: add utility routines for listing counters and their descriptions

commit cba4d4f8907f772279f86f18f915e0d74d33ac56
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 16:09:50 2018 -0700

    pmu-events: expand out skylake regex to simplify string matches
2018-05-26 18:12:50 +00:00
Brooks Davis
c2decc35c8 Support -DNO_CLEAN builds across r334223+r334224. 2018-05-25 20:42:28 +00:00
Matt Macy
773015538c libpmcstat: Don't build pmu tables on !amd64 until the corresponding
util routines have been written and tested. Currently building them
breaks the build on power64

Reported by:	emaste
2018-05-24 21:22:03 +00:00
Brooks Davis
810c4dcddf exect() was removed in 2018, not 2017. 2018-05-24 17:05:41 +00:00
Matt Macy
e98bbcf9ca libpmcstat: compile in events based on json description 2018-05-24 04:30:06 +00:00
Ed Maste
754a9d6c59 Bump min supported release for building HEAD to 10.3
r307825 and r333017 disallow building FreeBSD-HEAD from 9.x or 10.x
versions that have some clang issues.  The minimum supported Subversion
revisions and osreldates containing the fix are:

Version  Min Rev  osreldate
    9.x  r286035     903509
   10.x  r286033    1002501

9.3 is the final 9.x release and does not contain the r286035 fix.
10.3 is the first 10.x release with the fix.  Thus, in practice 10.3 is
the oldest release that can build HEAD.

Although it may still be possible to build HEAD from an up-to-date
stable/9 it's not worth maintaining the special case when the branch
itself is unsupported and there are no usable releases from that branch.

Old UPDATING entries can be removed and the Clang warning in UPDATING
may be updated, in a future commit.

Approved by:	imp
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D15209
2018-04-27 19:50:30 +00:00
Ed Maste
27d86a1ca4 Correct r307825 BOOTSTRAPPING test for unsupported upgrade
__FreeBSD_version has five digits after the major number, but the min
versions in the test had an extra 0 (so the test was always false).
2018-04-26 13:18:24 +00:00
Brooks Davis
d329275591 Allow -DNO_CLEAN builds across r332443. 2018-04-12 18:24:00 +00:00
Brooks Davis
d9b41671bf Support -DNO_CLEAN builds across r332119. 2018-04-06 17:23:47 +00:00
Brooks Davis
e4e7a46ba9 Allow -DNO_CLEAN buildworld after r332080. 2018-04-05 18:19:48 +00:00
Ed Maste
523bfea2a1 remove special handling for stale ptrace dependencies
r318957 added special handling for stale ptrace dependency files to
support a -DNO_CLEAN build in a tree last built before r305012.  That
revision is now over a year and a half old, so retire the special case.

Sponsored by:	The FreeBSD Foundation
2018-04-05 03:26:42 +00:00
Brooks Davis
62d6bc395a Allow -DNO_CLEAN builds over r332048. 2018-04-04 22:46:30 +00:00
Ed Maste
af94761a07 Correct comment date in r331255
Reported by:	jhb
2018-03-20 23:08:04 +00:00
Ed Maste
aa9ceac49d Remove stale generated assym.s files after r331254
So that -DNO_CLEAN builds work.

This should be reverted after a sufficient transition period (perhaps 3
or 6 months).

Sponsored by:	The FreeBSD Foundation
2018-03-20 18:02:21 +00:00
Kyle Evans
fb6d3be5db pkgbase: Fix post-install script for kernel packages
kernel.ucl uses a hardcoded boot/kernel for kldxref, which is the incorrect
directory when we're installing extra kernels that aren't the "default"
kernel (placed at boot/kernel).

Fix this by instead using a new %KERNELDIR% that we now replace in
Makefile.inc1 with "kernel" for the default kernel and "kernel.${_kernel}"
for these extra kernels so that, e.g. /boot/kernel.SHIVA, will get properly
kldxref'd upon update and avoid outdated linker.hints.

Reviewed by:	gjb
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D14689
2018-03-14 14:45:57 +00:00
Brooks Davis
2f5134834d Improve wording of error message when CROSS_TOOLCHAIN is not found.
Reported by:	emaste, jhb
2018-02-26 19:02:11 +00:00
Brooks Davis
0df651d7ee Allow CROSS_TOOLCHAIN to be a path to a file.
This allows working with custom cross toolchains without the need to
create files in /usr/local/share/toolchains.

Obtained from:	CheriBSD
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D14178
2018-02-25 20:21:30 +00:00
Bryan Drewery
55927f2836 Properly lookup values if they were empty.
Spotted by:	rpokala
X-MFC-With:	r329676
MFC after:	2 weeks
Sponsored by:	Dell EMC
2018-02-20 22:03:08 +00:00
Bryan Drewery
0f8995a910 Only lookup SVN once and rename to SVN_CMD.
MFC after:	2 weeks
Sponsored by:	Dell EMC
2018-02-20 22:03:04 +00:00
Bryan Drewery
c0d9f07d47 Avoid looking up VCS_REVISION for every sub-make call to Makefile.inc1.
MFC after:	2 weeks
Sponsored by:	Dell EMC
2018-02-20 21:53:39 +00:00
Bryan Drewery
49a1108ae7 Move SVNVERSION_CMD into the one place that uses it.
This code, which is basically `svnversion || svnliteversion`, generates
2 fstatat(2) for every directory in PATH for every Makefile parsed that
includes bsd.own.mk.  This can add up for things like generating a Ports
index (Poudriere) or building a dependency graph for base.

Sponsored by:	Dell EMC
MFC after:	2 weeks
2018-02-20 21:48:16 +00:00
Warner Losh
c1c35c1065 sort needs to be in ITOOLS since find -s was changed to find | sort. 2018-02-09 18:47:00 +00:00
Warner Losh
207efdb345 Add a note about why we have the conditional before including
bsd.compiler.mk. It's so fmake from older 9.x systems still
works (still a supported build config, and having the note here
will let us know when we can cull it more easily).

Also pull in a related change from include to sinclude from
arichardson@'s cross building work, as well as it's companion in
Makefile.inc1 with a note about why we do the odd thing there.

Submitted by: archardson
Differential Revision: https://reviews.freebsd.org/D14241
2018-02-07 16:28:26 +00:00
Ed Maste
4816408016 add retpoline compiler and linker feature flags
These features indicate that the compiler and linker support the
retpoline speculative execution vulnerability (CVE-2017-5715)
mitigation.

Reviewed by:	dim, imp
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D14228
2018-02-07 14:50:06 +00:00
Alex Richardson
fb1df20368 Don't hardcode /usr/bin as the path for mktemp in build tools
It won't work e.g. when crossbuilding from Ubuntu Linux as mktemp is in
/bin there.

Reviewed By:	bdrewery
Approved By:	jhb (mentor)
Differential Revision: https://reviews.freebsd.org/D13937
2018-02-06 15:41:35 +00:00
Wolfram Schneider
a564880c29 `make installkernel' should display a completed message if done
PR:		225159
Reviewed by:	bdrewery
Approved by: 	cem (mentor)
Differential Revision:	https://reviews.freebsd.org/D13940
2018-01-25 21:36:26 +00:00
Bryan Drewery
553b3bb89d test-system-compiler: Display X_ variants for compiler/linker.
Sponsored by:	Dell EMC
2018-01-24 18:09:44 +00:00
Bryan Drewery
1c0cac23cf X_COMPILER_* may not be defined.
Sponsored by:	Dell EMC
2018-01-24 18:08:37 +00:00
Alex Richardson
ebe45c64a0 Don't build share/syscons in build-tools stage if MK_SYSCONS == "no"
Reviewed By:	emaste, jhb
Approved By:	jhb (mentor)
Differential Revision: https://reviews.freebsd.org/D12926
2018-01-16 21:43:36 +00:00
Bryan Drewery
fe8be58826 Ensure MK_LDD for bootstrap-tools matches MK_LLD_BOOTSTRAP.
This fixes building libllvmminimal with MK_LLD support when WITHOUT_LLD
or WITHOUT_TOOLCHAIN are set, which are install flags not build tools
flags.

Reported by:	sbruno
Tested by:	sbruno
Sponsored by:	Dell EMC
2018-01-12 20:40:27 +00:00
Bryan Drewery
d52c29e7f2 Cache LINKER_FEATURES to fix the wrong ones being used.
Submitted by:	Dan McGregor <dan.mcgregor@usask.ca>
2018-01-12 01:09:30 +00:00
Ian Lepore
e93e5bb0fa Allow use of .WAIT in the LOCAL_DIRS and LOCAL_LIB_DIRS lists.
A comment in Makefile.inc1 has long stated that LOCAL_DIRS are built last,
after the base system.  Incremental improvements in parallel building over
the years have led to LOCAL_DIRS being built in parallel with base system
directories.  This change allows the .WAIT directive to appear in LOCAL_DIRS
and LOCAL_LIB_DIRS lists to give the user some control over parallel
building of local additions.

Differential Revision:	https://reviews.freebsd.org/D13622
2017-12-31 18:53:13 +00:00
Ed Maste
3670d4c79a Makefile.inc1: map mips MACHINE_/TARGET_ARCH to triples
This helps Clang MIPS builds.

Reviewed by:	bdrewery, jhb (earlier version)
Differential Revision:	https://reviews.freebsd.org/D12171
2017-12-05 21:42:09 +00:00
Bryan Drewery
9b43e3da5e native-xtools: Ensure GCC files are cleaned up.
Because we force enable MK_GCC when building we need to also force
enable it for the cleaning phase.  Otherwise the NXB_TARGET files
are found in the next build's kernel-toolchain phase and cause
an error.

Reported by:	sbruno
X-MFC-With:	r325001
MFC after:	1 month
Sponsored by:	Dell EMC
2017-12-05 21:36:08 +00:00
Bryan Drewery
4a51c7085e native-xtools: Fix build without META_MODE for GCC archs.
The initial kernel-toolchain is built with TARGET=MACHINE but
we want GCC to have files generated for TARGET=NXB_TARGET
instead later on.  Just clean the files between building of
the toolchain and nxb binaries which will let it rebuild
when needed.

Reported by:	sbruno
X-MFC-With:	r325001
MFC after:	1 month
Sponsored by:	Dell EMC
2017-12-05 02:23:36 +00:00
Hans Petter Selasky
388290b8be Fix for multi-threaded lib32 compat build on amd64.
Sponsored by:	Mellanox Technologies
2017-11-24 14:29:32 +00:00
Hans Petter Selasky
e833305789 Simplify the build dependencies when building the OFED libraries.
Suggested by:	bdrewery@
Sponsored by:	Mellanox Technologies
2017-11-24 09:01:08 +00:00
Hans Petter Selasky
937d37fc6c Merge ^/head r325842 through r325998. 2017-11-19 12:36:03 +00:00
Bryan Drewery
3faecc70e8 Fix 'make -n' for {reinstall,distribute}kernel.
PR:		201779
Sponsored by:	Dell
2017-11-18 20:01:02 +00:00
Hans Petter Selasky
7a310b0696 Update list of hardcoded OFED build order dependencies for
the buildworld target.

Sponsored by:	Mellanox Technologies
2017-11-16 15:13:28 +00:00
Ed Maste
92b800cc1a Sort pkgbase mtree metadata, for reproducible builds
Packaged base packages are created by running the stageworld and
stagekernel targets with -DNO_ROOT, and converting the resulting mtree
file into a set of pkg plists.  If stage* is run with multiple processes
the order of entries in the mtree file may be nondeterministic, and the
resulting package tbz also had nondeterministic file ordering.

The mtree file generated by -DNO_ROOT builds consists of one line per
file, with the filename starting in the first column, so is easily
sorted.  There's one exception: the first line of the mtree file is a
comment, but the # character sorts before the filenames anyhow and needs
no special treatment.

PR:		223673
Reviewed by:	bapt, gjb
Sponsored by:	The Linux Foundation, Core Infrastructure Initiative
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D13103
2017-11-15 18:03:31 +00:00
Warner Losh
ca987d4641 Move sys/boot to stand. Fix all references to new location
Sponsored by:	Netflix
2017-11-14 23:02:19 +00:00
Alex Richardson
7202739f7b Only build the libmagic build-tools if MK_FILE != no
Before this patch libmagic was always built in the build-tools phase
even if -DWITHOUT_FILE was specified.

Reviewed by:	emaste, jhb
Approved by:	jhb (mentor)
Differential Revision: https://reviews.freebsd.org/D12925
2017-11-13 20:07:08 +00:00
Bryan Drewery
f649ab27a3 cleanworld only needs a tree walk if the expected .OBJDIR is .CURDIR.
Sponsored by:	Dell EMC Isilon
2017-11-11 01:10:08 +00:00
Bryan Drewery
9cde684d49 native-xtools: Fix for GCC archs.
- This also adds in a _cleanobj step as needed.
- This redirects TARGET/TARGET_ARCH to NXB_TARGET/NXB_TARGET_ARCH in
  Makefile.inc1 as the main build needs to be for MACHINE rather
  than TARGET.

  First build the toolchain and then use that as an external toolchain
  to build the needed directories and NXB_TARGET-toolchain, all as
  MACHINE files though via TARGET_TRIPLE=MACHINE_TRIPLE.

  The NXBDIRS is evaluated in the 'everything' submake as it needs to be
  based on TARGET's src.opts.mk values, such as MK_GCC=yes when building
  on a MK_CLANG=yes MACHINE.  This can likely be changed to a specific
  _native-xtools-everything target later and the funky late evaluation
  of SUBDIR_OVERRIDE removed.

X-MFC-With:	r325001
Pointyhat to:	bdrewery
MFC after:	2 months
Sponsored by:	Dell EMC Isilon
2017-11-10 19:53:07 +00:00
Bryan Drewery
ae9f7248f9 rescue ipf: Remove hacks and link in libipf directly.
Sponsored by:	Dell EMC Isilon
2017-11-10 07:52:58 +00:00
Bryan Drewery
fa65e3a53d Handle some .OBJDIR == .CURDIR cases.
- If OBJROOT is SRCTOP then don't add on TARGET.TARGET_ARCH.  This
  only happens at the top-level, and for sub-directories when the
  user is clever with MAKEOBJDIRPREFIX=/.
- Don't bother checking 'test -w' on .CURDIR.
- Properly set OBJTOP/OBJROOT to SRCTOP in various needed cases.
- Check if the OBJDIR is writable even for *clean* targets since it
  determines which .OBJDIR the user gets;  If they cannot write to an
  existing eligible .OBJDIR then it needs to clean in .CURDIR instead.
- Add guard to cleanworld/cleanuniverse from removing SRCTOP.
- Ensure OBJTOP is proper for .OBJDIR=.CURDIR which fixes finding
  libraries since src.libnames.mk is based on OBJTOP.
- Avoid some chdir(2) for modifying .OBJDIR

Sponsored by:	Dell EMC Isilon
2017-11-10 02:09:33 +00:00
Warner Losh
3fa561a45b Note interactive shell errors for make buildenv and add a warning for
people tempted to add back the || true to get rid of them.

Sponsored by: Netflix
2017-11-07 15:01:38 +00:00
Bryan Drewery
996e475962 Fix nested MAKEOBJDIRPREFIX breaking various release/buildworld/toolchain targets.
This problem was caused by r325329 and r325350.

For the release(7) targets, some will run mm-mtree.sh which itself runs make
with a MAKEOBJDIRPREFIX.  The execution of that script leaks OBJROOT,
MAKEOBJDIR, and MAKELEVEL=1 in the environment.  This causes the mm-mtree makes
to not do some basic setup of OBJROOT and only use this special
MAKEOBJDIRPREFIX case which fails to empty out MAKEOBJDIRPREFIX for further
nested makes, such as a tree walk.  If that tree walk sets OBJROOT/OBJTOP such
as r325329 is doing, then the wrong OBJDIRs end up being used due to the
unemptied MAKEOBJDIRPREFIX being preferred over the proper MAKEOBJDIR.

Pointyhat to:	bdrewery
Sponsored by:	Dell EMC Isilon
2017-11-05 00:11:51 +00:00
Bryan Drewery
2abeba9d7a Follow-up r318105: Don't use NO_OBJ at top-level, use NO_OBJWALK.
NO_OBJ has a very specific meaning in sub-directories in that no object
directory will be made.  If a user wanted to skip the 'make obj' phase then
passing -DNO_OBJ would break all sub-directories from building properly.  Using
NO_OBJ internally also causes issue with NO_OBJ handling being added in
share/mk/bsd.init.mk soon.

Sponsored by:	Dell EMC Isilon
2017-11-03 23:22:03 +00:00
Bryan Drewery
2c36cefeaf Follow-up r325329: Store all WORLDTMP objects back in the same directory.
This still keeps the reduced MAKEOBJDIRPREFIX (SRCTOP) redundancy
removed in the OBJDIR, but now keeps all early phase objects
in the same directory rather than split per phase.

The problem of splitting per phase is that later phases want to link in
libraries from earlier phases and base their location on ${OBJTOP}.

Pointyhat to:	bdrewery
Reported by:	mjoras, Mark Millard
Sponsored by:	Dell EMC Isilon
2017-11-03 00:29:42 +00:00
Bryan Drewery
fe480405ec Follow-up r325342: Always ensure WORLDTMP exists.
Reported by:	Herbert J. Skuhra
Sponsored by:	Dell EMC Isilon
2017-11-02 21:58:18 +00:00
Bryan Drewery
89bd07e927 DIRDEPS_BUILD: Cleanup leftover WORLDTMP files.
This will force any existing objects to rebuild if their .meta
files reference files from WORLDTMP.  This is a problem after
the UNIFIED_OBJDIR effort caused buildworld and DIRDEPS_BUILD
to share an OBJDIR.  Without cleaning these files the
Makefile.depend files end up with odd tmp/legacy/... entries
since some bootstrap-tools linger from there and otherwise
don't rebuild.  Removing the files causes anything having
used WORLDTMP to rebuild.

Sponsored by:	Dell EMC Isilon
2017-11-02 20:45:04 +00:00
Bryan Drewery
4dc89c4d51 Reduce MAKEOBJDIRPREFIX path spam by specifying a direct objdir to use.
Sponsored by:	Dell EMC Isilon
2017-11-02 18:08:36 +00:00
Bryan Drewery
3d165b2f1e native-xtools/xdev: Store these in a location including both MACHINE/TARGET.
Given MACHINE/MACHINE_ARCH=amd64, TARGET=arm64 and TARGET_ARCH=aarch64,
this change will build them in a location such as:
    /usr/obj/usr/src/amd64.amd64/nxb/arm64.aarch64/
and
    /usr/obj/usr/src/amd64.amd64/xdev/aarch64-freebsd/

Sponsored by:	Dell EMC Isilon
2017-11-01 21:22:13 +00:00
Bryan Drewery
c10062b9bf Add a 'make cleanuniverse'.
This will remove all build files for the source directory
when MK_UNIFIED_OBJDIR is enabled.

Sponsored by:	Dell EMC Isilon
2017-11-01 21:22:10 +00:00
Bryan Drewery
dfa099890c Add option UNIFIED_OBJDIR, on by default, which moves the default build OBJDIR.
This changes the build OBJDIR from the older style of /usr/obj/<srcdir> for
native builds, and /usr/obj/<target>.<target_arch>/<srcdir> for cross builds to
a new simpler format of /usr/obj/<srcdir>/<target>.<target_arch>.  This
new format is used regardless of cross or native build.  It allows
easier management of multiple source tree object directories.

The UNIFIED_OBJDIR option will be removed and its feature made permanent
for the 12.0 release.

Relnotes:	yes (don't note UNIFIED_OBJDIR option since it will be removed)
Prior work:	D3711 D874
Reviewed by:	gjb, sjg
Discussed at:	https://lists.freebsd.org/pipermail/freebsd-arch/2016-May/017805.html
Discussed with:	emaste
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D12840
2017-11-01 21:22:05 +00:00
Bryan Drewery
f069674019 xdev: Follow-up r325087: Need to build lib/clang before lld.
LLD needs headers generated by the full libllvm.

X-MFC-With:	r325087
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-10-31 23:48:57 +00:00
Bryan Drewery
651c6845c7 kernel-toolchain: Fix improper build order after r325244.
Due to removing some targets that the previous .ORDER: ${WMAKE_TGTS}
set, it was no longer being respected; _build_tools was coming
immediately.

Pointyhat to:	bdrewery
X-MFC-with:	r325244
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-10-31 21:51:33 +00:00
Bryan Drewery
5a4121a800 kernel-toolchain: Skip world _obj and _cleanobj phases.
There's no good reason to treewalk the entire tree removing old OBJDIRS
and creating new ones when 'includes', 'libraries', and 'everything' are
all skipped.  The only shared directory in the existing toolchain target
and world is build-tools.  So handle cleaning in it directly if needed
only for the directories it wants to build.

The extra _obj/_cleanobj walks came in the initial kernel-toolchain
addition in r128189.

MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-10-31 19:02:14 +00:00
Bryan Drewery
7cd84b5641 build-tools: De-special-case the gcc tools build.
It merely wanted to use 'all' rather than 'build-tools' so just
add a build-tools target to the Makefile.

MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-10-31 19:02:05 +00:00
Bryan Drewery
555c6e60bc Follow-up r297998: Remove redundant TOOLS_PREFIX in XMAKE.
Sponsored by:	Dell EMC Isilon
2017-10-31 17:16:46 +00:00
Bryan Drewery
337698b78d native-xtools: Allow SYSTEM_COMPILER to be used after r325001.
Now that a proper sysroot is used and a separate target list, it should
be safe to skip building the initial cross-compiler.

X-MFC-With:	r325001
MFC after:	1 month
Sponsored by:	Dell EMC Isilon
2017-10-31 02:52:36 +00:00
Bryan Drewery
601594ed6a cleanworld: No need to cleandir if MK_AUTO_OBJ is enabled.
Sponsored by:	Dell EMC Isilon
2017-10-31 01:45:14 +00:00
Warner Losh
0255bc1370 Return proper status from buildenv.
make buildenv BUILDENV_SHELL=<some command> more useful. Remove '||
true' from the command line so that errors are properly
returned. There appears to be no reason for it, and it dates back to
the original commit by ru@.

Sponsored by: Netflix
2017-10-30 03:12:38 +00:00
Bryan Drewery
be1ef37b94 xdev: Consolidate duplicate cross-tools build/install list into XDEVDIRS.
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-10-29 01:21:52 +00:00
Bryan Drewery
86c4ac993d xdev: Remove excess '/' since XDTP is asserted to be an absolute path.
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-10-29 01:21:49 +00:00
Bryan Drewery
c16a698b5e xdev: Fix build for platforms needing LLD.
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-10-29 01:21:46 +00:00
Bryan Drewery
57820ca8df xdev: Add missing 'make obj' treewalk for 'make includes'.
It is possible that building headers requires an OBJDIR.
The other phases of xdev have their own 'make obj' calls
where needed, such as inside 'make libraries' itself.

MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-10-29 01:21:42 +00:00
Bryan Drewery
5bd47cc56f Change native-xtools to not install by default; add a native-xtools-install.
Without this the user has to mess with 'make -f Makefile.inc1 ...' to figure
out where the files are installed in the OBJDIR and then they need to copy them
to where they really wanted them. Using DESTDIR may be problematic after
r325001 as well.

The files will be installed to DESTDIR/NXTP where NXTP defaults to /nxb-bin.

MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-10-29 01:21:29 +00:00
Bryan Drewery
24830684f2 Fix xdev TARGET/TARGET_ARCH assertion and expand to native-xtools.
The top of Makefile.inc1 requires TARGET/TARGET_ARCH be defined.  Just
building 'make xdev' would already set them, so this error was never
triggered.  Moving it to Makefile fixes the problem.

MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-10-29 01:21:19 +00:00
Bryan Drewery
bb3f65ea64 Disable AUTO_OBJ as a make argument to ensure it really is disabled.
If the user sets this to yes as a make argument too we still want
to ensure these do not get enabled.

Sponsored by:	Dell EMC Isilon
2017-10-29 01:21:15 +00:00
Bryan Drewery
0a3c9523e0 Remove comment: debug files can be useful for build tools.
Sponsored by:	Dell EMC Isilon
2017-10-29 01:21:06 +00:00
Bryan Drewery
67d2f51ad6 Move some LIBCOMPAT tmpdir logic setup to its own build target.
Its own build target was already handling mtree extractions
just as _worldtmp did, so the other cleaning of the
tmpdir makes sense here as well.

Sponsored by:	Dell EMC Isilon
2017-10-29 01:21:02 +00:00
Bryan Drewery
a3d90ddfb2 MK_TESTS is disabled in early phases, no need for TESTBASE extraction here.
Sponsored by:	Dell EMC Isilon
2017-10-29 01:20:59 +00:00
Bryan Drewery
53f3deb166 FAST_DEPEND clean hack: Remove lib32/soft files properly.
Only remove them if the option is enabled and also handle libsoft
by using the proper LIBCOMPAT_OBJTREE.  LIBCOMPAT:D will expand
the text after it as a proper glob to the command line if LIBCOMPAT
is defined.

Sponsored by:	Dell EMC Isilon
2017-10-29 01:20:56 +00:00
Bryan Drewery
fb3d959c66 Move FAST_DEPEND cleanup hack into its own target and call from _cleanobj.
The _worldtmp target is for setting up WORLDTMP.  Nothing between _worldtmp
and _cleanobj will read these files.  Move to its own target since it is
so large.

Sponsored by:	Dell EMC Isilon
2017-10-29 01:20:53 +00:00
Bryan Drewery
354ffd5366 We don't need LIB32 directories extracted in early phase legacy/WORLDTMP dirs.
This reverts some changes from r289662

Sponsored by:	Dell EMC Isilon
2017-10-29 01:20:50 +00:00
Bryan Drewery
64ec67921d Disallow using a colon ':' in source checkout paths.
This leaks into the PATH handling for WORLDTMP and breaks
finding cross-tools.  The PATH handling could be fixed to
properly quote but is not worth the effort.

Also allow this sanity check to always run even with 'make -n'.

MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-10-29 01:20:47 +00:00
Bryan Drewery
5e3b906168 Move sendmail special check into a _sanity_check target.
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-10-29 01:20:43 +00:00
Bryan Drewery
a13a8505db Cleanup pieces missed in r315057 which made mandoc mandatory
Sponsored by:	Dell EMC Isilon
2017-10-25 21:46:39 +00:00
Bryan Drewery
ca95e9dd77 Fix native-xtools build to use a proper sysroot.
This takes longer but should reliably produce working binaries.
The old version linked against system libraries and headers
which would be a problem if building a native-xtools against
a newer source than the host system.  With a proper sysroot made
first this is not a problem.

This also allows:
- NXBDIRS to be built in parallel
- NXBDIRS to be installed to NXBDESTDIR in parallel
- SYSTEM_COMPILER logic to work again.  Note that because this change
  is adding a sysroot phase the compiler may be built up to twice now.
  The first is the "cross-compiler" even though it is for the native
  architecture, but it is still built to ensure the latest compiler
  is used to generate the binaries, unless SYSTEM_COMPILER allows
  /usr/bin/cc to be used.  Then the target compiler is built
  which is actually a cross-compiler since it runs on native host
  but generates TARGET.TARGET_ARCH binaries.

Note this also changes the path used for the OBJDIR.  It used to use
/usr/obj/target.target_arch/nxb/<srcdir> for objects and
/usr/obj/target.target_arch/nxb-bin for installed files, but now uses
/usr/obj/nxb/target.target_arch/<srcdir> for objects and
/usr/obj/nxb/target.target_arch/<srcdir>/nxb-bin for installed files.
  - NXBDESTDIR can be specified for where to install or queried with
    `make -f Makefile.inc1 TARGET=... TARGET_ARCH=... -V NXBDESTDIR`

This could potentially be improved to reuse an existing sysroot.  The
problem is with building the SUBDIR_OVERRIDE list it needs to use a
different OBJDIR since it is building all statically.  We don't want to
pollute the existing 'buildworld' OBJDIR and cause confusion on the next
build.  Using a separate OBJDIR for the 'everything' phase mostly works
except for some things like linking in INTERNALLIBS that exist in the
other OBJDIR.

MFC after:	1 month
Sponsored by:	Dell EMC Isilon
2017-10-25 21:46:36 +00:00
Bryan Drewery
b12371422a META_MODE: Follow-up r319593: Don't use host headers in a fresh new build.
Sponsored by:	Dell EMC Isilon
2017-10-25 21:46:30 +00:00
Baptiste Daroussin
71a6afa2dc Do not print error when running make check-old on system
without catpages directories

Reported by:	dim
2017-10-10 13:06:11 +00:00
Warner Losh
0b972ac92e Support armv7 builds for userland
Make armv7 as a new MACHINE_ARCH.

Copy all the places we do armv6 and add armv7 as basically an
alias. clang appears to generate code for armv7 by default. armv7 hard
float isn't supported by the the in-tree gcc, so it hasn't been
updated to have a new default.

Support armv7 as a new valid MACHINE_ARCH (and by extension
TARGET_ARCH).

Add armv7 to the universe build.

Differential Revision: https://reviews.freebsd.org/D12010
2017-10-05 23:01:33 +00:00
Cy Schubert
0c059da3e1 When building multiple kernels using KERNCONF, non-existent KERNCONF
files will produce an error and buildkernel will fail. Previously missing
KERNCONF files silently failed giving no indication as to why, only to
subsequently discover during installkernel that the desired kernel was
never built in the first place.

This is a rework of r302865.

This is the correct patch.

Reviewed by:	ngie (previous version, r302865)
MFC after:	2 months
Differential Revision:	D7167
2017-10-04 03:39:54 +00:00
Cy Schubert
84f54c4fd7 Revert r324246. This is not exactly what was intended. 2017-10-04 03:35:23 +00:00
Cy Schubert
74e0e295e4 When building multiple kernels using KERNCONF, non-existent KERNCONF
files will produce an error and buildkernel will fail. Previously missing
KERNCONF files silently failed giving no indication as to why, only to
subsequently discover during installkernel that the desired kernel was
never built in the first place.

This is a rework of r302865.

Reviewed by:	ngie (previous version, r302865)
MFC after:	2 months
Differential Revision:	D7167
2017-10-04 03:24:16 +00:00
Baptiste Daroussin
b505dd63c9 Do not print error when running make delete-old on system
without catpages directories
2017-09-26 05:33:15 +00:00
Ryan Libby
6d0ab07837 Makefile.inc1: bump elf tool chain version check after PR 215350 fix
This causes builds to use the in-tree libdwarf with the bug fix instead
of the host library.

Reviewed by:	bdrewery, emaste, markj
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D12239
2017-09-06 01:55:40 +00:00
Enji Cooper
e133ecbbb3 Respect MK_TCSH with build-tools and native-xtools
This helps reduce the WORLDTMP footprint slightly.

Based on a patch I submitted 5 years ago to GNATS.

PR:		174051
MFC after:	2 weeks
Relnotes:	yes (anyone who cross-builds with MK_TCSH=yes will run into
		     build failures if the host doesn't have tcsh(1))
Reminded by:	Fabian Keil <fk@fabiankeil.de>
2017-08-27 16:15:37 +00:00
Ed Maste
6323f724ea Set MK_LLD_IS_LD to MK_LLD_BOOTSTRAP for cross-tools
LLD_BOOTSTRAP is intended to control the linker used to link world and
kernel, while LLD_IS_LD is intended to control the linker installed in
that world.

Force LLD_IS_LD equal to LLD_BOOTSTRAP for the cross-tools build and
install phase, so that lld will be installed as the ld to run on the
host, when LLD_BOOTSTRAP is set.

PR:		221543
Reviewed by:	dim
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D12072
2017-08-23 12:47:10 +00:00
Glen Barber
53d1eddd44 Revert r322327, r322352, r322358:
Disconnect the dependency on the kernel package from the runtime
package.  There are a number of problems here:

 1) The runtime package installed into a chroot or a jail would
    include the kernel package, changing the behavior of how jails
    work now [1];

 2) As result of (1), it is possible a binary may incorrectly
    resolve kernel symbols [2]; in addition, it is possible there
    will be unexpected fallout with 32-bit jails on a 64-bit host
    kernel [2].

Noticed by:	brd [1]
Discussed with:	kib [2]
MFC after:	3 days
MFC note:	record-only to wipe from the merge tracker
Sponsored by:	The FreeBSD Foundation
2017-08-15 15:13:33 +00:00
Glen Barber
ebd2eeb3f8 Add SVNVERSION_CMD to bsd.own.mk, adding the capability to include
svnversion metadata to the runtime and kernel packages.

Instead of traversing src/sys, as is done by newvers.sh for uname(1),
a full tree walk is done to prevent userland and/or modifications
from not being reflected in a modified tree (M).

MFC after:	5 days
Sponsored by:	The FreeBSD Foundation
2017-08-11 19:21:40 +00:00
Glen Barber
1866d42eec Fix indentation from r322401.
MFC after:	3 days
MFC with:	r322401
Sponsored by:	The FreeBSD Foundation
2017-08-11 14:19:31 +00:00
Glen Barber
33f8570d0c Avoid creating kernel-dbg.txz distribution sets and kernel-debug
packages when MK_DEBUG_FILES is 'no'.

MFC after:	5 days
Sponsored by:	The FreeBSD Foundation
2017-08-11 14:18:11 +00:00
Glen Barber
1537337752 Indent nested conditionals for readability.
Sponsored by:	The FreeBSD Foundation
2017-08-10 14:18:58 +00:00
Glen Barber
16398a1141 Add a dependency on the kernel package for the runtime package.
The idea here is that, provided upstream pkg(8) maintainers accept
the proposed change, the kernel.ucl will contain a post-install
script causing pkg(8) to emit a message informing to reboot the
system after the kernel is upgraded using 'pkg upgrade', so the
new userland is installed on the running new kernel.  At present,
this functionality does not exist in pkg(8), but will help ensure
the upgrade path follows that from UPDATING.  To work around this
for now, evaluate ASSUME_ALWAYS_YES, and prompt the user if they
wish to proceed if not set to true.

Since there is a kernel dependency, and a non-GENERIC kernel may
be in use, update Makefile.inc1 to replace '%KERNCONF%' in the
runtime.ucl with the first-built kernel set either via command line
or in make.conf(5).

MFC after:	5 days
Sponsored by:	The FreeBSD Foundation
2017-08-09 19:16:54 +00:00
Dimitry Andric
8422e09e55 Follow-up to r321684 (Don't use libc++ when cross-building for gcc
arches), and handle two more cases where libc++ includes could be
incorrectly enabled, in case the host compiler is clang 5.0.0, and the
target (cross) compiler is gcc 4.2.1.

Noted by:	bdrewery
MFC after:	3 days
X-MFC-With:	321684
2017-08-07 16:23:53 +00:00
Bryan Drewery
91234af7d8 CCACHE_BUILD: Follow-up r321880: Fix some PATH issues with buildworld.
- bsd.compiler.mk: Must ensure that the CCACHE_WRAPPER_PATH comes first
  in PATH.
- Makefile.inc1: Must prepend the CCACHE_WRAPPER_PATH into BPATH as it
  overrides the PATH set in bsd.compiler.mk in sub-makes.  The PATH
  set in bsd.compiler.mk is not exported and doing so would cause it to
  then override the BPATH set from environment.  The only sane solution
  is to prepend into BPATH as needed.
CCACHE_PATH could possibly be used for some of this as well.

Sponsored by:	Dell EMC Isilon
2017-08-01 18:26:20 +00:00
Bryan Drewery
5aa73fa259 NO_CLEAN: No need to run delete-old if the directories don't exist.
X-MFC-With:	r321443
MFC after:	1 month
Sponsored by:	Dell EMC Isilon
2017-08-01 16:41:17 +00:00
Ian Lepore
dca576be69 In xdev-links, when installing symlinks to the cross-compiler pieces that
includes the OS version (armv6-freebsd12.0-cc, etc), use the OS version of
the compiler/world source code, not the version of the build host machine.
2017-08-01 14:54:25 +00:00
Dimitry Andric
1681d1de13 Don't use libc++ when cross-building for gcc arches
Since we imported clang 5.0.0, the version check in Makefile.inc1 which
checks whether to use libc++ fires even when the compiler for the target
architecture is gcc 4.2.1.  This is because only X_COMPILER_VERSION is
checked.  Also check X_COMPILER_TYPE, so it will only use libc++ when an
external gcc toolchain is used.

Reviewed by:	emaste, rpokala
MFC after:	3 days
Differential Revision: https://reviews.freebsd.org/D11776
2017-07-29 12:22:29 +00:00
Bryan Drewery
18388aa44a NO_CLEAN: Hide delete-old output.
It is full of distracting noise about UPDATING and may confuse
the user about what is actually being deleted.  It is also
possible to have directories removed on every run with
use of WITHOUT_ knobs that the mtree files do not
account for and for which the directories are incorrectly
in OLD_DIRS currently.

X-MFC-With:	r321443
MFC after:	1 month
Sponsored by:	Dell EMC Isilon
2017-07-25 20:51:06 +00:00
Bryan Drewery
b7c6a063fe Only build libzfs_core in 'make libraries' if needed.
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-07-25 20:43:37 +00:00
Bryan Drewery
63c534e716 Remove unneeded dependency for libzfs.
This dependency does nothing since cddl/lib/libzfs is never
added into the 'make libraries' dependency chain
directly.

MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-07-25 20:43:26 +00:00
Bryan Drewery
1ae1311d9c NO_CLEAN: Utilize delete-old to remove old orphaned libraries/headers in WORLDTMP.
This prevents situations with -DNO_CLEAN from finding stale headers or
libraries in places that no longer exist or have moved.  It avoids
the need to remove all of WORLDTMP by reusing what we already know
is obsolete.

MFC after:	1 month
Sponsored by:	Dell EMC Isilon
2017-07-24 23:32:46 +00:00
Bryan Drewery
dc75d44eca The .depend.obj cleanup hacks are only needed with -DNO_CLEAN.
Sponsored by:	Dell EMC Isilon
2017-07-24 23:32:43 +00:00
Bryan Drewery
cd2831005b Move llvm Options.inc hack from r321433 for NO_CLEAN to lib/clang/libllvm.
The files are only ever generated to .OBJDIR, not to WORLDTMP (as a
sysroot) and are only ever included from a compilation.  So using
a beforebuild target here removes the file before the compilation
tries to include it.

MFC after:	2 months
X-MFC-With:	r321369
2017-07-24 23:32:24 +00:00
Dimitry Andric
edcc769f3f Cleanup stale Options.inc files from the previous libllvm build for
clang 4.0.0.  Otherwise, these can get included before the two newly
generated ones (which are different) for clang 5.0.0.

Reported by:	Mark Millard
MFC after:	2 months
X-MFC-With:	r321369
2017-07-24 18:52:40 +00:00
Bryan Drewery
0f827d7a96 makeman: Don't show META_MODE ABI rebuild warnings.
Reported by:	dim
Sponsored by:	Dell EMC Isilon
2017-07-08 17:53:00 +00:00
Bryan Drewery
9bb3819b32 SYSTEM_COMPILER: Ensure there is not a stale compiler in WORLDTMP.
In a scenario of cross-building it is possible that an OBJDIR's WORLDTMP
contains an older compiler in WORLDTMP/usr/bin/cc that is not rebuilt
if SYSTEM_COMPILER logic is triggered.  This compiler was still
incorrectly used.  Address this by removing WORLDTMP/usr/bin/cc and all
of the hardlinked files associated with it.  Also do this for c++ for
GCC builds.

Sponsored by:	Dell EMC Isilon
MFC after:	1 week
2017-07-08 16:39:55 +00:00
Bryan Drewery
f45c780573 Fix create-kernel-packages with multiple BUILDKERNELS after r320284
Submitted by:	Kyle Evans <kevans91@ksu.edu>
Reviewed by:	ngie
2017-07-05 19:24:38 +00:00
Ed Maste
a28412b0db Clean up stale dependencies after r320278
Our current approach to dependency tracking cannot cope with switching
generated asm syscall stubs into C wrappers. Perpetuate the hack in
Makefile.inc1 to paper over the problem until we can take a holistic
approach to fixing dependency problems.

Differential Revision:	https://reviews.freebsd.org/D11344
2017-06-24 18:37:51 +00:00
Bryan Drewery
0369270f71 NO_ROOT: Remove excessive // when DESTDIR/DISTDIR are empty.
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-06-23 19:03:31 +00:00
Bryan Drewery
2d5cc53891 compiler-metadata: Properly handle cross-build OBJDIR.
MFC after:	3 days
Sponsored by:	Dell EMC Isilon
2017-06-23 18:27:03 +00:00
Bryan Drewery
275219cfe7 Expose only the create-packages-* targets since they set needed DEST/DIRDIR.
The other targets just fail confusingly otherwise.

MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-06-23 18:27:00 +00:00
Bryan Drewery
10c935294b packages: Parallelize individual kernel packaging.
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-06-23 18:26:57 +00:00
Bryan Drewery
f77b03aa2b packages: Allow actually building individual world packages in parallel.
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-06-23 18:26:54 +00:00
Bryan Drewery
48c22674a8 packages: Allow creating kernel/world packages in parallel.
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-06-23 18:26:51 +00:00
Bryan Drewery
00c44825d6 packages: Allow staging world/kernel in parallel.
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-06-23 18:26:47 +00:00
Bryan Drewery
dd7b4c153a packages: Allow stageworld/stagekernel to run with make jobs.
The -B was originally added in projects/release-pkg r289381 as a copy
of what 'make world' did at the time.  The -B was removed from
the 'installworld' call in 'world' in r303844 though.  The staging
of files is safe to run in parallel.

MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-06-23 18:26:33 +00:00
Bryan Drewery
d20bd9e42b Set compiler metadata for stageworld/distributeworld.
This fixes LD errors during 'make packages' but also for the unlikely case of
'buildworld' on 1 system and 'packages' on another [1].

PR:		212877 [1]
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-06-23 16:38:46 +00:00
Bryan Drewery
81b4423f05 Pass along LINKER_* vars during installworld and show in test-system-compiler.
Sponsored by:	Dell EMC Isilon
2017-06-22 21:03:30 +00:00
Bryan Drewery
817366f1c1 buildworld: Pass which world phase the build is in down to submakes.
This is useful for having directories behave differently depending
on the phase - such as enabling SUBDIR_PARALLEL or disabling
redundant building of library directories already done by
earlier 'make _libraries'.

Sponsored by:	Dell EMC Isilon
2017-06-20 22:08:02 +00:00
Bryan Drewery
35fb812adf buildworld: Define SYSROOT to WORLDTMP.
This is to allow downstream Makefiles to know for sure they are building
against a sysroot rather than only depending on ${DESTDIR} or other
assumptions.

This also exports it into buildenv.

MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-06-19 18:08:20 +00:00
Ed Maste
ff6c17c94e remove stale dependencies for utimens* wrappers removed in r319663
Use a similar approach to r318957 (which was for ptrace dependencies):
grep the .depend file for the old source file and delete the stale
dependency if found.

Reviewed by:	bdrewery
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D11091
2017-06-12 19:45:05 +00:00
Baptiste Daroussin
69db934679 Fix src.conf.5 generation
When makeman is generating src.conf(5) it tries to test all variation of options
including WITH_DIRDEPS_BUILD. it results in an error when filemon(4) is not
loaded.

Export variables that are needed to prevent this behaviour.

Helped by:	sjg
2017-06-08 00:41:25 +00:00
Bryan Drewery
c8fab231f1 META_MODE: Force a host tool rebuild for ino64 in r318736.
Sponsored by:	Dell EMC Isilon
2017-06-05 05:17:42 +00:00
Bryan Drewery
1a9cc831dd META_MODE: Add framework to force rebuilding for major ABI changes.
Normally META_MODE ignores host files for "meta mode" decisions on whether a
file should be rebuilt or not.  This is because a simple installworld can
update timestamps and cause the next build to rebuild all host tools, when the
previous ones may not have any changes in the source tree.  These tools are
normally still ABI compatible.  They are only rebuilt if NO_META_IGNORE_HOST is
set from the workaround/hack in r301467.

One of the major problems with this is when a host tool has objects spread
across many revisions that have mixed-ABI.  For example, if struct stat were to
change on the host, some objects for a tool may have different ideas of that
struct's definition.  If just 1 source file were modified and rebuilt and
linked into the tool, then that toll will have mixed-ABI objects and crash.
This exact thing happened with the ino64 commit in r301467 followed by a
trivial update to libbfd in r318750.  The resulting binary would crash in
buildworld.

Sponsored by:	Dell EMC Isilon
2017-06-05 05:17:39 +00:00
Bryan Drewery
86f2d54ff0 native-xtools: Connect makewhatis.
This speeds up some port staging, namely perl.
2017-06-03 23:13:46 +00:00
Bryan Drewery
9f68573426 native-xtools: Connect lld.
This will ensure that aarch64 gets a working native /usr/bin/ld rather
than requiring the aarch64-binutils hack in Poudriere, or emulating
the aarch64 lld.

PR:		217189
Reported by:	swills, jbeich
2017-06-02 00:57:59 +00:00
Ed Maste
bd1c3bad76 add a rescue/sh sanity check before installworld on the running system
FreeBSD does not guarantee kernel forward compatibility (that is,
running a newer userland on an older kernel).  The documented upgrade
procedure specifies that installkernel should be performed, followed by
a reboot and then installworld.  As a sanity check when installing onto
the running system (DESTDIR is / or unset), attempt to run "sh echo OK"
using rescue from the objdir.  If rescue fails (e.g., because the system
has not been rebooted and the "old" kernel lacks a system call required
by the to-be-installed world), abort the installation.

This should avoid ino64 foot-shooting when the proper upgrade procedure
is not followed.

Reviewed by:	allanjude, gjb, kib
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D10987
2017-05-30 15:51:48 +00:00
Ed Maste
55b87d4621 rm stale ptrace dependencies after r305012
This is similar to r318912, except that ptrace.[sS] was previously a
file in the source tree, not a generated assembly wrapper.

Check for the existence of ptrace.[sS] in the .depend file to determine
if we have to clean it up.  This is a bit hackish and will not be left
in place indefinitely, but provides a useful example case when
investigating a better solution in bmake.

Reviewed by:	bdrewery
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D10930
2017-05-26 16:03:28 +00:00
Ed Maste
90d4415d20 libc: rm stale generated files which are no longer syscalls
This is an attempt to help -DNO_CLEAN builds after r302092 (which
removed the pipe libc syscall wrapper) and r318736 (which removed
getdents, lstat, mknod, and stat).

Dependencies cannot cope with certain source tree changes,
particularly with respect to removing source files and replacing
generated files.  Handle these cases from _worldtmp in an ad-hoc
fashion.

Reviewed by:	bdrewery, cem
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D10876
2017-05-26 00:51:05 +00:00
Baptiste Daroussin
ac4ef3f9a9 Now that roff documentation has been disconnected from the build, it is no
longer necessary to have groff(1) as a bootstrap tool
2017-05-25 16:31:53 +00:00
Bryan Drewery
c596c2876f Add a -DNO_LIBS to skip building the libraries phase as well.
This is useful for cases where -DWORLDFAST is useful.

Sponsored by:	Dell EMC Isilon
2017-05-09 23:58:07 +00:00
Bryan Drewery
16892c7713 Support -DWORLDFAST to skip all build steps up to 'libraries' and 'everything'.
This allows for building the world against the already-created
host/sysroot environment.  It is not overly useful outside of cases of
large-impact changes such as a testing a new compiler.  It will
allow quickly getting back to an error in the target-phases of the
build where a new compiler is being used.

Sponsored by:	Dell EMC Isilon
2017-05-09 20:21:58 +00:00
Bryan Drewery
6d75a7a852 Support skipping 'make obj' tree-walk.
This is part of a larger effort for WITH_AUTO_OBJ and a WORLDFAST
option.

Sponsored by:	Dell EMC Isilon
2017-05-09 20:21:38 +00:00
John Baldwin
d5392a4191 Honor WITHOUT_LIB32 on mips64.
The closing paren for the list of architectures that should enable LIB32
by default was in the wrong place resulting in LIB32 always be enabled on
mips64 regardless of WITH_LIB32/WITHOUT_LIB32.

Submitted by:	Alex Richardson <Alexander.Richardson@cl.cam.ac.uk>
Obtained from:	CheriBSD
Sponsored by:	DARPA / AFRL
2017-05-08 18:51:13 +00:00
Baptiste Daroussin
a17665bcef Replace again GNU diff with BSD diff
After a firts failed attempt, BSD diff is now good enough to replace
GNU diff.

Relnotes:	yes
2017-04-20 19:24:51 +00:00
Glen Barber
5e50a31863 Use relative symlink for 'latest' to allow repo to work after being
moved.

Submitted by:	woodsb02
MFC after:	3 days
PR:		217197
Sponsored by:	The FreeBSD Foundation
2017-04-18 19:44:34 +00:00
Bryan Drewery
848d5e929b Cache compiler metadata and reuse it at installworld time.
Right after cross-tools, a compiler-metadata.mk file is created that
stores all of the bsd.compiler.mk metadata.  It is then read in
with a fail-safe during installworld time.

The file is explicitly removed when invoking cross-tools to ensure that
a stale file is not left around from odd manual 'make _cross-tools' ->
'make installworld' invocations.

This fixes several issues:
    - With WITH_SYSTEM_COMPILER (default yes on head and no on releng/11.0):
      If you build on a system where the bootstrap compiler does not
      build due to the host compiler matching the in-tree one, but then
      installworld on another system where that logic fails (a
      bootstrap compiler is needed), the installworld immediately fails
      with:
           sh: cc: not found
      Note that fixing this logic may then hit a case where a rebuild is
      attempted in installworld.  Normally cc would be ran with
      'CFLAGS+=ERROR-tried-to-rebuild-during-make-install' to cause an
      error such as:
          cc: error: no such file or directory: 'ERROR-tried-to-rebuild-during-make-install'
      However, now it will just fail with the 'cc: not found' error.
      Inspection of the compile line will show
      'ERROR-tried-to-rebuild-during-make-install';  It's not useful to
      set CC to anything other than 'cc' during install as it is more
      helpful to see the attempted compile rather than some other bogus
      error.
    - This now avoids running bsd.compiler.mk (cc executions) even more
      during installworld.  There are compiler-dependent SUBDIR in the
      tree which required having a compiler during install.

There is at least 1 case where CC is still executed in the install,
such as from a LOOKUP!= in secure/lib/libcrypto/Makefile.inc checking
for 'vzeroall' support.  This is not significant for installworld
as the lookup has a fallback (and hides its error) and only modifies CFLAGS,
thus it's not worth fixing.

PR:		212877
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-04-13 22:07:34 +00:00
Bryan Drewery
d5b29b60f8 Pass COMPILER_FEATURES down to submakes for installworld.
This is for WITH_SYSTEM_COMPILER, WITHOUT_CROSS_COMPILER, external
compiler, etc.

MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-04-13 19:27:46 +00:00
Ed Maste
ffe63c8bf0 Introduce LLD_BOOTSTRAP to control lld as bootstrap linker
Add WITH_LLD_BOOTSTRAP and WITHOUT_LLD_BOOTSTRAP knobs, similar to the
Clang bootstrap knobs.

Reviewed by:	dim
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D10249
2017-04-09 01:35:19 +00:00
Ed Maste
1bf5e133cb do not require binutils port when using lld as ld
r279908 added logic to Makefile.inc1 to automatically set
CROSS_BINUTILS_PREFIX for architectures not supported by the in-tree
binutils: arm64 when first introduced, and later riscv64 as well.

LLVM's LLD linker is now included in the base system, and is enabled by
default for arm64 and capable of linking world and kernel. Thus, avoid
automatically setting CROSS_BINUTILS_PREFIX and requiring the binutils
port if WITH_LLD_IS_LD is true.

Reviewed by:	kan
Relnotes:	Yes
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D10310
2017-04-07 23:37:22 +00:00
Bryan Drewery
dbbb03c244 META_MODE: Fix build-tools still sometimes rebuilding during target build.
In a cross-build, the build-tools are native host binaries.  We do not
want to rebuild them when building for the target.  Bmake previously
did not support checking .NOMETA on an existing target, so .NOMETA_CMP
was used here.  However, .NOMETA_CMP still triggers meta mode conditions
if the number of commands or the command changes.  In r312467 the paths
to build ncurses files were modified and thus triggered meta mode to
rebuild the build tools (make_keys, make_hash) in ncurses during the
target build.  Bmake 20160604 committed in r301462 changed .NOMETA to
also skip meta mode logic for an existing .meta file as well, thus it
is now the proper fix here.

I explored moving the build-tools output to WORLDTMP/tools with
relatively good success, but have concerns that doing so would be
problematic for downstream vendors who use LOCAL_TOOL_DIRS and
expect the tools to be in current OBJDIR for the target.  It also
adds more complexity into finding the tools during target build
and handling of where they are for rescue/rescue and
mkcsmapper_static/mkesdb_static which should really not be connected in
build-tools anyway.

MFC after:	2 weeks
Reported by:	many
Sponsored by:	Dell EMC Isilon
2017-04-06 18:21:59 +00:00
Bryan Drewery
3507006763 Rename _cc to _gcc to be more clear.
MFC after:	1 week
Sponsored by:	Dell EMC Isilon
2017-03-21 22:09:00 +00:00
Baptiste Daroussin
2c5ddcf8b6 Temporary readd GNU diff
etcupdate requires --change-group-format it is not easy to implement
in bsd diff so for now readd GNU diff
2017-03-19 17:19:59 +00:00
Baptiste Daroussin
5714b7010f Fix native-xtools after replacement of GNU diff 2017-03-12 03:33:20 +00:00
Baptiste Daroussin
97ab006d51 Remove the WITHOUT_MANDOCDB option
mandoc database is activated since FreeBSD 11.0, let's remove the previous
database format for FreeBSD 12.0
2017-03-11 06:51:21 +00:00
Bryan Drewery
0e4e8f1b33 Added comments for why nmtree/libmd are bootstrapped. 2017-03-06 20:32:17 +00:00
Bryan Drewery
51c6e78d91 Fix bootstrapping mtree after r313404 for older systems.
r313404 made libnetbsd require sha384.h from libmd.  Libmd added it in
r292782.  Update BOOTSTRAPPING to account for this.

Reported by:	bde
Reviewed by:	ngie
2017-03-05 21:16:50 +00:00
Dimitry Andric
6ae9acde63 Merge ^/head r313896 through r314128. 2017-02-23 07:45:58 +00:00
Warner Losh
6fc94bc49f Document why cat is a bootstrap tool. 2017-02-21 18:49:30 +00:00
Enji Cooper
74464d3ca4 Include ${LOCALBASE}/bin in $PATH when running "make checkworld"
Some of the tests in devel/atf // devel/kyua rely on the tools being in $PATH,
which means that the tests fail when run via "make checkworld" because $PATH
is restricted to exclude directory elements like "${LOCALBASE}/bin".

MFC after:	1 week
Sponsored by:	Dell EMC Isilon
2017-02-18 21:47:32 +00:00
Enji Cooper
66df1425ff Quote path to doxygen/kyua in test(1) -x check
This is a basic stopgap against ${LOCALBASE} containing spaces in it

MFC after:	1 week
Sponsored by:	Dell EMC Isilon
2017-02-18 21:41:50 +00:00
Bryan Drewery
68a558eca2 xdev: Fix after libc++ update in r300873.
The xdev build needed the same fixes as libcompat and external toolchain
support needed for handling of --sysroot, -L, -B, etc.

MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-02-18 00:08:13 +00:00
Bryan Drewery
e36b831f4b xdev: Build yacc which is needed for recent libpcap updates.
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-02-17 22:51:34 +00:00
Bryan Drewery
9e580940c1 META_MODE+xdev: Don't rebuild build-tools during normal build.
See r297997 for more information.

MFC after:	1 week
Sponsored by:	Dell EMC Isilon
2017-02-17 22:51:21 +00:00
Dimitry Andric
f9edb08480 Merge ^/head r313055 through r313300. 2017-02-05 20:03:05 +00:00
Bryan Drewery
e297227038 Remove LOCAL_LIB_DIRS warning added in r275839.
The case for which this was added, r274807, causes this warning to
always show.  LOCAL_DIRS=foo LOCAL_LIB_DIRS=foo/lib.  The only case in
which r274807 is a problem is if foo/Makefile does not contain
SUBDIR+=lib, which is a normal convention.  LOCAL_LIB_DIRS is a special
hack only to get a library into the _generic_libs list for the
'make libraries' bootstrapping phase.  The old behavior changed in
r274807 was only in head during the 10.0 cycle, so the warning was
only ever needed until release anyhow.

Reported by:	ngie
MFC after:	1 week
Sponsored by:	Dell EMC Isilon
2017-02-04 02:15:49 +00:00
Enji Cooper
c6dc13326b Fix typo in variable name (_REDUNDENT_LIB_DIRS -> _REDUNDANT_LIB_DIRS)
MFC after:	3 days
Sponsored by:	Dell EMC Isilon
2017-02-04 01:21:48 +00:00
Bryan Drewery
27f134b64a native-xtools: Add missing readelf.
The switch to elftoolchain's readelf in r280859 caused native-xtools
to no longer build readelf.  This fixes poudriere builds not using
a native readelf when expected.

Reported by:	strejda on freenode
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-02-03 16:27:23 +00:00
Dimitry Andric
65575c1424 Merge ^/head r312894 through r312967. 2017-01-29 22:00:47 +00:00
Yoshihiro Takahashi
2b375b4edd Remove pc98 support completely.
I thank all developers and contributors for pc98.

Relnotes:	yes
2017-01-28 02:22:15 +00:00
Ed Maste
d63b10f090 Update clang400 branch for LD_AS_LLD -> LD_IS_LLD 2017-01-27 20:49:43 +00:00
Dimitry Andric
14f850f3df Merge ^/head r312720 through r312893. 2017-01-27 20:01:46 +00:00
Ed Maste
f52e4bdfd7 Rename LLD_AS_LD to LLD_IS_LD, for consistency with CLANG_IS_CC
Reported by:	Dan McGregor <dan.mcgregor usask.ca>
2017-01-27 01:59:12 +00:00
Ed Maste
737872e978 Also apply WITH_LLD_AS_LD to build tools
Previously WITH_LLD_AS_LD installed LLD as /usr/bin/ld in the target
system, but still used the GNU BFD ld to link the binaries in that
target. LLD 4.0.0 can link the FreeBSD/amd64 world and kernel so use
LLD as the build-time linker as well when the knob is set.

Reviewed by:	dim
Relnotes:	Yes
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D9226
2017-01-25 21:05:48 +00:00
Baptiste Daroussin
7666f5006c Import mandoc cvs snapshot 20170121 (pre 1.14)
Note that mandoc does not use anymore sqlite3 but a home made database format
An important improvement has been made as well in makewhatis performance:
Tests on my laptop shows makewhatis on the entire system goes from 26s to 12s
2017-01-21 13:17:25 +00:00
John Baldwin
69b4d461be Enable /usr/lib32 for o32 binaries on mips64.
Build and install an o32 set of libraries on mips64 suitable for
running o32 binaries via COMPAT_FREEBSD32. Enable COMPAT_FREEBSD32 in
MALTA64.

Reviewed by:	jmallett, imp
Sponsored by:	DARPA / AFRL
Differential Revision:	https://reviews.freebsd.org/D9032
2017-01-06 23:30:54 +00:00
Enji Cooper
94ef145e6b Only bake krb5_config.h support in to ssh(3), etc if both MK_GSSAPI and
MK_KERBEROS_SUPPORT != no

This fixes the odd case where someone specified MK_GSSAPI=no and
MK_KERBEROS_SUPPORT=yes (which admittedly, probably doesn't make sense,
but the build system doesn't prevent this case today, and it didn't when
I filed the bug back in 2011 either).

MFC after:	2 weeks
PR:		159745
2017-01-02 20:29:50 +00:00
Dimitry Andric
715652a404 Make native-xtools build correctly after clang/llvm 3.9.0 import
During the clang/llvm 3.9.0 import, the build structure for it was
completely revamped.  This broke the native-xtools target.

It first attempts to build libllvmminimal, then the llvm-tblgen and
clang-tblgen executables, but these fail to link because they are linked
to the 'full' libllvm by default, as they normally are during the
'world' stage.

To make these link against libllvmminimal instead, define TOOLS_PREFIX,
similarly as during the bootstrap-tools phase.  The value itself is
empty, as we don't really want to use a prefix.

Reviewed by:	imp
PR:		215684
MFC after:	3 days
Differential Revision:	https://reviews.freebsd.org/D9026
2017-01-02 19:33:22 +00:00
Matthew Seaman
aeb76c0bb7 Revert r309339, thus re-instating r309314
The original problem with conflicting definitions of ${PKG_CMD} was
solved by r427523 in ports (see https://reviews.freebsd.org/D8677), so
this should be safe now.

Reviewed by:	gjb
Approved by:	gjb
Differential Revision:	https://reviews.freebsd.org/D8120
2016-12-16 23:05:47 +00:00
Baptiste Daroussin
629582ca46 Now that external gcc directly natively links to libc++ we can remove
the dirty hack made to fake libstdc++
2016-12-10 18:29:39 +00:00
Warner Losh
158c18ffb4 dd is currently a bootstrap tool. It really doesn't have any business
being a bootstrap tool. However, for reproducible build output,
FreeBSD added dd status=none because it was otherwise difficult to
suppress the status information, but retain any errors that might
happen. There's no real reason that dd has to be a build tool, other
than we use status=none unconditional. Remove dd from a bootstrap tool
entirely by only using status=none when available. This may also help
efforts to build the system on non-FreeBSD hosts as well.

Differential Revision: https://reviews.freebsd.org/D8605
2016-12-02 14:44:38 +00:00
Glen Barber
ebc7f2b8de Revert r309314, which breaks installing ports.
Requested by:	antoine
Differential Revision:	https://reviews.freebsd.org/D8120 (related)
Sponsored by:	The FreeBSD Foundation
2016-11-30 22:00:25 +00:00
Dimitry Andric
26e287836b Cleanup old debug dirs in delete-old-dirs target
Any .debug or .symbols files under /usr/lib/debug which correspond to
OLD_FILES entries in ObsoleteFiles.inc are also automatically cleaned up
by the delete-old target.  Make this also apply to any OLD_DIRS entries.

Reviewed by:	emaste
MFC after:	3 days
Differential Revision: https://reviews.freebsd.org/D8683
2016-11-30 19:06:47 +00:00
Matthew Seaman
91c915facf Allow a user-overridable setting 'PKG_CMD' to control the command used
to create a repo during 'make packages'

This would have been useful for a situation I found myself in where
pkg(8) had been upgraded to a version that wanted the FBSD_1.5 ABI
version but libc.so.7 had not been upgraded, and only provided
FBSD_1.4. I found I needed to update libc in order to run pkg, and I
also needed to use pkg to update libc... Which is why pkg-static
exists, but there's currently no way to tell the build system to use
pkg-static instead of pkg.

This creates a variable PKG_CMD, default value 'pkg', that can be
overridden from the command line.

Reviewed by:	gjb
Approved by:	gjb
Differential Revision:	https://reviews.freebsd.org/D8120
2016-11-30 07:16:29 +00:00
Ed Maste
0aa5466e7d Add WITH_LLD_AS_LD build knob
If set it installs LLD as /usr/bin/ld.  LLD (as of version 3.9) is not
capable of linking the world and kernel, but can self-host and link many
substantial applications. GNU ld continues to be used for the world and
kernel build, regardless of how this knob is set.

It is on by default for arm64, and off for all other CPU architectures.

Sponsored by:	The FreeBSD Foundation
2016-11-25 13:15:28 +00:00
Dimitry Andric
67bc8c8b9e Merge ^/head r308491 through r308841. 2016-11-19 16:05:55 +00:00
Ruslan Bukin
7804dd5212 Add full softfloat and hardfloat support for RISC-V.
Hardfloat is now default (use riscv64sf as TARGET_ARCH
for softfloat).

Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D8529
2016-11-16 15:21:32 +00:00
Bryan Drewery
7c6d0bb1a0 Move libsysdecode-specific hack out of buildworld.
This should fix the lib32 build since it was not removing the generated
ioctl.c.  This file is generated by a find(1) call, so cannot use normal
dependency tracking methods.

Reported by:	jhb
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2016-11-13 00:11:15 +00:00
Dimitry Andric
2828dafcf3 Merge ^/head r308227 through r308490. 2016-11-10 22:12:19 +00:00
Antoine Brodin
111bf579e3 Add limits(1) to native-xtools so that it can be used in qemu user-mode jails 2016-11-10 16:27:34 +00:00
Ed Maste
c7fbd7722d Connect new LLVM-based libgcc_eh & libgcc_s to the build
Compiler-rt and LLVM's libunwind provide a suitable replacement for
libgcc.a, libgcc_eh.a, and libgcc_s.so.

Remove the now-unused LLVM_LIBUNWIND block from gnu/lib/libgcc.

PR:		213480 [exp-run]
Reviewed by:	brooks, ed
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D8189
2016-11-04 19:35:49 +00:00
Dimitry Andric
a2b802ce70 Merge ^/head r303250 through r308226. 2016-11-02 19:18:24 +00:00
Jonathan Anderson
e4195e2e12 Add rules to build LLVM IR binaries and libraries.
Running `make libfoo.ll` or `make libfoo.bc` within a library directory
will now give us an LLVM IR version of the library, and `make foo.full.ll`
or `make foo.full.bc` will give us an IR version of a binary.

As part of this change, we add an LLVM_LINK variable to sys.mk that can be
specified/overridden using an external toolchain.

Reviewed by:	bdrewery, brooks
Approved by:	rwatson (mentor)
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D8388
2016-11-01 21:27:42 +00:00
Dimitry Andric
02ebdc7823 Merge ^/head r307736 through r308146. 2016-10-31 19:02:42 +00:00
Ruslan Bukin
5bca221511 Add full softfloat and hardfloat support for MIPS.
This adds new target architectures for hardfloat:
mipselhf mipshf mips64elhf mips64hf.

Tested in QEMU only.

Sponsored by:	DARPA, AFRL
Sponsored by:	HEIF5
Differential Revision:	https://reviews.freebsd.org/D8376
2016-10-31 15:33:58 +00:00
Warner Losh
9013696194 Tweak the UPDATING message a bit about the upgrade path.
Add some automation into Makefile.inc1 to to enforce known good
upgrade from source paths.
2016-10-23 18:00:08 +00:00
Justin Hibbits
dc9b124d66 Create a new MACHINE_ARCH for Freescale PowerPC e500v2
Summary:
The Freescale e500v2 PowerPC core does not use a standard FPU.
Instead, it uses a Signal Processing Engine (SPE)--a DSP-style vector processor
unit, which doubles as a FPU.  The PowerPC SPE ABI is incompatible with the
stock powerpc ABI, so a new MACHINE_ARCH was created to deal with this.
Additionaly, the SPE opcodes overlap with Altivec, so these are mutually
exclusive.  Taking advantage of this fact, a new file, powerpc/booke/spe.c, was
created with the same function set as in powerpc/powerpc/altivec.c, so it
becomes effectively a drop-in replacement.  setjmp/longjmp were modified to save
the upper 32-bits of the now-64-bit GPRs (upper 32-bits are only accessible by
the SPE).

Note: This does _not_ support the SPE in the e500v1, as the e500v1 SPE does not
support double-precision floating point.

Also, without a new MACHINE_ARCH it would be impossible to provide binary
packages which utilize the SPE.

Additionally, no work has been done to support ports, work is needed for this.
This also means no newer gcc can yet be used.  However, gcc's powerpc support
has been refactored which would make adding a powerpcspe-freebsd target very
easy.

Test Plan:
This was lightly tested on a RouterBoard RB800 and an AmigaOne A1222
(P1022-based) board, compiled against the new ABI.  Base system utilities
(/bin/sh, /bin/ls, etc) still function appropriately, the system is able to boot
multiuser.

Reviewed By:	bdrewery, imp
Relnotes:	yes
Differential Revision:	https://reviews.freebsd.org/D5683
2016-10-22 01:57:15 +00:00
Dimitry Andric
5763f79695 Merge ^/head r307383 through r307735. 2016-10-21 16:29:40 +00:00
Ed Maste
c615275ccc Don't build libssp as a prereq lib if WITHOUT_SSP is set
Reviewed by:	brooks
Differential Revision:	https://reviews.freebsd.org/D8301
2016-10-20 21:29:59 +00:00
Ed Maste
801b39194a Fix MK_SHAREDOCS test from r306864
Groff is needed only if MK_GROFF and MK_SHAREDOCS are both true.
2016-10-19 17:42:45 +00:00
Dimitry Andric
242b248284 Merge ^/head r306412 through r306905. 2016-10-09 13:30:57 +00:00
Baptiste Daroussin
24ff5a3223 groff is not needed in the bootstrap tools if the system is built
WITHOUT_SHAREDOCS

MFC after:	2 weeks
2016-10-08 18:57:11 +00:00
Ed Maste
335bcabec9 Merge ^/head r306303 through 306411. 2016-09-28 19:29:01 +00:00
Ed Maste
f987297fc9 Add a WITHOUT_DIALOG src.conf(5) knob
It also turns off dependencies (bsdinstall, bsdconfig, dpv, tzsetup).

Reviewed by:	dteske
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D7969
2016-09-27 18:08:38 +00:00
Dimitry Andric
2aeb03806a Merge ^/head r305220 through r305300. 2016-09-02 19:44:23 +00:00
Bryan Drewery
fcc803dfce Bump __FreeBSD_version for crunchgen META_MODE fix in r305254.
MFC after:	2 weeks
Sponsored by:	EMC / Isilon Storage Division
2016-09-01 23:22:31 +00:00
Dimitry Andric
986e05bc2a Completely revamp the way llvm, clang and lldb are built.
* Bootstrap llvm-tblgen and clang-tblgen with a minimal llvm static
  library, that has no other dependencies.
* Roll up all separate llvm libraries into one big static libllvm.
* Similar for all separate clang and lldb static libraries.
* For all these libraries, generate their .inc files only once.
* Link all llvm tools (including extra) against the big libllvm.
* Link clang and clang-format against the big libllvm and libclang.
* Link lldb against the big libllvm, libclang and liblldb.

N.B.: This is work in progress, some details may still be missing.

It also heavily depends on bsd.*.mk's support for SRCS and DPSRCS with
relative pathnames, which apparently does not always work as expected.
For building llvm, clang and lldb though, it seems to work just fine.

The main idea behind this restructuring is maintainability and build
peformance.  The previous large number of very small libraries, each
with their own generated files and dependencies was slow to traverse
and hard to understand.

Possible future improvements:
* Only build certain targets, e.g. for most regular users having just
  one target will be fine.  This will shave off some build time.
* Building the big llvm, clang and lldb libraries as shared (private)
  libraries.
* Adding other components from the LLVM project, such as lld.
2016-08-26 22:44:22 +00:00
Bryan Drewery
4d611b2b2d Fix in-tree GCC builds after r304681.
There were a few issues.
- In-tree GCC won't have X_COMPILER_TYPE defined but will have
  WANT_COMPILER_TYPE==gcc set from the SYSTEM_COMPILER logic that can
  be used.  Make the clang check specific to clang as well to ensure
  -target doesn't leak into a GCC build.
- When using a cross-compiler GCC (with a default sysroot or arch) and also
  passing --sysroot, it basically forgets all internal paths for
  libraries.  We've already worked around this quite a bit for
  the external toolchains.  Now for the in-tree bootstrap cross-compiler
  GCC, also pass in the needed -B${WORLDTMP}/usr/lib to find the crt
  object files, but also -isystem and -L to fix the paths.  This creates
  quite a spammy build log, but it is clear and still achieves the goals
  and stays consistent between internal and external build flags.
  Reducing the spam by using the '=' prefix feature will help and be
  done later.

MFC after:	3 days
X-MFC-With:	r304681
Reported by:	bz
Pointyhat to:	bdrewery
Sponsored by:	EMC / Isilon Storage Division
2016-08-23 19:29:37 +00:00
Bryan Drewery
bb0cd1b514 Always pass in -target and --sysroot flags for the build.
The internal bootstrap compiler has a default sysroot set by TOOLS_PREFIX
and target set by TARGET/TARGET_ARCH.  However, there are several needs to
always pass an explicit --sysroot and -target.
- External compiler needs sysroot and target flags.
- External ld needs sysroot.
- To be clear about the use of a sysroot when using the internal compiler.
- Easier debugging.
- Allowing WITH_SYSTEM_COMPILER+WITH_META_MODE to work together due to
  the flip-flopping build command when sometimes using external and
  sometimes using internal.
- Allow using no lld which has support for default paths.

The default sysroot in the bootstrap compiler is not changed.  The
buildenv compiler will still work with its default and will also
include -target/--sysroot from CC in the environment.

MFC after:	3 days
Discussed with:	emaste, brooks (BSDCam)
Reviewed by:	emaste
Sponsored by:	EMC / Isilon Storage Division
2016-08-23 15:20:32 +00:00
Bryan Drewery
0295d98c14 Trim unneeded bootstrap after r301470 made 9.1 the minimum supported release.
MFC after:	3 days
Sponsored by:	EMC / Isilon Storage Division
2016-08-16 12:13:12 +00:00
Bryan Drewery
c8de0c57d7 Avoid showing the bootstrap make command for check-old, etc.
Reported by:	koobs
MFC after:	1 week
Sponsored by:	EMC / Isilon Storage Division
2016-08-12 13:52:51 +00:00
Li-Wen Hsu
14009f5e90 Only remove empty directories before packaging.
This preserves files are intentionally empty, most of them are in tests.txz

Reviewed by:	bdrewery
MFC after:	3 days
2016-08-10 18:22:42 +00:00
Mark Johnston
87ae82101f Add libprocstat prebuild library dependencies.
X-MFC-With:	r303531
2016-07-30 03:43:55 +00:00
Mark Johnston
a7e13d50fe librtld_db: Use the auxv to figure out where to look up loader symbols.
Previously, librtld_db just hardcoded /libexec/ld-elf.so, which isn't
correct for processes that aren't using the native ABI. With this change,
librtld_db can be used to inspect non-native processes; in particular,
dtrace -c now works for 32-bit executables on amd64.

MFC after:	1 month
2016-07-30 03:05:23 +00:00
Ed Maste
fd406aa3de Remove ${OBJDUMP} as it is not used by the base system
It was added to sys.mk relatively recently (r274503) for EFI builds
but is no longer used by the base system. The in-tree binutils are
outdated, will not be updated, and will be removed in the future.
Remove it from the toolchain build now to slightly simplify the build
and make sure we don't grow an accidental dependency.

Note that this affects only the toolchain build, and does not affect
/usr/bin/objdump in the built world.

Reviewed by:	bdrewery
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D6460
2016-07-27 14:58:15 +00:00
Bryan Drewery
7337126677 Don't disable binutils/elftoolchain bootstrapping with external compiler.
This was a regression from r300349.

Setting MK_CROSS_COMPILER=no forces the compiler bootstraping *and* the
binutils/elftoolchain bootstrapping to be disabled in share/mk/src.opts.mk.
The only intent with using an external compiler is to disable bootstrapping of
the compiler.  The binutils/elftoolchain bootstrapping must still occur unless
XAS is set.  This did not affect WITH_SYSTEM_COMPILER.

Now that setting an external compiler sets both MK_CLANG_BOOTSTRAP and
MK_GCC_BOOTSTRAP to no, and MK_CROSS_COMPILER does the same via
share/mk/src/opts.mk, remove redundant logic that checks for
MK_CROSS_COMPILER.  It will not always be true now that MK_CROSS_COMPILER==no
when an external compiler is used and --sysroot/-target is needed.

Reported by:	sbruno
Pointyhat to:	bdrewery
MFC after:	3 days
Sponsored by:	EMC / Isilon Storage Division
2016-07-24 18:50:11 +00:00
Bryan Drewery
a92545878a Fix empty WANT_COMPILER_TYPE when neither compiler is bootstrapped.
Bug in r303272.

MFC after:	3 days
X-MFC-With:	r303272
Sponsored by:	EMC / Isilon Storage Division
2016-07-24 18:28:14 +00:00
Bryan Drewery
b17f5b2261 SYSTEM_COMPILER: Rework the logic to allow a 'make test-system-compiler'.
1. Always calculate what the expected values are.
2. Add 'make test-system-compiler' to show all of the computed values
   vs the wanted values.
3. Extend the .info line to buildkernel/kernel-toolchain/toolchain/_cross-tools.
4. Consolidate all of the logic to one condition.

MFC after:	3 days
Sponsored by:	EMC / Isilon Storage Division
2016-07-24 18:05:05 +00:00
Bryan Drewery
4143850b00 Add LOCAL_XTOOL_DIRS to add into cross-tools.
Sponsored by:	EMC / Isilon Storage Division
2016-07-21 06:06:11 +00:00
Ed Maste
fe50ba0583 Include makewhatis in ITOOLS when MK_MAN_UTILS is true
Previously it was conditional on MK_MAN. It's possible to build
FreeBSD with man pages but without man page tools. MK_MAN_UTILS
is the conditional used in share/man/Makefile for determining whether
makewhatis is executed at install time, so it is the proper one for
ITOOLS as well.

PR:		210142
MFC after:	1 week
2016-07-19 18:05:25 +00:00
Cy Schubert
bcc238eaed Revert r302865 and it's sister r302866 as r302865 causes some breakage.
Reported by:	Guy Yur <guyyur@gmail.com>
2016-07-15 15:09:48 +00:00
Cy Schubert
507cef9302 When building multiple kernels using KERNCONF, non-existent KERNCONF
files will produce an error and buildkernel will fail. Previously missing
KERNCONF files silently failed giving no indication as to why, only to
subsequently discover during installkernel that the desired kernel was
never built in the first place.

Reviewed by:		ngie@
MFC after:		1 week
Differential Revision:	D7167
2016-07-15 04:43:38 +00:00
Bryan Drewery
b987685fe0 Revert r302670 and r302671 for now.
MACHINE_CPUARCH smells like MACHINE except for arm64/aarch64 which
has it backwards.
2016-07-13 01:35:53 +00:00
Bryan Drewery
79156eb7c9 Create a TARGET_CPUARCH thing to go with MACHINE_CPUARCH.
MFC after:	3 days
Sponsored by:	EMC / Isilon Storage Division
Differential Revision:	https://reviews.freebsd.org/D7160
2016-07-12 18:57:28 +00:00
Glen Barber
6cdb0cba3e Fix TARGET_TRIPLE for 12.0-CURRENT.
Submitted by:	rene
Sponsored by:	The FreeBSD Foundation
2016-07-11 21:55:56 +00:00
Bryan Drewery
a356b78462 WITH_SYSTEM_COMPILER: Don't enable for xdev or native-xtools.
Otherwise they get no compiler.

Reported by:	swills
Tested by:	swills
Approved by:	re (gjb)
Sponsored by:	EMC / Isilon Storage Division
2016-07-07 21:03:50 +00:00
Bryan Drewery
48413367ef WITH_SYSTEM_COMPILER: Avoid using GCC4.8+ logic on native GCC4.2 platforms.
As the XXX notes, these should really be checking MK_GNUCXX since there is
already a version check in share/mk/src.opts.mk to disable it.  Fixing that
here is more complex though.  This could also be using X_COMPILER_FEATURES
but uses X_COMPILER_VERSION to keep in sync with the src.opts.mk logic.

Tested by:	andreast
Sponsored by:	EMC / Isilon Storage Division
Approved by:	re (gjb)
2016-06-29 01:50:30 +00:00
Bryan Drewery
0739409184 Fix native powerpc64 build of lib32 with in-tree GCC.
- This was broken by r300350 and r300885.
- Add some comments around the external GCC logic since it is spread out
  and in need of some cleanup.
- The problem was that X_COMPILER_TYPE is always defined from CC->XCC's
  default, so if /usr/bin/cc is GCC (as it is on native powerpc64) then
  X_COMPILER_TYPE was getting GCC and triggering the external logic in
  Makefile.libcompat.  It was intended to always provide -isystem with
  GCC since --sysroot is used into the lib32 sysroot which won't modify
  the header path without the -isystem.  The use of the libc++/std=c++11
  override was only intended to be used for external compilers though
  (more accurately GCC 4.8+ but that's a separate assumption to
  cleanup).  Apply the same logic from Makefile.inc1 to Makefile.libcompat
  to only add the libc++ override when needed for external compilers.

Pointyhat to:	bdrewery
Tested with:	native ppc64 (swills), universe, ppc64 xtoolchain,
		amd64 xtoolchain, sparc64 cross-build of ppc64 (host GCC 4.2)
Reported by:	andreast, swills
Approved by:	re (gjb)
Sponsored by:	EMC / Isilon Storage Division
2016-06-15 23:58:03 +00:00
Jilles Tjoelker
d80f1dd1d7 build: Add legacy support for futimens() and utimensat().
In order to allow using utimensat() in install(1), add futimens() and
utimensat() to -legacy.

The files futimens.c and utimensat.c are modified copies of the files under
lib/libc/sys/ since the libc versions use symbols that do not exist in the
libc on the build system (sys_futimens and sys_utimensat) . I expect the
next non-sweeping change to both sets of files to be to delete them, anyway.

This will allow reverting r299942 (which is a revert of r299850) enabling
nanosecond timestamps in install(1).

Reviewed by:	bdrewery
2016-06-09 21:57:34 +00:00
Bryan Drewery
aa06dfc9c3 legacy: Avoid building/installing headers twice.
Sponsored by:	EMC / Isilon Storage Division
2016-06-05 23:56:28 +00:00
Bryan Drewery
4d40df1bff Add a MINIMUM_SUPPORTED_OSREL and bump it to 900044.
This is actually a revision in the stable/9 branch released as 9.1.  The
localedef build requires xlocale from this period.

Sponsored by:	EMC / Isilon Storage Division
2016-06-05 23:05:10 +00:00
Bryan Drewery
95348a57d0 Fix incorrect logic in r301394.
Reported by:	Mark Millard
2016-06-04 17:38:17 +00:00
Bryan Drewery
b4dd422926 Follow-up r301287: Pass external compiler metadata when used.
This fixes WITH_SYSTEM_COMPILER, WITHOUT_CROSS_COMPILER, external compiler, to
pass the external compiler metadata rather than the ${CC} metadata.  On
a build host that has clang as CC it was passing the clang metadata rather than
GCC metadata during the build.
2016-06-04 17:17:10 +00:00
Bryan Drewery
bbfcc681bc WITHOUT_CROSS_COMPILER: Fix installworld.
Since no WORLDTMP/usr/bin/cc is created, cc cannot be found
during installworld time since /usr/bin is not in the PATH.
Pass along the known compiler metadata to allow installworld
to work.  The same fix was used for WITH_SYSTEM_COMPILER.

A better route would be to store a cookie in buildworld
containing this compiler metadata and then using that
at install time, rather than rerunning cc.

Reported by:	Mark Millard
Sponsored by:	EMC / Isilon Storage Division
2016-06-03 20:49:49 +00:00
Dimitry Andric
c371433e4a For clang, move the definition of FREEBSD_CC_VERSION into its own header
file, lib/clang/freebsd_cc_version.h, instead of reusing Version.inc.
The header is only included from one .cpp file in the clang tree.

This minimizes the number of .cpp files that need to be rebuilt if the
version is bumped.

Discussed with:	bdrewery
2016-06-03 16:17:36 +00:00
Bryan Drewery
f661dbee81 GCC External: Revert r300886, r300904, r300917, r300918
The fix in r300873 is mostly enough.  A fix for lib32 will be
committed.separately.
2016-05-29 06:20:15 +00:00
Bryan Drewery
ce00342bb2 Move external GCC compiler hacks to bsd.sys.mk.
This allows respecting -nostdinc, -nostdinc++ and -nostdlib before
making the decision to add in -isystem, etc.  The -isystem flags
are problematic for building lib/libc++ and lib/libcxxrt which wants
to only use its own headers.

More information the need of these flags can be found at
https://gcc.gnu.org/ml/gcc/2016-03/msg00219.html

This also reverts r300873.

Sponsored by:	EMC / Isilon Storage Division
2016-05-27 23:03:44 +00:00
Glen Barber
5929ee2961 Make PKG_VERSION aware of '-ALPHA*'.
Submitted by:	Ben Woods
Sponsored by:	The FreeBSD Foundation
2016-05-27 20:33:14 +00:00
Bryan Drewery
eb9b07bb77 WITH_AUTO_OBJ: Fix crunchgen builds.
Passing MAKEOBJDIRPREFIX to the main prog build (rescue) would confuse
WITH_AUTO_OBJ and cause it to create a recursed object directory that
then broke the actual prog build.  This is normally not a problem since
we do not call 'make -f prog.mk obj' before building anything in it.

Crunchgen(1) also assumes that if -o is not passed then if an object
directory does not already exist then it should build in the source
directories.  The normal buildworld process will have already ran 'make
obj' in each of the component directories so this is not a problem.
With WITH_AUTO_OBJ though this is not the case.  So we must tell
crunchgen(1) that MK_AUTO_OBJ=yes will create the directory and to not
require it be present before generating its Makefile.

Sponsored by:	EMC / Isilon Storage Division
2016-05-26 23:20:36 +00:00