Some background: in the GSoC project, libbe/Makefile lived in lib/libbe. I
created projects/bectl branch, maintained the above for all of five
minutes before I misread Makefile.inc1 and decided that it couldn't possibly
build outside of cddl/, so I kicked the Makefile out into the cddl/ build
and all was good. The misreading was of the bit where .WAIT is added to
SUBDIR after lib, libexec but prior to building bin and cddl *only during
the install targets*, which is the critical part.
Fast forward- buildworld was still broken in my branch unbeknownst to me
because I didn't nuke my OBJDIR. Combing through Makefile.inc1 eventually
revealed the necessary magic to make sure that libbe's dependencies are
specified well enough, and it becomes clear what needs done to make a
non-cddl/ build work. This is an interesting prospect, because the build
split is kind of annoying to work with.
IGNORE_PRAGMA is added to avoid dropping WARNS by one more. This was
previously pulled in via cddl/Makefile.inc.
In the create-world-packages target we manually piece this together (unless
it is undefined), without the DISTDIR. Normally DISTDIR is empty (unset) and
no one notices. Now DISTDIR is a well known long-standing PORTS environment
variable and if that is set in the local environment the path to METALOG
is wrong as it no longer is ${DESTDIR}/METALOG.
Long-term we should start to avoid "publicly well known" names for global
variables, for now just piece ${DISTDIR} in as well. This allows
create-world-packages to continue if DISTDIR is set in the env.
There are several scripts and targets solely used to generate install
media, make sure DB_FROM_SRC is used in that case in order to prevent
checking the host database, which is irrelevant when generating
install binaries.
Sponsored by: Citrix Systems R&D
PR: 230459
Reviewed by: gjb
Differential revision: https://reviews.freebsd.org/D16638
- Missing include path
- Fully specify libzfs's dependencies (except for deps pulled in by other
deps) in Makefile.inc1
- Drop WARNS back down to 2 for libbe(3). I do this with much hesitation,
but the libzfs headers are apparently a hot warning-filled mess as far as
GCC 4.2 is concerned.
libl is needed for config(8), which is a bootstrap-tool. It is possible to
build a system WITHOUT_TOOLCHAIN to exclude lex and thus, libl. We still
need to support building from this kind of host, though.
While here, group the config(8) dependencies together and add a small
explanation. These can likely both be scoped more clearly, but this will
need some further investigation.
Reported by: rgrimes (not WITHOUT_TOOLCHAIN, but provoked investigation)
MFC after: immediately
distributeworld is used to generate install media, so it makes no
sense to check the host database since the install media can be
generated from any box, regardless of the version of FreeBSD it's
running.
Sponsored by: Citrix Systems R&D
Reviewed by: ian, gjb
Differential revision: https://reviews.freebsd.org/D16507
r335653 flipped the order in which hints/env files are concatenated to match
the order in which vars are processed by the kernel. This is the other
hammer to drop.
Use nv(9) to de-dupe entries within a single `hint` or `env` file, using the
latest value specified for a key. This leaves some duplicates if a variable
is specified in multiple hint/env files or via `envvar` in a kernel config,
but the reversed order of concatenation (from r335653) makes this a
non-issue as the latest-specified version will be seen first.
This change also silently rewrote hint bits to use the same sanitization
process that ian@ wrote for r335642. To the kernel, hints and env vars are
basically the same thing through early boot, then get merged into the
dynamic environment once kmem becomes available and the dynamic environment
is created. They should be subjected to the same restrictions.
libnv has been added to -legacy for the time being to support the build of
config(8) with the new cnvlist API.
Tested with: universe (11 host & 12 host)
MFC after: 1 month
The build of this would have MK_LLVM_TARGET_ALL=yes for bootstrap-tools
but MK_LLVM_TARGET_ALL=no for cross-tools and thus would rebuild between
the two.
MFC after: 2 weeks
X-MFC-with: r335707
Sponsored by: Dell EMC
Currently the mtree calls in Makefile.inc1 all change the directory owner
to match the spec file. However, we should not be doing this during
distributeworld if -DNO_ROOT is passed. Additionally, when creating the
WORLDTMP directory hierachy there is no need to change the owner to root so
we now always pass the -W flag when populating WORLDTMP.
This is also required for building FreeBSD on Linux/Mac since the required
groups/users will not exist there which is how I discovered this issue.
Reviewed By: emaste, bdrewery, imp
Approved By: brooks (mentor)
Differential Revision: https://reviews.freebsd.org/D14185
The xtoolchain GCC packages have not required these flags since ports
commits r465416 and r466701. The in-tree GCC 4.2.1 has also been patched
in r335716 and r335717 to correctly honor --sysroot when looking for
includes and libraries.
Reviewed by: bdrewery
Sponsored by: DARPA / AFRL
Differential Revision: https://reviews.freebsd.org/D16055
Need to handle LLD_BOOTSTRAP separately (for archs like i386).
This would be much better off with an off-by-default option like
SHARED_TOOLCHAIN that universe force-enabled. Then a normal buildworld
would store the toolchain there if enabled and otherwise in WORLDTMP
with only the 1 arch selected.
MFC after: 3 weeks
Sponsored by: Dell EMC
This will disable the new LLVM_TARGET_ALL option which will only
enable the required target.
This only impacts the bootstrap compiler in WORLDTMP, not the target compiler
that will be installed.
MFC after: 2 weeks
Reviewed by: sbruno, dim (earlier version)
Sponsored by: Dell EMC
Differential Revision: https://reviews.freebsd.org/D16021
LLVM_TARGET_* will auto be set based on LLVM_TARGET_ALL and MK_CLANG.
If LLVM_TARGET_ALL is disabled, during a cross-build, then SYSTEM_COMPILER
and SYSTEM_LINKER are auto disabled.
This option should be used by users rather than the per-arch LLVM_TARGET
options as it is simpler to maintain for them should the supported
target list change.
MFC after: 2 weeks
Reviewed by: sbruno, dim
Sponsored by: Dell EMC
Differential Revision: https://reviews.freebsd.org/D16020
Rather then combining hardlink creation for the geom(8) binary with
shared library build, move libraries to src/lib/geom so they are
built and installed normally. Create a common Makefile.classes
which is included by both lib/geom/Makefile and sbin/geom/Makefile
so the symlink and libraries stay in sync.
The relocation of libraries allows libraries to be build for 32-bit
compat. This also reduces the number of non-standard builds in
the system.
This commit is not sufficent to run a 32-bit /sbin/geom on a 64-bit
system out of the box as it will look in the wrong place for libraries
unless GEOM_LIBRARY_PATH is set appropriatly in the environment.
Reviewed by: bdrewery
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D15360
We always install ELF Tool Chain's elfcopy as objcopy, so to avoid
confusion rename the src directory containing our reach-over Makefile
to match.
Requested by: jhb
Sponsored by: The FreeBSD Foundation
This works similar to WITH_SYSTEM_COMPILER added in r300354. It only
supports lld via WITH_LLD_BOOTSTRAP.
When both SYSTEM_COMPILER and SYSTEM_LINKER logic passes then libclang
will not build in cross-tools. If either check fails though then
libclang is built.
The .info is reworked to notify when libclang will be built since if
either clang or lld needs to be rebuilt, but not the other, the
notification can lead to confusion on why "clang is building".
-fuse-ld= is not used with this method so some combinations of compiler
and linker are expected to fail.
A new 'make test-system-linker' target is added to see the logic results.
Makefile.inc1:
CROSS_BINUTILS_PREFIX support had to be moved higher up so that XLD
could be set and MK_LLD_BOOTSTRAP disabled before checking SYSTEM_LINKER
logic as done with SYSTEM_COMPILER. This also required moving where
bsd.linker.mk was read since XLD needs to be set before parsing it. This
creates a situation where src.opts.mk can not test LINKER_FEATURES or
add LLD_BOOTSTAP to BROKEN_OPTIONS.
Reviewed by: emaste (earlier version)
Sponsored by: Dell EMC
Differential Revision: https://reviews.freebsd.org/D15894
MK_LLD is for the installed lld while MK_LLD_BOOTSTRAP is for the build
tool. For WITH_SYSTEM_LINKER it is necesarry to separate the logic of
these two. When building libllvm TOOLS_PREFIX will be defined and
MK_LLD_BOOTSTRAP should be checked instead.
Sponsored by: Dell EMC
Differential Revision: https://reviews.freebsd.org/D15837
LLD_BOOTSTRAP (build) is independent of LLD_IS_LD (installed) so they
should not be based on each other.
This is related to upcoming WITH_SYSTEM_LINKER work.
Reviewed by: emaste
Sponsored by: Dell EMC
Differential Revision: https://reviews.freebsd.org/D15836
It is XCC used during the build of target binaries that replaces the
bootstrap compiler.
Also slightly tweak style.
MFC after: 2 weeks
Sponsored by: Dell EMC
This could lead to 'sh: head: not found' warnings which were a symptom
of running 'ld --version' during installworld. This was only happening
with XCC or XLD set. It is intended that cc and ld do not run during
installworld. The metadata for these are already stored in
compiler-metadata.mk added in r316794.
This also removes redundant CROSSENV additions that were for
WITH_SYSTEM_COMPILER, WITHOUT_CROSS_COMPILER, and WITHOUT_TOOLCHAIN
which all don't have a cc or ld in their PATH during install.
Reported by: Mark Millard
MFC after: 2 weeks
Sponsored by: Dell EMC
Aligns the build with the FreeBSD traditional approach to not build in
contrib/, and to track inter-dependencies between libraries.
With help from: bdrewery
Reviewed by: bdrewery, hselasky
Sponsored by: Mellanox Technologies
MFC after: 2 weeks
Differential revision: https://reviews.freebsd.org/D15648
vendor provided pmu-events tables and sundry cleanups.
The vendor pmu-events tables provide counter descriptions, default
sample rates, event, umask, and flag values for all the counter
configuration permutations. Using this gives us:
- much simpler kernel code for the MD component
- helpful long and short event descriptions
- simpler user code
- sample rates that won't overload the system
Update man page with newer sample types and remove unused sample type.
vendor provided pmu-events tables and sundry cleanups.
The vendor pmu-events tables provide counter descriptions, default
sample rates, event, umask, and flag values for all the counter
configuration permutations. Using this gives us:
- much simpler kernel code for the MD component
- helpful long and short event descriptions
- simpler user code
- sample rates that won't overload the system
Update man page with newer sample types and remove unused sample type.
Squashed commit of the following:
commit 4459d43eff815bec08ccc5533dbe5de846f03128
Author: Matt Macy <mmacy@mattmacy.io>
Date: Sat May 26 00:06:31 2018 -0700
libpmc: fix pmu function signatures for non amd64
commit a2cb8bbc586c65d41f9b291430a2261ec67b59fe
Author: Matt Macy <mmacy@mattmacy.io>
Date: Fri May 25 22:38:11 2018 -0700
pmcstat: fix indentation of usage
commit f686954b15ff56a833ac80404898977cb80a265b
Author: Matt Macy <mmacy@mattmacy.io>
Date: Fri May 25 22:19:49 2018 -0700
pmclog(3): add callchain and pmcallocatedyn, remove pcsample
commit 73e13a0d2e9498c81c150d14d022050cee7511bb
Author: Matt Macy <mmacy@mattmacy.io>
Date: Fri May 25 22:19:00 2018 -0700
pmclog.h: GC pcsample field
commit 3e93ffd65da641fa657539dad3c48e281f8b5798
Author: Matt Macy <mmacy@mattmacy.io>
Date: Fri May 25 22:05:57 2018 -0700
hwpmc: make Intel core CPUs use external event tables
commit 634f5fae1e1644ac324003136c66cd9c619d1c93
Author: Matt Macy <mmacy@mattmacy.io>
Date: Fri May 25 22:00:06 2018 -0700
pmclog: update log record types, bump PMC_MAJOR
- explicitly make log record types a multiple of 8 bytes
- hook in pmu event types for pmc_allocate records
- remove references to no longer PCSAMPLE record
commit 83d84fcd2d65bdf6ddcb2e155a22f0cfa2a9c225
Author: Matt Macy <mmacy@mattmacy.io>
Date: Fri May 25 21:52:10 2018 -0700
libpmc: add support for having vendor table driven pmc_allocate
commit 9e6ad63c40c2fce8404847ace5078ca6cb33a736
Author: Matt Macy <mmacy@mattmacy.io>
Date: Fri May 25 19:11:33 2018 -0700
hwpmc_core: add accessors for EVSEL & UMASK, make IAP_UMASK useful to user
commit 859dceb93daa6419a48c794db99b6758e5b041c9
Author: Matt Macy <mmacy@mattmacy.io>
Date: Fri May 25 19:09:45 2018 -0700
pmcstat: update usage and man page as well as make -L consistent with pmccontrol
commit 79c7d8597e28c2eb13f5f9113e65ec2792ca57b1
Author: Matt Macy <mmacy@mattmacy.io>
Date: Fri May 25 18:07:03 2018 -0700
pmu_util: add support for all current intel event keywords
commit d8089c7f6a6c8527f38324252b1ffb47004694c6
Author: Matt Macy <mmacy@mattmacy.io>
Date: Fri May 25 17:45:00 2018 -0700
add description for new arguments
commit 058336740bab53c62ec88a3a026ea848cf3878c6
Author: Matt Macy <mmacy@mattmacy.io>
Date: Fri May 25 17:38:15 2018 -0700
libpmc: move pmu_events table and pmu_utils out of libpmcstat so that they can be used by pmc_allocate
commit 049b66b382e2f833c3f47bc8df9e750cb265709f
Author: Matt Macy <mmacy@mattmacy.io>
Date: Fri May 25 16:12:41 2018 -0700
pmcstat: hook pmu_events counter description utility routines in
commit f5e01e7b37a691dc045e1aa16b3ebdd162515de8
Author: Matt Macy <mmacy@mattmacy.io>
Date: Fri May 25 16:11:59 2018 -0700
pmu_events: add utility routines for listing counters and their descriptions
commit cba4d4f8907f772279f86f18f915e0d74d33ac56
Author: Matt Macy <mmacy@mattmacy.io>
Date: Fri May 25 16:09:50 2018 -0700
pmu-events: expand out skylake regex to simplify string matches
r307825 and r333017 disallow building FreeBSD-HEAD from 9.x or 10.x
versions that have some clang issues. The minimum supported Subversion
revisions and osreldates containing the fix are:
Version Min Rev osreldate
9.x r286035 903509
10.x r286033 1002501
9.3 is the final 9.x release and does not contain the r286035 fix.
10.3 is the first 10.x release with the fix. Thus, in practice 10.3 is
the oldest release that can build HEAD.
Although it may still be possible to build HEAD from an up-to-date
stable/9 it's not worth maintaining the special case when the branch
itself is unsupported and there are no usable releases from that branch.
Old UPDATING entries can be removed and the Clang warning in UPDATING
may be updated, in a future commit.
Approved by: imp
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D15209
r318957 added special handling for stale ptrace dependency files to
support a -DNO_CLEAN build in a tree last built before r305012. That
revision is now over a year and a half old, so retire the special case.
Sponsored by: The FreeBSD Foundation
So that -DNO_CLEAN builds work.
This should be reverted after a sufficient transition period (perhaps 3
or 6 months).
Sponsored by: The FreeBSD Foundation
kernel.ucl uses a hardcoded boot/kernel for kldxref, which is the incorrect
directory when we're installing extra kernels that aren't the "default"
kernel (placed at boot/kernel).
Fix this by instead using a new %KERNELDIR% that we now replace in
Makefile.inc1 with "kernel" for the default kernel and "kernel.${_kernel}"
for these extra kernels so that, e.g. /boot/kernel.SHIVA, will get properly
kldxref'd upon update and avoid outdated linker.hints.
Reviewed by: gjb
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D14689
This allows working with custom cross toolchains without the need to
create files in /usr/local/share/toolchains.
Obtained from: CheriBSD
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D14178
This code, which is basically `svnversion || svnliteversion`, generates
2 fstatat(2) for every directory in PATH for every Makefile parsed that
includes bsd.own.mk. This can add up for things like generating a Ports
index (Poudriere) or building a dependency graph for base.
Sponsored by: Dell EMC
MFC after: 2 weeks
bsd.compiler.mk. It's so fmake from older 9.x systems still
works (still a supported build config, and having the note here
will let us know when we can cull it more easily).
Also pull in a related change from include to sinclude from
arichardson@'s cross building work, as well as it's companion in
Makefile.inc1 with a note about why we do the odd thing there.
Submitted by: archardson
Differential Revision: https://reviews.freebsd.org/D14241
These features indicate that the compiler and linker support the
retpoline speculative execution vulnerability (CVE-2017-5715)
mitigation.
Reviewed by: dim, imp
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D14228
It won't work e.g. when crossbuilding from Ubuntu Linux as mktemp is in
/bin there.
Reviewed By: bdrewery
Approved By: jhb (mentor)
Differential Revision: https://reviews.freebsd.org/D13937
This fixes building libllvmminimal with MK_LLD support when WITHOUT_LLD
or WITHOUT_TOOLCHAIN are set, which are install flags not build tools
flags.
Reported by: sbruno
Tested by: sbruno
Sponsored by: Dell EMC
A comment in Makefile.inc1 has long stated that LOCAL_DIRS are built last,
after the base system. Incremental improvements in parallel building over
the years have led to LOCAL_DIRS being built in parallel with base system
directories. This change allows the .WAIT directive to appear in LOCAL_DIRS
and LOCAL_LIB_DIRS lists to give the user some control over parallel
building of local additions.
Differential Revision: https://reviews.freebsd.org/D13622
Because we force enable MK_GCC when building we need to also force
enable it for the cleaning phase. Otherwise the NXB_TARGET files
are found in the next build's kernel-toolchain phase and cause
an error.
Reported by: sbruno
X-MFC-With: r325001
MFC after: 1 month
Sponsored by: Dell EMC
The initial kernel-toolchain is built with TARGET=MACHINE but
we want GCC to have files generated for TARGET=NXB_TARGET
instead later on. Just clean the files between building of
the toolchain and nxb binaries which will let it rebuild
when needed.
Reported by: sbruno
X-MFC-With: r325001
MFC after: 1 month
Sponsored by: Dell EMC
Packaged base packages are created by running the stageworld and
stagekernel targets with -DNO_ROOT, and converting the resulting mtree
file into a set of pkg plists. If stage* is run with multiple processes
the order of entries in the mtree file may be nondeterministic, and the
resulting package tbz also had nondeterministic file ordering.
The mtree file generated by -DNO_ROOT builds consists of one line per
file, with the filename starting in the first column, so is easily
sorted. There's one exception: the first line of the mtree file is a
comment, but the # character sorts before the filenames anyhow and needs
no special treatment.
PR: 223673
Reviewed by: bapt, gjb
Sponsored by: The Linux Foundation, Core Infrastructure Initiative
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D13103
Before this patch libmagic was always built in the build-tools phase
even if -DWITHOUT_FILE was specified.
Reviewed by: emaste, jhb
Approved by: jhb (mentor)
Differential Revision: https://reviews.freebsd.org/D12925
- This also adds in a _cleanobj step as needed.
- This redirects TARGET/TARGET_ARCH to NXB_TARGET/NXB_TARGET_ARCH in
Makefile.inc1 as the main build needs to be for MACHINE rather
than TARGET.
First build the toolchain and then use that as an external toolchain
to build the needed directories and NXB_TARGET-toolchain, all as
MACHINE files though via TARGET_TRIPLE=MACHINE_TRIPLE.
The NXBDIRS is evaluated in the 'everything' submake as it needs to be
based on TARGET's src.opts.mk values, such as MK_GCC=yes when building
on a MK_CLANG=yes MACHINE. This can likely be changed to a specific
_native-xtools-everything target later and the funky late evaluation
of SUBDIR_OVERRIDE removed.
X-MFC-With: r325001
Pointyhat to: bdrewery
MFC after: 2 months
Sponsored by: Dell EMC Isilon
- If OBJROOT is SRCTOP then don't add on TARGET.TARGET_ARCH. This
only happens at the top-level, and for sub-directories when the
user is clever with MAKEOBJDIRPREFIX=/.
- Don't bother checking 'test -w' on .CURDIR.
- Properly set OBJTOP/OBJROOT to SRCTOP in various needed cases.
- Check if the OBJDIR is writable even for *clean* targets since it
determines which .OBJDIR the user gets; If they cannot write to an
existing eligible .OBJDIR then it needs to clean in .CURDIR instead.
- Add guard to cleanworld/cleanuniverse from removing SRCTOP.
- Ensure OBJTOP is proper for .OBJDIR=.CURDIR which fixes finding
libraries since src.libnames.mk is based on OBJTOP.
- Avoid some chdir(2) for modifying .OBJDIR
Sponsored by: Dell EMC Isilon
This problem was caused by r325329 and r325350.
For the release(7) targets, some will run mm-mtree.sh which itself runs make
with a MAKEOBJDIRPREFIX. The execution of that script leaks OBJROOT,
MAKEOBJDIR, and MAKELEVEL=1 in the environment. This causes the mm-mtree makes
to not do some basic setup of OBJROOT and only use this special
MAKEOBJDIRPREFIX case which fails to empty out MAKEOBJDIRPREFIX for further
nested makes, such as a tree walk. If that tree walk sets OBJROOT/OBJTOP such
as r325329 is doing, then the wrong OBJDIRs end up being used due to the
unemptied MAKEOBJDIRPREFIX being preferred over the proper MAKEOBJDIR.
Pointyhat to: bdrewery
Sponsored by: Dell EMC Isilon
NO_OBJ has a very specific meaning in sub-directories in that no object
directory will be made. If a user wanted to skip the 'make obj' phase then
passing -DNO_OBJ would break all sub-directories from building properly. Using
NO_OBJ internally also causes issue with NO_OBJ handling being added in
share/mk/bsd.init.mk soon.
Sponsored by: Dell EMC Isilon
This still keeps the reduced MAKEOBJDIRPREFIX (SRCTOP) redundancy
removed in the OBJDIR, but now keeps all early phase objects
in the same directory rather than split per phase.
The problem of splitting per phase is that later phases want to link in
libraries from earlier phases and base their location on ${OBJTOP}.
Pointyhat to: bdrewery
Reported by: mjoras, Mark Millard
Sponsored by: Dell EMC Isilon
This will force any existing objects to rebuild if their .meta
files reference files from WORLDTMP. This is a problem after
the UNIFIED_OBJDIR effort caused buildworld and DIRDEPS_BUILD
to share an OBJDIR. Without cleaning these files the
Makefile.depend files end up with odd tmp/legacy/... entries
since some bootstrap-tools linger from there and otherwise
don't rebuild. Removing the files causes anything having
used WORLDTMP to rebuild.
Sponsored by: Dell EMC Isilon
Given MACHINE/MACHINE_ARCH=amd64, TARGET=arm64 and TARGET_ARCH=aarch64,
this change will build them in a location such as:
/usr/obj/usr/src/amd64.amd64/nxb/arm64.aarch64/
and
/usr/obj/usr/src/amd64.amd64/xdev/aarch64-freebsd/
Sponsored by: Dell EMC Isilon
This changes the build OBJDIR from the older style of /usr/obj/<srcdir> for
native builds, and /usr/obj/<target>.<target_arch>/<srcdir> for cross builds to
a new simpler format of /usr/obj/<srcdir>/<target>.<target_arch>. This
new format is used regardless of cross or native build. It allows
easier management of multiple source tree object directories.
The UNIFIED_OBJDIR option will be removed and its feature made permanent
for the 12.0 release.
Relnotes: yes (don't note UNIFIED_OBJDIR option since it will be removed)
Prior work: D3711 D874
Reviewed by: gjb, sjg
Discussed at: https://lists.freebsd.org/pipermail/freebsd-arch/2016-May/017805.html
Discussed with: emaste
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D12840
Due to removing some targets that the previous .ORDER: ${WMAKE_TGTS}
set, it was no longer being respected; _build_tools was coming
immediately.
Pointyhat to: bdrewery
X-MFC-with: r325244
MFC after: 2 weeks
Sponsored by: Dell EMC Isilon
There's no good reason to treewalk the entire tree removing old OBJDIRS
and creating new ones when 'includes', 'libraries', and 'everything' are
all skipped. The only shared directory in the existing toolchain target
and world is build-tools. So handle cleaning in it directly if needed
only for the directories it wants to build.
The extra _obj/_cleanobj walks came in the initial kernel-toolchain
addition in r128189.
MFC after: 2 weeks
Sponsored by: Dell EMC Isilon
It merely wanted to use 'all' rather than 'build-tools' so just
add a build-tools target to the Makefile.
MFC after: 2 weeks
Sponsored by: Dell EMC Isilon
Now that a proper sysroot is used and a separate target list, it should
be safe to skip building the initial cross-compiler.
X-MFC-With: r325001
MFC after: 1 month
Sponsored by: Dell EMC Isilon
make buildenv BUILDENV_SHELL=<some command> more useful. Remove '||
true' from the command line so that errors are properly
returned. There appears to be no reason for it, and it dates back to
the original commit by ru@.
Sponsored by: Netflix
It is possible that building headers requires an OBJDIR.
The other phases of xdev have their own 'make obj' calls
where needed, such as inside 'make libraries' itself.
MFC after: 2 weeks
Sponsored by: Dell EMC Isilon
Without this the user has to mess with 'make -f Makefile.inc1 ...' to figure
out where the files are installed in the OBJDIR and then they need to copy them
to where they really wanted them. Using DESTDIR may be problematic after
r325001 as well.
The files will be installed to DESTDIR/NXTP where NXTP defaults to /nxb-bin.
MFC after: 2 weeks
Sponsored by: Dell EMC Isilon
The top of Makefile.inc1 requires TARGET/TARGET_ARCH be defined. Just
building 'make xdev' would already set them, so this error was never
triggered. Moving it to Makefile fixes the problem.
MFC after: 2 weeks
Sponsored by: Dell EMC Isilon
Its own build target was already handling mtree extractions
just as _worldtmp did, so the other cleaning of the
tmpdir makes sense here as well.
Sponsored by: Dell EMC Isilon
Only remove them if the option is enabled and also handle libsoft
by using the proper LIBCOMPAT_OBJTREE. LIBCOMPAT:D will expand
the text after it as a proper glob to the command line if LIBCOMPAT
is defined.
Sponsored by: Dell EMC Isilon
The _worldtmp target is for setting up WORLDTMP. Nothing between _worldtmp
and _cleanobj will read these files. Move to its own target since it is
so large.
Sponsored by: Dell EMC Isilon
This leaks into the PATH handling for WORLDTMP and breaks
finding cross-tools. The PATH handling could be fixed to
properly quote but is not worth the effort.
Also allow this sanity check to always run even with 'make -n'.
MFC after: 2 weeks
Sponsored by: Dell EMC Isilon
This takes longer but should reliably produce working binaries.
The old version linked against system libraries and headers
which would be a problem if building a native-xtools against
a newer source than the host system. With a proper sysroot made
first this is not a problem.
This also allows:
- NXBDIRS to be built in parallel
- NXBDIRS to be installed to NXBDESTDIR in parallel
- SYSTEM_COMPILER logic to work again. Note that because this change
is adding a sysroot phase the compiler may be built up to twice now.
The first is the "cross-compiler" even though it is for the native
architecture, but it is still built to ensure the latest compiler
is used to generate the binaries, unless SYSTEM_COMPILER allows
/usr/bin/cc to be used. Then the target compiler is built
which is actually a cross-compiler since it runs on native host
but generates TARGET.TARGET_ARCH binaries.
Note this also changes the path used for the OBJDIR. It used to use
/usr/obj/target.target_arch/nxb/<srcdir> for objects and
/usr/obj/target.target_arch/nxb-bin for installed files, but now uses
/usr/obj/nxb/target.target_arch/<srcdir> for objects and
/usr/obj/nxb/target.target_arch/<srcdir>/nxb-bin for installed files.
- NXBDESTDIR can be specified for where to install or queried with
`make -f Makefile.inc1 TARGET=... TARGET_ARCH=... -V NXBDESTDIR`
This could potentially be improved to reuse an existing sysroot. The
problem is with building the SUBDIR_OVERRIDE list it needs to use a
different OBJDIR since it is building all statically. We don't want to
pollute the existing 'buildworld' OBJDIR and cause confusion on the next
build. Using a separate OBJDIR for the 'everything' phase mostly works
except for some things like linking in INTERNALLIBS that exist in the
other OBJDIR.
MFC after: 1 month
Sponsored by: Dell EMC Isilon
Make armv7 as a new MACHINE_ARCH.
Copy all the places we do armv6 and add armv7 as basically an
alias. clang appears to generate code for armv7 by default. armv7 hard
float isn't supported by the the in-tree gcc, so it hasn't been
updated to have a new default.
Support armv7 as a new valid MACHINE_ARCH (and by extension
TARGET_ARCH).
Add armv7 to the universe build.
Differential Revision: https://reviews.freebsd.org/D12010
files will produce an error and buildkernel will fail. Previously missing
KERNCONF files silently failed giving no indication as to why, only to
subsequently discover during installkernel that the desired kernel was
never built in the first place.
This is a rework of r302865.
This is the correct patch.
Reviewed by: ngie (previous version, r302865)
MFC after: 2 months
Differential Revision: D7167
files will produce an error and buildkernel will fail. Previously missing
KERNCONF files silently failed giving no indication as to why, only to
subsequently discover during installkernel that the desired kernel was
never built in the first place.
This is a rework of r302865.
Reviewed by: ngie (previous version, r302865)
MFC after: 2 months
Differential Revision: D7167
This causes builds to use the in-tree libdwarf with the bug fix instead
of the host library.
Reviewed by: bdrewery, emaste, markj
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D12239
This helps reduce the WORLDTMP footprint slightly.
Based on a patch I submitted 5 years ago to GNATS.
PR: 174051
MFC after: 2 weeks
Relnotes: yes (anyone who cross-builds with MK_TCSH=yes will run into
build failures if the host doesn't have tcsh(1))
Reminded by: Fabian Keil <fk@fabiankeil.de>
LLD_BOOTSTRAP is intended to control the linker used to link world and
kernel, while LLD_IS_LD is intended to control the linker installed in
that world.
Force LLD_IS_LD equal to LLD_BOOTSTRAP for the cross-tools build and
install phase, so that lld will be installed as the ld to run on the
host, when LLD_BOOTSTRAP is set.
PR: 221543
Reviewed by: dim
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D12072
Disconnect the dependency on the kernel package from the runtime
package. There are a number of problems here:
1) The runtime package installed into a chroot or a jail would
include the kernel package, changing the behavior of how jails
work now [1];
2) As result of (1), it is possible a binary may incorrectly
resolve kernel symbols [2]; in addition, it is possible there
will be unexpected fallout with 32-bit jails on a 64-bit host
kernel [2].
Noticed by: brd [1]
Discussed with: kib [2]
MFC after: 3 days
MFC note: record-only to wipe from the merge tracker
Sponsored by: The FreeBSD Foundation
svnversion metadata to the runtime and kernel packages.
Instead of traversing src/sys, as is done by newvers.sh for uname(1),
a full tree walk is done to prevent userland and/or modifications
from not being reflected in a modified tree (M).
MFC after: 5 days
Sponsored by: The FreeBSD Foundation
The idea here is that, provided upstream pkg(8) maintainers accept
the proposed change, the kernel.ucl will contain a post-install
script causing pkg(8) to emit a message informing to reboot the
system after the kernel is upgraded using 'pkg upgrade', so the
new userland is installed on the running new kernel. At present,
this functionality does not exist in pkg(8), but will help ensure
the upgrade path follows that from UPDATING. To work around this
for now, evaluate ASSUME_ALWAYS_YES, and prompt the user if they
wish to proceed if not set to true.
Since there is a kernel dependency, and a non-GENERIC kernel may
be in use, update Makefile.inc1 to replace '%KERNCONF%' in the
runtime.ucl with the first-built kernel set either via command line
or in make.conf(5).
MFC after: 5 days
Sponsored by: The FreeBSD Foundation
arches), and handle two more cases where libc++ includes could be
incorrectly enabled, in case the host compiler is clang 5.0.0, and the
target (cross) compiler is gcc 4.2.1.
Noted by: bdrewery
MFC after: 3 days
X-MFC-With: 321684
- bsd.compiler.mk: Must ensure that the CCACHE_WRAPPER_PATH comes first
in PATH.
- Makefile.inc1: Must prepend the CCACHE_WRAPPER_PATH into BPATH as it
overrides the PATH set in bsd.compiler.mk in sub-makes. The PATH
set in bsd.compiler.mk is not exported and doing so would cause it to
then override the BPATH set from environment. The only sane solution
is to prepend into BPATH as needed.
CCACHE_PATH could possibly be used for some of this as well.
Sponsored by: Dell EMC Isilon
Since we imported clang 5.0.0, the version check in Makefile.inc1 which
checks whether to use libc++ fires even when the compiler for the target
architecture is gcc 4.2.1. This is because only X_COMPILER_VERSION is
checked. Also check X_COMPILER_TYPE, so it will only use libc++ when an
external gcc toolchain is used.
Reviewed by: emaste, rpokala
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D11776
It is full of distracting noise about UPDATING and may confuse
the user about what is actually being deleted. It is also
possible to have directories removed on every run with
use of WITHOUT_ knobs that the mtree files do not
account for and for which the directories are incorrectly
in OLD_DIRS currently.
X-MFC-With: r321443
MFC after: 1 month
Sponsored by: Dell EMC Isilon
This dependency does nothing since cddl/lib/libzfs is never
added into the 'make libraries' dependency chain
directly.
MFC after: 2 weeks
Sponsored by: Dell EMC Isilon
This prevents situations with -DNO_CLEAN from finding stale headers or
libraries in places that no longer exist or have moved. It avoids
the need to remove all of WORLDTMP by reusing what we already know
is obsolete.
MFC after: 1 month
Sponsored by: Dell EMC Isilon
The files are only ever generated to .OBJDIR, not to WORLDTMP (as a
sysroot) and are only ever included from a compilation. So using
a beforebuild target here removes the file before the compilation
tries to include it.
MFC after: 2 months
X-MFC-With: r321369
clang 4.0.0. Otherwise, these can get included before the two newly
generated ones (which are different) for clang 5.0.0.
Reported by: Mark Millard
MFC after: 2 months
X-MFC-With: r321369
In a scenario of cross-building it is possible that an OBJDIR's WORLDTMP
contains an older compiler in WORLDTMP/usr/bin/cc that is not rebuilt
if SYSTEM_COMPILER logic is triggered. This compiler was still
incorrectly used. Address this by removing WORLDTMP/usr/bin/cc and all
of the hardlinked files associated with it. Also do this for c++ for
GCC builds.
Sponsored by: Dell EMC Isilon
MFC after: 1 week
Our current approach to dependency tracking cannot cope with switching
generated asm syscall stubs into C wrappers. Perpetuate the hack in
Makefile.inc1 to paper over the problem until we can take a holistic
approach to fixing dependency problems.
Differential Revision: https://reviews.freebsd.org/D11344
The -B was originally added in projects/release-pkg r289381 as a copy
of what 'make world' did at the time. The -B was removed from
the 'installworld' call in 'world' in r303844 though. The staging
of files is safe to run in parallel.
MFC after: 2 weeks
Sponsored by: Dell EMC Isilon
This fixes LD errors during 'make packages' but also for the unlikely case of
'buildworld' on 1 system and 'packages' on another [1].
PR: 212877 [1]
MFC after: 2 weeks
Sponsored by: Dell EMC Isilon
This is useful for having directories behave differently depending
on the phase - such as enabling SUBDIR_PARALLEL or disabling
redundant building of library directories already done by
earlier 'make _libraries'.
Sponsored by: Dell EMC Isilon
This is to allow downstream Makefiles to know for sure they are building
against a sysroot rather than only depending on ${DESTDIR} or other
assumptions.
This also exports it into buildenv.
MFC after: 2 weeks
Sponsored by: Dell EMC Isilon
Use a similar approach to r318957 (which was for ptrace dependencies):
grep the .depend file for the old source file and delete the stale
dependency if found.
Reviewed by: bdrewery
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D11091
When makeman is generating src.conf(5) it tries to test all variation of options
including WITH_DIRDEPS_BUILD. it results in an error when filemon(4) is not
loaded.
Export variables that are needed to prevent this behaviour.
Helped by: sjg
Normally META_MODE ignores host files for "meta mode" decisions on whether a
file should be rebuilt or not. This is because a simple installworld can
update timestamps and cause the next build to rebuild all host tools, when the
previous ones may not have any changes in the source tree. These tools are
normally still ABI compatible. They are only rebuilt if NO_META_IGNORE_HOST is
set from the workaround/hack in r301467.
One of the major problems with this is when a host tool has objects spread
across many revisions that have mixed-ABI. For example, if struct stat were to
change on the host, some objects for a tool may have different ideas of that
struct's definition. If just 1 source file were modified and rebuilt and
linked into the tool, then that toll will have mixed-ABI objects and crash.
This exact thing happened with the ino64 commit in r301467 followed by a
trivial update to libbfd in r318750. The resulting binary would crash in
buildworld.
Sponsored by: Dell EMC Isilon
This will ensure that aarch64 gets a working native /usr/bin/ld rather
than requiring the aarch64-binutils hack in Poudriere, or emulating
the aarch64 lld.
PR: 217189
Reported by: swills, jbeich
FreeBSD does not guarantee kernel forward compatibility (that is,
running a newer userland on an older kernel). The documented upgrade
procedure specifies that installkernel should be performed, followed by
a reboot and then installworld. As a sanity check when installing onto
the running system (DESTDIR is / or unset), attempt to run "sh echo OK"
using rescue from the objdir. If rescue fails (e.g., because the system
has not been rebooted and the "old" kernel lacks a system call required
by the to-be-installed world), abort the installation.
This should avoid ino64 foot-shooting when the proper upgrade procedure
is not followed.
Reviewed by: allanjude, gjb, kib
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D10987
This is similar to r318912, except that ptrace.[sS] was previously a
file in the source tree, not a generated assembly wrapper.
Check for the existence of ptrace.[sS] in the .depend file to determine
if we have to clean it up. This is a bit hackish and will not be left
in place indefinitely, but provides a useful example case when
investigating a better solution in bmake.
Reviewed by: bdrewery
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D10930
This is an attempt to help -DNO_CLEAN builds after r302092 (which
removed the pipe libc syscall wrapper) and r318736 (which removed
getdents, lstat, mknod, and stat).
Dependencies cannot cope with certain source tree changes,
particularly with respect to removing source files and replacing
generated files. Handle these cases from _worldtmp in an ad-hoc
fashion.
Reviewed by: bdrewery, cem
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D10876
This allows for building the world against the already-created
host/sysroot environment. It is not overly useful outside of cases of
large-impact changes such as a testing a new compiler. It will
allow quickly getting back to an error in the target-phases of the
build where a new compiler is being used.
Sponsored by: Dell EMC Isilon
The closing paren for the list of architectures that should enable LIB32
by default was in the wrong place resulting in LIB32 always be enabled on
mips64 regardless of WITH_LIB32/WITHOUT_LIB32.
Submitted by: Alex Richardson <Alexander.Richardson@cl.cam.ac.uk>
Obtained from: CheriBSD
Sponsored by: DARPA / AFRL
Right after cross-tools, a compiler-metadata.mk file is created that
stores all of the bsd.compiler.mk metadata. It is then read in
with a fail-safe during installworld time.
The file is explicitly removed when invoking cross-tools to ensure that
a stale file is not left around from odd manual 'make _cross-tools' ->
'make installworld' invocations.
This fixes several issues:
- With WITH_SYSTEM_COMPILER (default yes on head and no on releng/11.0):
If you build on a system where the bootstrap compiler does not
build due to the host compiler matching the in-tree one, but then
installworld on another system where that logic fails (a
bootstrap compiler is needed), the installworld immediately fails
with:
sh: cc: not found
Note that fixing this logic may then hit a case where a rebuild is
attempted in installworld. Normally cc would be ran with
'CFLAGS+=ERROR-tried-to-rebuild-during-make-install' to cause an
error such as:
cc: error: no such file or directory: 'ERROR-tried-to-rebuild-during-make-install'
However, now it will just fail with the 'cc: not found' error.
Inspection of the compile line will show
'ERROR-tried-to-rebuild-during-make-install'; It's not useful to
set CC to anything other than 'cc' during install as it is more
helpful to see the attempted compile rather than some other bogus
error.
- This now avoids running bsd.compiler.mk (cc executions) even more
during installworld. There are compiler-dependent SUBDIR in the
tree which required having a compiler during install.
There is at least 1 case where CC is still executed in the install,
such as from a LOOKUP!= in secure/lib/libcrypto/Makefile.inc checking
for 'vzeroall' support. This is not significant for installworld
as the lookup has a fallback (and hides its error) and only modifies CFLAGS,
thus it's not worth fixing.
PR: 212877
MFC after: 2 weeks
Sponsored by: Dell EMC Isilon
Add WITH_LLD_BOOTSTRAP and WITHOUT_LLD_BOOTSTRAP knobs, similar to the
Clang bootstrap knobs.
Reviewed by: dim
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D10249
r279908 added logic to Makefile.inc1 to automatically set
CROSS_BINUTILS_PREFIX for architectures not supported by the in-tree
binutils: arm64 when first introduced, and later riscv64 as well.
LLVM's LLD linker is now included in the base system, and is enabled by
default for arm64 and capable of linking world and kernel. Thus, avoid
automatically setting CROSS_BINUTILS_PREFIX and requiring the binutils
port if WITH_LLD_IS_LD is true.
Reviewed by: kan
Relnotes: Yes
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D10310
In a cross-build, the build-tools are native host binaries. We do not
want to rebuild them when building for the target. Bmake previously
did not support checking .NOMETA on an existing target, so .NOMETA_CMP
was used here. However, .NOMETA_CMP still triggers meta mode conditions
if the number of commands or the command changes. In r312467 the paths
to build ncurses files were modified and thus triggered meta mode to
rebuild the build tools (make_keys, make_hash) in ncurses during the
target build. Bmake 20160604 committed in r301462 changed .NOMETA to
also skip meta mode logic for an existing .meta file as well, thus it
is now the proper fix here.
I explored moving the build-tools output to WORLDTMP/tools with
relatively good success, but have concerns that doing so would be
problematic for downstream vendors who use LOCAL_TOOL_DIRS and
expect the tools to be in current OBJDIR for the target. It also
adds more complexity into finding the tools during target build
and handling of where they are for rescue/rescue and
mkcsmapper_static/mkesdb_static which should really not be connected in
build-tools anyway.
MFC after: 2 weeks
Reported by: many
Sponsored by: Dell EMC Isilon
