Commit Graph

3126 Commits

Author SHA1 Message Date
Dimitry Andric
cbafd2630b Add support for selectively enabling LLVM targets
This makes it possible, through src.conf(5) settings, to select which
LLVM targets you want to build during buildworld.  The current list is:

* (WITH|WITHOUT)_LLVM_TARGET_AARCH64
* (WITH|WITHOUT)_LLVM_TARGET_ARM
* (WITH|WITHOUT)_LLVM_TARGET_MIPS
* (WITH|WITHOUT)_LLVM_TARGET_POWERPC
* (WITH|WITHOUT)_LLVM_TARGET_SPARC
* (WITH|WITHOUT)_LLVM_TARGET_X86

To not influence anything right now, all of these are on by default, in
situations where clang is enabled.

Selectively turning a few targets off manually should work.  Turning on
only one target should work too, even if that target does not correspond
to the build architecture.  (In that case, LLVM_NATIVE_ARCH will not be
defined, and you can only use the resulting clang executable for
cross-compiling.)

I performed a few measurements on one of the FreeBSD.org reference
machines, building clang from scratch, with all targets enabled, and
with only the x86 target enabled.  The latter was ~12% faster in real
time (on a 32-core box), and ~14% faster in user time.  For a full
buildworld the difference will probably be less pronounced, though.

Reviewed by:	bdrewery
MFC after:	1 week
Differential Revision: https://reviews.freebsd.org/D11077
2018-06-22 15:00:00 +00:00
Bryan Drewery
6e6be2bbe2 Reduce exec and fstat overhead for non-build targets.
This is mostly targetting 'make showconfig' and 'make test-system-*' for
the benefit of makeman and universe-one-clang work.

Sponsored by:	Dell EMC
2018-06-20 17:20:39 +00:00
Bryan Drewery
a7d84af890 Add WITH_SYSTEM_LINKER, on by default, that avoids building lld when possible.
This works similar to WITH_SYSTEM_COMPILER added in r300354.  It only
supports lld via WITH_LLD_BOOTSTRAP.

When both SYSTEM_COMPILER and SYSTEM_LINKER logic passes then libclang
will not build in cross-tools.  If either check fails though then
libclang is built.

The .info is reworked to notify when libclang will be built since if
either clang or lld needs to be rebuilt, but not the other, the
notification can lead to confusion on why "clang is building".

-fuse-ld= is not used with this method so some combinations of compiler
and linker are expected to fail.

A new 'make test-system-linker' target is added to see the logic results.

Makefile.inc1:
  CROSS_BINUTILS_PREFIX support had to be moved higher up so that XLD
  could be set and MK_LLD_BOOTSTRAP disabled before checking SYSTEM_LINKER
  logic as done with SYSTEM_COMPILER.  This also required moving where
  bsd.linker.mk was read since XLD needs to be set before parsing it.  This
  creates a situation where src.opts.mk can not test LINKER_FEATURES or
  add LLD_BOOTSTAP to BROKEN_OPTIONS.

Reviewed by:	emaste (earlier version)
Sponsored by:	Dell EMC
Differential Revision:	https://reviews.freebsd.org/D15894
2018-06-20 16:10:14 +00:00
Ruslan Bukin
2f7f197723 Don't use TARGET_ARCH in this file, use MACHINE_ARCH instead.
TARGET_ARCH is not defined when building ports/packages.

Sponsored by:	DARPA, AFRL
2018-06-19 13:28:02 +00:00
Konstantin Belousov
f39bffc62c Rework ofed build.
Aligns the build with the FreeBSD traditional approach to not build in
contrib/, and to track inter-dependencies between libraries.

With help from:	bdrewery
Reviewed by:	bdrewery, hselasky
Sponsored by:	Mellanox Technologies
MFC after:	2 weeks
Differential revision:	https://reviews.freebsd.org/D15648
2018-06-16 15:05:05 +00:00
Warner Losh
f0f8a1af01 NVME support is only for x86 and powerpc64.
Implement MK_NVME now that the expression for where NVMe is
complicated. Default it to "yes" for x86 and powerpc64 and
no everywhere else. Use it in camcontrol to define WITH_NVME
for those platforms where we support nvme.

This should fix the newly introduced nvme files to camcontrol
which were building everywhere.

Pointy Hat To: imp
Sponsored by: Netflix
2018-06-14 01:15:19 +00:00
Bryan Drewery
7ccdc06829 Use simpler sed invocation.
X-MFC-With:	r334791
MFC after:	1 week
Submitted by:	danfe
2018-06-07 19:39:11 +00:00
Bryan Drewery
1930b98d49 Stop using head(1) which is not available in installworld.
installworld should not be executing this anyhow but there is some
obscure case doing it still.  The head(1) binary is not part of
ITOOLS and there's no need to add it.

MFC after:	1 week
Sponsored by:	Dell EMC
2018-06-07 16:16:22 +00:00
Ruslan Bukin
a8db1fa14d Sort GCC warning flags and remove duplicates.
Sponsored by:	DARPA, AFRL
2018-06-04 16:21:18 +00:00
Brad Davis
123ec1b8b9 Move all of the directory path into the DIR part of the component and make the
NAME be only the filename.

This makes it possible to use the DIR as part of the dep in the future.

Approved by:	bapt (mentor)
2018-05-31 13:26:12 +00:00
Ed Maste
1bf8f95706 Enable lld as the system linker by default on amd64
The migration to LLVM's lld linker has been in progress for quite some
time - about three years ago I opened an upstream LLVM meta-bug to track
issues using lld as FreeBSD's linker, and about 1.5 years ago requested
the first exp-run with lld as the system linker.

As of r327783 we enabled LLD_BOOTSTRAP by default on amd64, using lld as
the linker to link the kernel and world, but GNU ld was still installed
as /usr/bin/ld.

The vast majority of issues observed when building ports with lld as the
system linker have now been solved, so set LLD_IS_LD by default on amd64
and install lld as /usr/bin/ld.  A small number of port failures remain
and these will be addressed in the near future.

Thanks to antoine@ for handling the exp-runs, krion@ for investigating
many port failures and adding LLD_UNSAFE or other fixes or workarounds,
and everyone who helped investigate, fix or tag ports.

PR:		214864 (exp-run)
Sponsored by:	The FreeBSD Foundation
2018-05-30 18:04:25 +00:00
Hans Petter Selasky
b00ab7548b MFV r333789: libpcap 1.9.0 (pre-release)
MFC after:	1 month
Sponsored by:	Mellanox Technologies
2018-05-28 08:12:18 +00:00
Bryan Drewery
bbec13875a rescue: Restore 'make depend' call to fix WITH_META_MODE after r334008.
The rescue/crunchgen build avoids linking binaries for the objects it is
building by doing 'make foo.o bar.o' rather than 'make all'.  This breaks the
implicit 'beforebuild: depend' dependency which ensured that all source files
were generated and up-to-date before building the object files.  This
manifested as a WITH_META_MODE build problem for bin/sh in the rescue build
with syntax.{c,h} not properly being regenerated or recognized as changed in
the dependency graph.

Sponsored by:	Dell EMC
MFC after:	1 week
Reported by:	many
2018-05-24 18:49:19 +00:00
Ed Maste
12db404753 Revert r334081 (-Wmain) as it broke the build on gcc architectures 2018-05-23 21:26:33 +00:00
Eitan Adler
9a6efe4a0f bsd.sys.mk: add -Wmain to WARNS=1
Even in very low "WARNS" conditions, 'main' ought to be reasonable.
2018-05-23 08:35:55 +00:00
Eitan Adler
5d1ee539dd bsd.sys.mk: add links and update a bit
- add links to more modern resources
- remove 'k&r' which is unused in FreeBSD
- remove stray comment
2018-05-23 07:54:58 +00:00
Ed Maste
1069f41bb8 Force WITHOUT_FREEBSD_UPDATE when WITHOUT_PORTSNAP is set
freebsd-update depends on phttpget from portsnap. We could move phttpget
out of portsnap and build it as long as WITHOUT_FREEBSD_UPDATE and
WITHOUT_PORTSNAP are not both set, but for now just make the dependency
explicit.

PR:		228220
Reported by:	Dries Michiels
Sponsored by:	The FreeBSD Foundation
2018-05-16 01:33:48 +00:00
Bryan Drewery
b3a8db007a Fix stale dependencies on libcasper libraries.
Libcasper and its modules have no static libraries so don't define
paths to them.  This fixes LIBADD automatically adding DPADD
entries for casper.

Reported by:	sbruno
Sponsored by:	Dell EMC Isilon
2018-05-15 23:46:49 +00:00
Ed Maste
ff8f1e8332 Error out on attempt to link amd64 kernel with old binutils linker
As of r333461 we require ifunc support to link a working amd64 kernel.
The default in-tree bootstrap linker is lld and it has the required
support, as does any modern out-of-tree binutils linker.  The in-tree
GNU ld is from binutils 2.17.50 and it does not have ifunc support,
so produce an error rather than a broken kernel.

Reviewed by:	kib
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D15378
2018-05-10 20:10:02 +00:00
John Baldwin
806b3bbe97 Recognize the base/gcc compiler as GCC.
The existing patterns for 'cc --version' output do not work for GCC
built from the base/gcc port.

Reviewed by:	imp
Differential Revision:	https://reviews.freebsd.org/D15357
2018-05-09 15:20:39 +00:00
Brad Davis
c74ace92a4 Enable directory creation with FILESDIR.
This is part of packaging base work.

Reviewed by:	will
Approved by:	bapt (mentor), allanjude (mentor)
Differential Revision:	https://reviews.freebsd.org/D15130
2018-05-09 13:44:54 +00:00
Konstantin Belousov
db08bfce2f Created static libc PIC/no-SSP library to be used by rtld.
Rtld is not compatible with SSP, and since we link libc_pic.a to rtld
to have the basic support like memory and string copy functions, we
have to both carefully limit libc use, and to provide the ssp support
shims.  This change makes the libc use in rtld more straighforward but
still limited, and allows to remove the shims, to be done in the next
commit.

Submitted by:	Luis Pires
Reviewed by:	bdrewery, brooks
Differential revision:	https://reviews.freebsd.org/D15283
2018-05-09 10:28:24 +00:00
Sean Bruno
608cdf904b Remove accidentally committed debug statement that induced the following
warning:

make[3]: "/usr/src/share/mk/bsd.prog.mk" line 274: warning: duplicate
script for target "_scriptsinstall" ignored
make[3]: "/usr/src/share/mk/bsd.prog.mk" line 274: warning: using
previous script for "_scriptsinstall" defined here

Reviewed by:	kevans
2018-05-05 18:39:54 +00:00
Kyle Evans
a2584d1b34 bsdgrep: annihilate our in-tree TRE, previously disabled by default
It was an old TRE that had plenty of bugs and no performance gain over
regex(3). I disabled it by default in r323615, and there was some confusion
about what the knob does- likely due to poor naming on my part- to the tune
of "well, it sounds like it should speed things up" (mentioned by multiple
people).

To compound this, I have no intention of maintaining a second regex
implementation. If someone would like to step up and volunteer to maintain a
lean-and-mean implementation for grep, this is OK, but we have very few
volunteers to maintain even our primary regex implementation.
2018-05-04 03:13:25 +00:00
Ed Maste
2dce6bfbb2 Build lld as long as we have a C++11 host compiler
As with Clang, build our toolchain components by default when the host
compiler is capable of doing so, to make them available for testing and
experimentation.

Sponsored by:	The FreeBSD Foundation
2018-05-02 23:43:33 +00:00
Ed Maste
b25a46f7f7 Add support for linker-type-specific flags
r332090 added a LINKER_TYPE test to add the --no-rosegment flag when
linking the i386 loader components with lld.  Instead, introduce a
general mechanism for setting LDFLAGS for a specific linker type,
and use it for --no-rosegment.

Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D14998
2018-04-19 20:58:09 +00:00
Ruslan Bukin
3783c1b196 Set correct float abi (float abi double) for hard-float build, so
__riscv_float_abi_double macro will be defined by compiler.

The options are:
o lp64  __riscv_float_abi_soft
o lp64f __riscv_float_abi_single
o lp64d __riscv_float_abi_double

Sponsored by:	DARPA, AFRL
2018-04-18 13:58:42 +00:00
Bryan Drewery
4f6de62991 Use known SRCTOP if possible to determine SYSDIR.
Suggested by:	sjg
Sponsored by:	Dell EMC
2018-04-12 20:48:17 +00:00
Bryan Drewery
711c77093c Fix using wrong SYSDIR after r331683.
This was inadvertently overriding the first found SYSDIR with the last
of /usr/src which could result in the wrong headers being used if not
building from /usr/src.

SYSDIR?= is not used here to avoid evaluating the exists() when unneeded.

Reported by:	rgrimes, sjg, Mark Millard
Pointyhat to:	bdrewery
Sponsored by:	Dell EMC
2018-04-12 20:27:57 +00:00
Kyle Evans
54095bf428 Re-sort LOADER options
These have become unsorted from everything else. This is desync'd from
stable/11 due to some hand-merging that was done there, so the MFC of this
will look slightly different.

MFC after:	3 days
2018-04-06 19:49:57 +00:00
Ruslan Bukin
cde6fa2857 Add new shared library -- libopencsd.
OpenCSD is an ARM CoreSight(tm) trace packets decoder.

- Connect libopencsd to the arm64 build.
- Install opencsd headers to /usr/include/opencsd/

Sponsored by:	DARPA, AFRL
2018-04-04 14:31:56 +00:00
Jung-uk Kim
9ef029b045 Catch up with Clang 6.0.
MFC after:	3 days
2018-04-03 23:11:08 +00:00
Bryan Drewery
7d9f50a4ba Avoid looping if SYSDIR already known.
Sponsored by:	Dell EMC
2018-03-28 16:28:24 +00:00
Bryan Drewery
ffae081b1e Avoid upwards directory walk in kernel build for finding known SYSDIR.
Sponsored by:	Dell EMC
2018-03-28 16:28:20 +00:00
Emmanuel Vadot
c2f5940db3 Add dtb overlays support
DTB Overlays are useful to change/add nodes to a dtb without the need to
modify it.
Add support for building dtbo during buildkernel.
The goal of DTBO present in the FreeBSD source tree is to fill a gap in
time when we submit changes upstream (Linux). Instead of waiting 2 to 4 months
we can add a DTBO in tree in the meantime.
This is not for adding DTBO for capes/hat/addon boards, those will be
better to put in a ports.
This is also not for enabling a i2c/spi/pwm controller on certain pins,
each user have a different use case for those (which pins to use etc ...)
and we cannot have all possible configuration.

Add a dtbo for sun8i-h3-sid which add the SID node missing in upstream dts.

Reviewed by:	kevans
Differential Revision:	https://reviews.freebsd.org/D14782
2018-03-24 21:30:24 +00:00
Ruslan Bukin
30b3274fff Add new shared library -- libipt.
libipt is the Intel Processor Trace (Intel PT) packets decoder.

- Include libipt to amd64 build.
- Install libipt headers to /usr/include/libipt/

Sponsored by:	DARPA, AFRL
2018-03-21 14:37:04 +00:00
John Baldwin
1e18e7d203 Add support for MIPS to LLVM's libunwind.
This is originally based on a patch from David Chisnall for soft-float
N64 but has since been updated to support O32, N32, and hard-float ABIs.
The soft-float O32, N32, and N64 support has been committed upstream.
The hard-float changes are still in review upstream.

Enable LLVM_LIBUNWIND on mips when building with a suitable (C+11-capable)
toolchain.  This has been tested with external GCC for all ABIs and
O32 and N64 with clang.

Reviewed by:	emaste
Obtained from:	CheriBSD (original N64 patch)
Sponsored by:	DARPA / AFRL
Differential Revision:	https://reviews.freebsd.org/D14701
2018-03-20 15:44:17 +00:00
Hans Petter Selasky
e808190a59 Add kernel and userspace code to dump the firmware state of supported
ConnectX-4/5 devices in mlx5core.

The dump is obtained by reading a predefined register map from the
non-destructive crspace, accessible by the vendor-specific PCIe
capability (VSC). The dump is stored in preallocated kernel memory and
managed by the mlx5tool(8), which communicates with the driver using a
character device node.

The utility allows to store the dump in format
    <address> <value>
into a file, to reset the dump content, and to manually initiate the
dump.

A call to mlx5_fwdump() should be added at the places where a dump
must be fetched automatically. The most likely place is right before a
firmware reset request.

Submitted by:	kib@
MFC after:	1 week
Sponsored by:	Mellanox Technologies
2018-03-08 15:21:56 +00:00
Ed Maste
89257265fb Disable LLD_BOOTSTRAP under WITHOUT_CROSS_COMPILER
LLD is a cross toolchain component. It shouldn't be built when
requesting a build without building a cross compiler.

(CROSS_COMPILER is somewhat unfortunately named; in any case, lld
should be treated as GNU binutils here.)

Submitted by:	Dan McGregor <dan.mcgregor at usask.ca>
MFC after:	1 week
2018-03-07 17:33:41 +00:00
Bryan Drewery
743ab0187f Ignore MAKEOBJDIRPREFIX from src-env.conf in sub-makes.
A sub-make, such as in 'make buildworld', may want to override MAKEOBJDIRPREFIX
but is unable to do so if src-env.conf is forcing it to another value.  Without
using '?=' the sub-make may use the wrong .OBJDIR.

Reported by:	eadler
Sponsored by:	Dell EMC
2018-03-03 23:23:01 +00:00
Warner Losh
f86dd99686 Create LOADER_UBOOT, and LOADER_OFW. Move these options out of
Makefile.${MACHINE_ARCH} and remove the now-empty files. Generate the
*32 directories on the necessary architectures (well, currently only
amd64) on the fly. Remove LOADER_EFI variable and co-locate it with
EFI.

Differential Review: https://reviews.freebsd.org/D14546
2018-03-01 19:50:55 +00:00
Bryan Drewery
a12a1b473b Allow overriding .MAKE.MAKEFILE_PREFERENCE.
This will be utilized by Ports tools to avoid some extra
stat(2) calls.

MFC after:	3 days
2018-02-28 20:06:03 +00:00
Warner Losh
c1593b9ef2 Doc __DEFAULT_YES_OPTIONS, __DEFAULT_NO_OPTIONS and BROKEN_OPTIONS
in one paragraph to highlight the relationship between them.

Suggested by: kevans@
2018-02-27 17:36:01 +00:00
Warner Losh
9e74797f30 Move EFI up to common makefile. There's no need for all these .if's
based on architecture.

Sponsored by: Netflix
2018-02-27 17:35:29 +00:00
Alan Somers
b8f7774edf tests: require ksh93 in the Kyuafiles for all atf-ksh93 test programs 2018-02-24 15:13:20 +00:00
Bryan Drewery
fc287c10fb Fix installation with read-only OBJDIR.
Reported by:	npn
Sponsored by:	Dell EMC
2018-02-24 01:33:17 +00:00
Alan Somers
2fae26bd8b Add the ZFS test suite
It was originally written by Sun as part of the STF (Solaris test framework).
They open sourced it in OpenSolaris, then HighCloud partially ported it to
FreeBSD, and Spectra Logic finished the port.  We also added many testcases,
fixed many broken ones, and converted them all to the ATF framework.  We've had
help along the way from avg, araujo, smh, and brd.

By default most of the tests are disabled.  Set the disks Kyua variable to
enable them.

Submitted by:	asomers, will, justing, ken, brd, avg, araujo, smh
Sponsored by:	Spectra Logic Corp, HighCloud
2018-02-23 16:31:00 +00:00
Bryan Drewery
49a1108ae7 Move SVNVERSION_CMD into the one place that uses it.
This code, which is basically `svnversion || svnliteversion`, generates
2 fstatat(2) for every directory in PATH for every Makefile parsed that
includes bsd.own.mk.  This can add up for things like generating a Ports
index (Poudriere) or building a dependency graph for base.

Sponsored by:	Dell EMC
MFC after:	2 weeks
2018-02-20 21:48:16 +00:00
Warner Losh
fcdb1f0317 Eliminate bsd.stand.mk and -fPIC 32-bit intel builds
OK. We don't really need a bsd.stand.mk, and it was causing a -fPIC
for the toolchain to be added (bogusly) when building on amd64. Pull
all relevant defs back into defs.mk and delete bsd.stand.mk.

This saves about 15-20k on i386 loader and zfsloader which when
combined with Lua give us a lot more stack space in those constrained
environments.
2018-02-16 00:17:32 +00:00
Warner Losh
7cafeaa1fd Add Lua as a scripting langauge to /boot/loader
liblua glues the lua run time into the boot loader. It implements all
the runtime routines that lua expects. In addition, it has a few
standard 'C' headers that nueter various aspects of the LUA build that
are too specific to lua to be in libsa. Many refinements from the
original code to improve implementation and the number of included lua
libraries. Use int64_t for lua_Number. Have "/boot/lua" be the default
module path. Numerous cleanups from the original GSoC project,
including hacking libsa to allow lua to be built with only one change
outside luaconf.h.

Add the final bit of lua glue to bring in liblua and plug into the
multiple interpreter framework, previously committed.

Add LOADER_LUA option, currently off by default.

Presently, this is an experimental option. One must opt-in to using
this by defining WITH_LOADER_LUA and WITHOUT_FORTH. It's been
lightly tested, so keep a backup copy of your old loader handy.
The menu code, coming in the next commit, hasn't been exhaustively
tested. A LUA boot loader is 60k larger than a FORTH one, which is
80k larger than a no-interpreter one. Subtle changes in size
may tip things past some subtle limit (the binary is ~430k now
when built with LUA). A future version may offer coexistance.

Bump FreeBSD version to 1200058 to mark the milestone.

Pedro Souza's 2014 Summer of Code project. Rui Paulo, Pedro Arthur,
Zakary Nafziger and Wojciech A. Koszek also contributed. Warner Losh
reworked it extensively into its current form.

Obtained from: https://wiki.freebsd.org/SummerOfCode2014/LuaLoader
Sponsored by: Google Summer of Code
Relnotes: Yes
MFC After: 1 month
Differential Review: https://reviews.freebsd.org/D14295
2018-02-12 15:31:53 +00:00