2845 Commits

Author SHA1 Message Date
Bryan Drewery
cd19ecdbdc Similar to r296013 for NO_ROOT, force SUBDIR_PARALLEL for buildworld WORLDTMP staging.
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-06-21 18:56:53 +00:00
Bryan Drewery
85ec50508a objwarn should be .PHONY.
Otherwise in META_MODE it may create an objwarn.meta if only bsd.obj.mk
is included;  bsd.sys.mk already had .PHONY: objwarn.

MFC after:	3 days
Sponsored by:	Dell EMC Isilon
2017-06-21 17:11:49 +00:00
Bryan Drewery
05f3e1fe94 Fix 'make clean all' to work again.
This likely broke completely with r308599.

Apply the same fix for 'make destroy' which is a DIRDEPS_BUILD thing.

PR:		219819
Reported by:	trasz
MFC after:	3 days
Sponsored by:	Dell EMC Isilon
2017-06-20 20:52:06 +00:00
Bryan Drewery
fb40136990 LIBADD: Try to support partial tree checkouts in some limited cases.
LIBADD is only supported for in-tree builds because we do not install
share/mk/src.libnames.mk (which provides LIBADD support) into /usr/share/mk.
So if a partial checkout is done then the LIBADDs are ignored and no LDADD is
ever added.

Provide limited support for this case for when LIBADD is composed entirely of
base libraries.  This is to avoid clashes with ports and other out-of-tree
LIBADD uses that should not be mapped to LDADD and because we do not want to
support LIBADD out-of-tree right now.

Reported by:	mckusick, kib
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-06-20 20:34:30 +00:00
Ed Maste
a793c109ae Fall back to GPL dtc(1) when we lack a C++11 compiler
The BSD licensed device tree compiler is written in C++11.

Reported by:	jhibbits
Reviewed by:	jhibbits, manu
Sponsored by:	The FreeBSD Foundation
2017-06-20 15:51:09 +00:00
Emmanuel Vadot
011717d344 Switch back to the BSDL DTC (Device Tree Compiler).
The BSDL dtc has grown the needed features (overlays mostly) and is able to
compile all of our base DTS.
You can use WITH_GPL_DTC is you need the GPL one or DTC= in make.conf(5)
to specify an alternate location for the compiler to use.

Discussed with: emaste, imp
2017-06-20 02:09:50 +00:00
Bryan Drewery
c99b67a794 Utilize SYSROOT from r320119 in places where DESTDIR may be wanting WORLDTMP.
Since buildenv exports SYSROOT all of these uses will now look in
WORLDTMP by default.

sys/boot/efi/loader/Makefile
        A LIBSTAND hack is no longer required for buildenv.

MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-06-19 20:47:24 +00:00
Bryan Drewery
ad706c65e1 Follow-up r320061: Need to respect make.conf/env LIBDIR overrides.
This fixes the lib32 build from creating all stale .depend files.

X-MFC-With:	320061
MFC after:	1 week
Sponsored by:	Dell EMC Isilon
2017-06-19 18:08:02 +00:00
Bryan Drewery
d0ab0ec2fa Fix Makefiles which override LIBDIR to not add incorrect dependencies into .depend.
This fixes these cases which would rebuild every time:
    make[6]: /usr/obj/usr/src/libexec/rtld-elf/tests/libpythagoras/.depend, 1: ignoring stale .depend for /usr/obj/usr/src/tmp/usr/tests/libexec/rtld-elf/libm.a
    make[6]: /usr/obj/usr/src/lib/libxo/tests/encoder/.depend, 1: ignoring stale .depend for /usr/obj/usr/src/tmp/usr/tests/lib/libxo/libxo.a
    make[7]: /usr/obj/usr/src/lib/libthr/tests/dlopen/dso/.depend, 1: ignoring stale .depend for /usr/obj/usr/src/tmp/usr/tests/lib/libthr/dlopen/libpthread.a

The problem is that some Makefiles will override LIBDIR to where they want
their library to install.  bsd.libnames.mk will then use ${LIBDIR} to define
where *existing* libraries are.  This then leads to looking for the
libraries in the *target* place rather than the *expected* place.

We may want to expand this (and all of the other *DIR variables in bsd.own.mk)
into something like what Ports has, a PREFIX and a LOCALBASE.  PREFIX being
where things are being installed to and LOCALBASE being where they already are.

For now store the default expected LIBDIR into LIBDIR_BASE and use that for
library locations.

Reported by:	sbruno
MFC after:	1 week
Sponsored by:	Dell EMC Isilon
2017-06-17 20:33:11 +00:00
Bryan Drewery
7970b89399 WITH_META_MODE: End each ERROR_CMD CMD line with ';'.
This makes it easier to debug multi-line command failures.

X-MFC-With:	r319862
MFC after:	1 week
Sponsored by:	Dell EMC Isilon
2017-06-16 21:48:42 +00:00
Bryan Drewery
2bc743eea1 Fix more incorrect library directories fix 'stale .depend' rebuilds.
Reported by:	sbruno
MFC after:	3 days
Sponsored by:	Dell EMC Isilon
2017-06-16 21:37:05 +00:00
Bryan Drewery
ddbc759f99 Fix LIBAMU location to fix 'stale .depend' rebuilds in usr.sbin/amd.
This originally came in r275052.

Reported by:	sbruno
MFC after:	3 days
Sponsored by:	Dell EMC Isilon
2017-06-16 20:47:12 +00:00
Bryan Drewery
51deaaab90 WITH_META_MODE: Don't try showing command if .ERROR_META_FILE is empty.
This was sed'ing on stdin for failing .PHONY targets.

Reported by:	Mark Millard
X-MFC-With:	r319862
MFC after:	1 week
Sponsored by:	Dell EMC Isilon
2017-06-16 04:28:10 +00:00
Bryan Drewery
95a2495fc2 META_MODE: Show .ERROR_CMD in error.
This uses a hack to get the CMD from the meta file rather than
.ERROR_CMD since bmake currently blanks the value for non-jobs
mode.

Reviewed by:	sjg (indirectly)
MFC after:	1 week
Sponsored by:	Dell EMC Isilon
2017-06-12 19:13:29 +00:00
Bryan Drewery
9369f35755 META_MODE: NO_FILEMON should imply nofilemon.
This fixes NO_FILEMON to properly still use .depend.OBJ files
for dependency tracking.

MFC after:	1 week
Sponsored by:	Dell EMC Isilon
2017-06-12 18:44:14 +00:00
Baptiste Daroussin
174b3709c8 Finish removal of groff 2017-06-07 23:26:28 +00:00
Baptiste Daroussin
738919c039 Remove groff from base
All manpages in base are now compatible with mandoc(1), all roff documentation
will be relocated in the doc tree. man(1) can now use groff from the ports tree
if it needs.

Also remove checknr(1) and colcrt(1) which are only useful with groff.

Approved by:	(no objections on the mailing lists)
2017-06-07 23:00:34 +00:00
Bryan Drewery
1a9cc831dd META_MODE: Add framework to force rebuilding for major ABI changes.
Normally META_MODE ignores host files for "meta mode" decisions on whether a
file should be rebuilt or not.  This is because a simple installworld can
update timestamps and cause the next build to rebuild all host tools, when the
previous ones may not have any changes in the source tree.  These tools are
normally still ABI compatible.  They are only rebuilt if NO_META_IGNORE_HOST is
set from the workaround/hack in r301467.

One of the major problems with this is when a host tool has objects spread
across many revisions that have mixed-ABI.  For example, if struct stat were to
change on the host, some objects for a tool may have different ideas of that
struct's definition.  If just 1 source file were modified and rebuilt and
linked into the tool, then that toll will have mixed-ABI objects and crash.
This exact thing happened with the ino64 commit in r301467 followed by a
trivial update to libbfd in r318750.  The resulting binary would crash in
buildworld.

Sponsored by:	Dell EMC Isilon
2017-06-05 05:17:39 +00:00
Bryan Drewery
ffe0978fbe META_MODE: Allow not ignoring host headers with NO_META_IGNORE_HOST_HEADERS.
See r301467 for more details on NO_META_IGNORE_HOST.  Usually the full
list of host ignores should have no real impact on the host tools.  The
headers however may reliably define what the ABI is for the host.  It
may be useful to allow using the headers for the build but still not
caring about things like /bin/sh, /lib/libedit.so, etc.

Sponsored by:	Dell EMC Isilon
2017-06-05 05:17:36 +00:00
Bryan Drewery
53b3b05944 META_MODE: Move ignoring of /usr/local/etc/libmap.d to proper place.
This was added in r318194 but local.meta.sys.mk is only used for
DIRDEPS_BUILD.
2017-06-01 23:04:44 +00:00
Eric van Gyzen
c9d34d6652 Fix INSTALL_AS_USER with external nsswitch databases
The INSTALL_AS_USER option tells "install" to use the current
user name as the owner of the installed file.  The "install"
command executed by the build is statically linked, so it does not
load nsswitch modules, such as nss_ldap.so, so it fails when
the user is only defined in such a database.

Fix it to use the current UID instead of user name.  This works
for all users.  I expect it is also slightly more efficient.

Reviewed by:	sjg
MFC after:	3 days
Sponsored by:	Dell EMC
Differential Revision:	https://reviews.freebsd.org/D10862
2017-05-28 00:45:28 +00:00
Eric van Gyzen
4f2bab59e7 Fix INSTALL_AS_USER
Move INSTALL_AS_USER into bsd.init.mk to maximize the chance that
it has final authority over fooOWN and fooGRP.

Reviewed by:	sjg
MFC after:	3 days
Sponsored by:	Dell EMC
Differential Revision:	https://reviews.freebsd.org/D10810
2017-05-28 00:43:12 +00:00
Simon J. Gerraty
7e6cd74879 Tell bmake (meta mode) to ignore changes to /usr/local/etc/libmap.d/*
Differential Revision:	D10685
Reviewed by:	bdrewery
2017-05-11 19:49:22 +00:00
Bryan Drewery
f59d3d55b6 PROGS+META_MODE: Avoid rebuilding common sources when recursing.
This could be seen in lib/libkvm/tests where kvm_test_common.o was
a common dependency, but one of the recursed progs had a special
CFLAGS+= -I that changed the build command.  This would cause
all recursed builds to rebuild while fighting over the meta file
and object file.

Reported by:	Mark Millard
MFC after:	1 week
Sponsored by:	Dell EMC Isilon
2017-05-09 18:45:34 +00:00
Bryan Drewery
fbbb9824a3 Fix syntax error in parse_path after r316952.
Also fix bad whitespace in sort_unique after r314809.

The parse_path syntax error came up in DIRDEPS_BUILD as the following
and emptied out all Makefile.depend files due to it:

	# python share/mk/meta2deps.py
	  File "share/mk/meta2deps.py", line 538
	    rdir = os.path.realpath(dir)
	    ^
	IndentationError: unexpected indent

Sponsored by:	Dell EMC Isilon
2017-05-08 20:44:21 +00:00
Baptiste Daroussin
1fc317e374 Build zstandard with threading enabled 2017-05-06 10:59:10 +00:00
Bryan Drewery
1f265bbd8e Fix some "don't build" optimizations from r308599 not working due to a typo.
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-05-06 05:37:36 +00:00
Ed Maste
c602f5f3f5 revert r313473 (Disable LLD_IS_LD option combinations that fail)
r316647 corrected the build of tblgen and libllvm as dependencies for
LLD so undo the temporary seat-belt.

We still want to extend the build infrastructure to automatically detect
the case where the host LLD can be used instead of building a bootstrap
LLD, and likely extend libllvmminimal to meet LLD's needs for cases
where the build includes LLD but not Clang.

Sponsored by:	The FreeBSD Foundation
2017-04-30 16:12:23 +00:00
John Baldwin
73ad3fb536 Add a new GDB_LIBEXEC option to install gdb and kgdb to /usr/libexec.
When this option is enabled, only gdb and kgdb are installed to
/usr/libexec for use by crashinfo(8). Other bits of GDB such as
gdbserver and gdbtui are not installed. For this option to be
effective, GDB must be enabled.

Rework r317094 to re-enable GDB on all platforms but enable
GDB_LIBEXEC on platforms for which the GDB in ports is a superset of
functionality.

Reviewed by:	emaste, kib
Suggested by:	kib
Relnotes:	yes
Differential Revision:	https://reviews.freebsd.org/D10449
2017-04-25 18:08:56 +00:00
Ed Maste
e9e768f7e3 bsdgrep: disable GNU_GREP_COMPAT by default
The GNU extension bits in the base system are old, no longer faithful
to upstream, and surprising in some regards. Switch to documenting
WITH_GNU_GREP_COMPAT and default GNU_GREP_COMPAT to OFF in the name of
good behavior.

According to http://www.regular-expressions.info, GNU extensions:

-  Add missing quantifiers to BREs: \?, \+

-  Add branching to BREs: \|

-  Add backreferences (\1 through \9) to EREs

-  Add \w, \W, \s, and \S corresponding to :alnum:, [^[:alnum:]],
   :space:, and [^[:space:]] respectively

-  Add word boundaries and anchors:
   \b: word boundary
   \B: not word boundary
   \<: Strt of word
   \>: End of word
   \`: Start of subject string
    \': End of subject string

These extensions are still available in /usr/bin/grep by default today,
as it is still GNU grep.  As part of the bsdgrep migration plan these
extensions may be added to bsdgrep's regex support if necessary.

Submitted by:	Kyle Evans <kevans91 at ksu.edu>
Reviewed by:	cem
Differential Revision:	https://reviews.freebsd.org/D10114
2017-04-21 14:50:29 +00:00
Ed Maste
3f39ffc893 bsdgrep: add BSD_GREP_FASTMATCH knob for built-in fastmatch
Bugs have been found in the fastmatch implementation as used in bsdgrep.
Some have been fixed (r316495) while fixes for others are in review
(D10098).

In comparison with the fastmatch implementation, Kyle Evans found that:

- regex(3)'s performance with literal expressions offers a speed
  improvement over fastmatch

- regex(3)'s performance, both with simple BREs and EREs, seems to be
  comparable

The regex implementation was imported in r226035, and the commit message
reports:

    This is a temporary solution until the whole regex library is
    not replaced so that BSD grep development can continue and the
    backported code gets some review and testing. This change only
    improves scalability slightly, there is no big performance boost
    yet but several minor bugs have been found and fixed.

Introduce a WITH_/WITHOUT_BSD_GREP_FASTMATCH knob to support testing
of both approaches.

PR:		175314, 194823
Submitted by:	Kyle Evans <kevans91 at ksu.edu>
Reviewed by:	bdrewery (in part)
Differential Revision:	https://reviews.freebsd.org/D10282
2017-04-21 14:36:09 +00:00
Enji Cooper
609b7b269d Add a knob, WITH*_RPCBIND_WARMSTART_SUPPORT, to allow the end-user to build
rpcbind(8) with/without warmstart support.

The knob defaults to off to preserve POLA for the feature.

See rpcbind(8) for more details about the warmstart feature.

MFC after:	7 weeks
Relnotes:	yes
Sponsored by:	Dell EMC Isilon
2017-04-19 20:45:46 +00:00
Ed Maste
722a4cb445 Remove WITHOUT_GNU and WITHOUT_GNU_SUPPORT src.conf.knobs
These have no effect (and WITHOUT_GNU is documented as having no
effect). I intend to later introduce a WITHOUT_GPL knob to serve a
similar purpose as WITHOUT_GNU's previously documented intent, but with
a more accurate name. To avoid confusion over the transition though just
remove the existing, nonfunctional ones.
2017-04-19 18:59:38 +00:00
Simon J. Gerraty
d86e8e9c57 If MAKEOBJDIRPREFIX is a prefix match for .CURDIR, then __objdir is .CURDIR
Reviewed by:	bdrewery
2017-04-18 23:59:15 +00:00
John Baldwin
99b8bccddc Disable in-tree GDB by default on x86, mips, and powerpc.
GDB in ports contains all of the functionality as GDB in base
(including kgdb) for these platforms along with additional
functionality.  In-tree GDB remains enabled on ARM and sparc64.
GDB in ports does not currently support kernel debugging on arm,
and ports GDB for sparc64 has not been tested (though it does
include sparc64 support).

Reviewed by:	bdrewery, emaste, imp
Relnotes:	yes
Sponsored by:	DARPA / AFRL
Differential Revision:	https://reviews.freebsd.org/D10399
2017-04-18 16:27:48 +00:00
Baptiste Daroussin
e229090553 Import zstandard 1.1.4 in base
zstandard is a new compression library/tool which is very fast at
compression/decompression

For now import as a private library
2017-04-15 20:05:22 +00:00
Simon J. Gerraty
c0339e7dfd Latest version, we need to realpath anyway to check if an obj
dependency is also a src dependency.
2017-04-15 03:28:13 +00:00
Simon J. Gerraty
6029706346 Update to latest, handle case of __objdir=obj 2017-04-15 01:52:49 +00:00
Ed Maste
e592841459 Make WITHOUT_TOOLCHAIN imply WITHOUT_LLD.
LLD is a toolchain component.
2017-04-10 18:21:30 +00:00
Ed Maste
ffe63c8bf0 Introduce LLD_BOOTSTRAP to control lld as bootstrap linker
Add WITH_LLD_BOOTSTRAP and WITHOUT_LLD_BOOTSTRAP knobs, similar to the
Clang bootstrap knobs.

Reviewed by:	dim
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D10249
2017-04-09 01:35:19 +00:00
Alexander Kabaev
19f7eeed61 Do not use -msoft-float with intention of disabling FP on aarch64
GNU GCC does does recognise it as a valid option and we already
use -mgeneral-regs-only that has the desired effect.

Reviewed by: emaste
Differential Revision:	https://reviews.freebsd.org/D10306
2017-04-07 22:58:25 +00:00
John Baldwin
ea22493a46 Explicitly set the desired MIPS ABI in toolchain flags.
Specifically, set '-mabi=XX' in AFLAGS, CFLAGS, and LDFLAGS.  This permits
building MIPS worlds and binaries with a toolchain whose default output
does not match the desired TARGET_ARCH.

_LDFLAGS (which is used with LD instead of with CC) required an update as
LD does not accept the -mabi flags (so they must be stripped from LDFLAGS
when generating _LDFLAGS).  For bare uses of LD (rather than linking via
CC), the desired ABI must be set by setting an explicit linker emulation
as done in r316514 for kernels and kernel modules.

Reviewed by:	imp
Sponsored by:	DARPA / AFRL
Differential Revision:	https://reviews.freebsd.org/D10085
2017-04-07 20:02:01 +00:00
John Baldwin
5429af5f5b Don't set the MIPS endianness flags in both ACFLAGS and CFLAGS.
This should no longer be necessary after r316620 as all places that
use ACFLAGS should already be using CFLAGS.

Reviewed by:	imp
Sponsored by:	DARPA / AFRL
Differential Revision:	https://reviews.freebsd.org/D10085
2017-04-07 19:56:12 +00:00
Andrew Turner
16e77e0036 Add -fPIC to the standalone build flags on arm64. This is needed as
loader.efi is position independend, however we were not building it as
such causing a build failure when building with lld.

Sponsored by:	DARPA, AFRL
2017-04-07 14:30:51 +00:00
Bryan Drewery
dbbb03c244 META_MODE: Fix build-tools still sometimes rebuilding during target build.
In a cross-build, the build-tools are native host binaries.  We do not
want to rebuild them when building for the target.  Bmake previously
did not support checking .NOMETA on an existing target, so .NOMETA_CMP
was used here.  However, .NOMETA_CMP still triggers meta mode conditions
if the number of commands or the command changes.  In r312467 the paths
to build ncurses files were modified and thus triggered meta mode to
rebuild the build tools (make_keys, make_hash) in ncurses during the
target build.  Bmake 20160604 committed in r301462 changed .NOMETA to
also skip meta mode logic for an existing .meta file as well, thus it
is now the proper fix here.

I explored moving the build-tools output to WORLDTMP/tools with
relatively good success, but have concerns that doing so would be
problematic for downstream vendors who use LOCAL_TOOL_DIRS and
expect the tools to be in current OBJDIR for the target.  It also
adds more complexity into finding the tools during target build
and handling of where they are for rescue/rescue and
mkcsmapper_static/mkesdb_static which should really not be connected in
build-tools anyway.

MFC after:	2 weeks
Reported by:	many
Sponsored by:	Dell EMC Isilon
2017-04-06 18:21:59 +00:00
Bryan Drewery
396561c9fb META_MODE: Fix not using .depend files when no OBJDIR is present.
By default bmake does not allow meta mode to work unless an OBJDIR is
present.  It allows this if curdirok= is set with a value not starting
with [0NnFf], but usually it is "yes".

MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-03-17 18:08:36 +00:00
Baptiste Daroussin
75bf90db94 Finish removal of WITHOUT_MANDOCDB 2017-03-12 03:35:01 +00:00
John Baldwin
fc674935bc Fix a couple of typos and reword some sentences in bsd.README.
Reviewed by:	bdrewery
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D9922
2017-03-08 05:49:58 +00:00
Jung-uk Kim
982ee4e2f5 Catch up with Clang 4.0.0. 2017-03-07 19:00:27 +00:00
Enji Cooper
67833f2936 Add bsd.man.mk references for MAN under bsd.lib.mk and bsd.prog.mk
The latter set of manpages directly consume bsd.man.mk, so the bsd.man.mk
behavior should be the source of truth for underlying behavior, whereas
the other manpage fragment descriptions should document how they tweak
the variable behavior, if at all (bsd.prog.mk does tweak the default
value, as noted in its description)

MFC after:	1 week
Sponsored by:	Dell EMC Isilon
2017-03-07 17:53:53 +00:00