Commit Graph

2850 Commits

Author SHA1 Message Date
ngie
c14f6dfb65 Sort the tests alphabetically before adding them to the Kyuafiles
This is being done to aid in debugging test runs, in the event the
output shifts due to refactored Makefiles, added tests, etc.

MFC after:	1 month
2017-07-17 18:20:54 +00:00
kib
84be924362 Provide libdl.
Create libdl.so.1 as a filter for libc.so.7 which exports public dl*
functions. The functions are resolved from the rtld instead, the goal
of creating library is to avoid errors from the static linker due to
missed libdl. For static binaries, an empty .o is compiled into
libdl.a so that static binaries still get dl stubs from libc.a.

Right now lld cannot create filter objects, disable libdl on arm64
when binutils are not used.

Reviewed by:	bdrewery, dim (previos version); emaste
Exp run:	PR 220525, done by antoine
Sponsored by:	The FreeBSD Foundation
MFC after:	1 month
Differential revision:	https://reviews.freebsd.org/D11504
2017-07-10 14:59:21 +00:00
ngie
a448519a3c Formalize LEAPSECONDS and OLDTIMEZONES in share/zoneinfo/... as
`MK_ZONEINFO_LEAPSECONDS_SUPPORT == yes` and
`MK_ZONEINFO_OLD_TIMEZONES_SUPPORT == yes`.

Keep `LEAPSECONDS` and `OLDTIMEZONES` for backwards compatibility,
but print out a warning notifying users that they should use the new
variables, in an effort to migrate them to the variables. This is being
done mostly for automated build tools, etc, that might rely on these
variables being set. The variables will be removed in the future on
^/head, e.g., after ^/stable/12 is cut.

MFC after:      1 month
Relnotes:       yes
Reviewed by:	bdrewery
Differential Revision:	D11376
2017-07-06 04:19:33 +00:00
emaste
16cc7b4e31 Do not build clang for all riscv*, not just riscv64
Previous test matching on "riscv64" was invalidated by the addition of
riscv64sf.

Sponsored by:	The FreeBSD Foundation
2017-07-01 19:10:59 +00:00
jlh
5c2b155b69 Disable RCMDS by default.
This was announced in this thread:
https://lists.freebsd.org/pipermail/freebsd-arch/2017-June/018239.html

Applying plan proposed by ngie@ in:
https://lists.freebsd.org/pipermail/freebsd-arch/2017-June/018249.html

The port has been submitted as net/bsdrcmds in r444814.

Approved by:	bapt, roberto, and others
2017-07-01 10:04:42 +00:00
emaste
e386983c91 bsd.linker.mk: add band-aid for linker invocation failure
In some cases bsd.linker.mk reports an error like:

  make[4]: ".../share/mk/bsd.linker.mk" line 56:
  Unknown linker from LD=ld -m elf32ppc_fbsd:"

For now change this to a .warning, and then assume GNU ld 2.17.50.
At present the linker type detection is used only for enabling build-id,
and we can carry on without it when type detection fails.

Also, show errors from ${LD} --version to aid in failure diagnosis.
Successful invocations of ${LD} --version produce no output on stderr
so this will not create any spam in non-failing builds.

Tested by:	swills
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D11424
2017-06-30 16:34:17 +00:00
emaste
3bde37655a Introduce LINKER_FEATURES to avoid duplicating version logic
Submitted by:	bdrewery
Reported by:	kib
2017-06-23 17:21:37 +00:00
emaste
5816cc6184 change GNU ld LINKER_TYPE from binutils to bfd
GNU binutils includes two linkers: ld.bfd and ld.gold.  For clarity use
LINKER_TYPE=bfd to refer to ld.bfd, the original binutils linker that
identifies itself as "GNU ld".

Discussed with:	bdrewery
2017-06-23 03:55:00 +00:00
bdrewery
656ec7cc2c Provide proper values for X_LINKER_TYPE/VERSION when XLD == LD.
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-06-22 22:53:10 +00:00
bdrewery
3132dadd89 Don't overwrite already-set LINKER_VERSION/LINKER_TYPE.
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-06-22 22:53:07 +00:00
bdrewery
fe8fd0e30d LINKER_VERSION: Support external binutils.
The ports binutils stores the version in the 5th word so just look for
a version using a pattern instead.

Reported by:	rpokala
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-06-22 21:22:12 +00:00
bdrewery
74c01096d1 Support cached linker values in environment.
This is similar to r289659 for bsd.compiler.mk.

MFC after:	2 weeks
Reviewed by:	emaste
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D11310
2017-06-22 21:03:27 +00:00
bdrewery
d53b98880f Support XLD for setting X_LINKER_TYPE and X_LINKER_VERSION.
This is similar to r300350 for bsd.compiler.mk.

MFC after:	2 weeks
Reviewed by:	emaste
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D11309
2017-06-22 21:03:24 +00:00
bdrewery
7cf0e0ad69 Add basic bsd.linker.mk auto included from bsd.compiler.mk.
This will provide LINKER_TYPE and LINKER_VERSION.

MFC after:	2 weeks
Reviewed by:	emaste
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D11308
2017-06-22 21:03:20 +00:00
bdrewery
4a42972a1d Fix spelling error.
Reported by:	arc
Sponsored by:	Dell EMC Isilon
2017-06-22 20:32:23 +00:00
bdrewery
63cec4ec0f Rework logic for skipping .depend/.meta file read/stat/writes.
- Rename _SKIP_READ_DEPEND to _SKIP_DEPEND since it also avoids writing.
- This now uses .NOMETA to avoid reading any .meta files related to
  DEPENDOBJS.  Objects not in OBJS/DEPENDOBJS may still have their .meta
  files read in if they are in the dependency graph.
- This also avoids statting .meta and .depend files in the META_MODE +
  -DNO_FILEMON case.

MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-06-22 05:34:41 +00:00
bdrewery
6eab5891b5 Remove logic for setting .MAKE.DEPENDFILE=/dev/null already covered by _SKIP_BUILD.
_SKIP_BUILD is defined in bsd.init.mk.

MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-06-21 20:10:58 +00:00
bdrewery
70a89a6800 Similar to r296013 for NO_ROOT, force SUBDIR_PARALLEL for buildworld WORLDTMP staging.
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-06-21 18:56:53 +00:00
bdrewery
4f4f919cf3 objwarn should be .PHONY.
Otherwise in META_MODE it may create an objwarn.meta if only bsd.obj.mk
is included;  bsd.sys.mk already had .PHONY: objwarn.

MFC after:	3 days
Sponsored by:	Dell EMC Isilon
2017-06-21 17:11:49 +00:00
bdrewery
57488d2ac8 Fix 'make clean all' to work again.
This likely broke completely with r308599.

Apply the same fix for 'make destroy' which is a DIRDEPS_BUILD thing.

PR:		219819
Reported by:	trasz
MFC after:	3 days
Sponsored by:	Dell EMC Isilon
2017-06-20 20:52:06 +00:00
bdrewery
690a0d91eb LIBADD: Try to support partial tree checkouts in some limited cases.
LIBADD is only supported for in-tree builds because we do not install
share/mk/src.libnames.mk (which provides LIBADD support) into /usr/share/mk.
So if a partial checkout is done then the LIBADDs are ignored and no LDADD is
ever added.

Provide limited support for this case for when LIBADD is composed entirely of
base libraries.  This is to avoid clashes with ports and other out-of-tree
LIBADD uses that should not be mapped to LDADD and because we do not want to
support LIBADD out-of-tree right now.

Reported by:	mckusick, kib
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-06-20 20:34:30 +00:00
emaste
c4e2c09d06 Fall back to GPL dtc(1) when we lack a C++11 compiler
The BSD licensed device tree compiler is written in C++11.

Reported by:	jhibbits
Reviewed by:	jhibbits, manu
Sponsored by:	The FreeBSD Foundation
2017-06-20 15:51:09 +00:00
manu
482531ced3 Switch back to the BSDL DTC (Device Tree Compiler).
The BSDL dtc has grown the needed features (overlays mostly) and is able to
compile all of our base DTS.
You can use WITH_GPL_DTC is you need the GPL one or DTC= in make.conf(5)
to specify an alternate location for the compiler to use.

Discussed with: emaste, imp
2017-06-20 02:09:50 +00:00
bdrewery
6c4b8e235b Utilize SYSROOT from r320119 in places where DESTDIR may be wanting WORLDTMP.
Since buildenv exports SYSROOT all of these uses will now look in
WORLDTMP by default.

sys/boot/efi/loader/Makefile
        A LIBSTAND hack is no longer required for buildenv.

MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-06-19 20:47:24 +00:00
bdrewery
c3d7eb1d7b Follow-up r320061: Need to respect make.conf/env LIBDIR overrides.
This fixes the lib32 build from creating all stale .depend files.

X-MFC-With:	320061
MFC after:	1 week
Sponsored by:	Dell EMC Isilon
2017-06-19 18:08:02 +00:00
bdrewery
b40354c0a7 Fix Makefiles which override LIBDIR to not add incorrect dependencies into .depend.
This fixes these cases which would rebuild every time:
    make[6]: /usr/obj/usr/src/libexec/rtld-elf/tests/libpythagoras/.depend, 1: ignoring stale .depend for /usr/obj/usr/src/tmp/usr/tests/libexec/rtld-elf/libm.a
    make[6]: /usr/obj/usr/src/lib/libxo/tests/encoder/.depend, 1: ignoring stale .depend for /usr/obj/usr/src/tmp/usr/tests/lib/libxo/libxo.a
    make[7]: /usr/obj/usr/src/lib/libthr/tests/dlopen/dso/.depend, 1: ignoring stale .depend for /usr/obj/usr/src/tmp/usr/tests/lib/libthr/dlopen/libpthread.a

The problem is that some Makefiles will override LIBDIR to where they want
their library to install.  bsd.libnames.mk will then use ${LIBDIR} to define
where *existing* libraries are.  This then leads to looking for the
libraries in the *target* place rather than the *expected* place.

We may want to expand this (and all of the other *DIR variables in bsd.own.mk)
into something like what Ports has, a PREFIX and a LOCALBASE.  PREFIX being
where things are being installed to and LOCALBASE being where they already are.

For now store the default expected LIBDIR into LIBDIR_BASE and use that for
library locations.

Reported by:	sbruno
MFC after:	1 week
Sponsored by:	Dell EMC Isilon
2017-06-17 20:33:11 +00:00
bdrewery
df1398b14a WITH_META_MODE: End each ERROR_CMD CMD line with ';'.
This makes it easier to debug multi-line command failures.

X-MFC-With:	r319862
MFC after:	1 week
Sponsored by:	Dell EMC Isilon
2017-06-16 21:48:42 +00:00
bdrewery
488ac365fc Fix more incorrect library directories fix 'stale .depend' rebuilds.
Reported by:	sbruno
MFC after:	3 days
Sponsored by:	Dell EMC Isilon
2017-06-16 21:37:05 +00:00
bdrewery
3ed41bbad6 Fix LIBAMU location to fix 'stale .depend' rebuilds in usr.sbin/amd.
This originally came in r275052.

Reported by:	sbruno
MFC after:	3 days
Sponsored by:	Dell EMC Isilon
2017-06-16 20:47:12 +00:00
bdrewery
af137d191a WITH_META_MODE: Don't try showing command if .ERROR_META_FILE is empty.
This was sed'ing on stdin for failing .PHONY targets.

Reported by:	Mark Millard
X-MFC-With:	r319862
MFC after:	1 week
Sponsored by:	Dell EMC Isilon
2017-06-16 04:28:10 +00:00
bdrewery
55bd2850e7 META_MODE: Show .ERROR_CMD in error.
This uses a hack to get the CMD from the meta file rather than
.ERROR_CMD since bmake currently blanks the value for non-jobs
mode.

Reviewed by:	sjg (indirectly)
MFC after:	1 week
Sponsored by:	Dell EMC Isilon
2017-06-12 19:13:29 +00:00
bdrewery
659ce4bc88 META_MODE: NO_FILEMON should imply nofilemon.
This fixes NO_FILEMON to properly still use .depend.OBJ files
for dependency tracking.

MFC after:	1 week
Sponsored by:	Dell EMC Isilon
2017-06-12 18:44:14 +00:00
bapt
c0d513aaf1 Finish removal of groff 2017-06-07 23:26:28 +00:00
bapt
ac6edc194c Remove groff from base
All manpages in base are now compatible with mandoc(1), all roff documentation
will be relocated in the doc tree. man(1) can now use groff from the ports tree
if it needs.

Also remove checknr(1) and colcrt(1) which are only useful with groff.

Approved by:	(no objections on the mailing lists)
2017-06-07 23:00:34 +00:00
bdrewery
3deaa65c41 META_MODE: Add framework to force rebuilding for major ABI changes.
Normally META_MODE ignores host files for "meta mode" decisions on whether a
file should be rebuilt or not.  This is because a simple installworld can
update timestamps and cause the next build to rebuild all host tools, when the
previous ones may not have any changes in the source tree.  These tools are
normally still ABI compatible.  They are only rebuilt if NO_META_IGNORE_HOST is
set from the workaround/hack in r301467.

One of the major problems with this is when a host tool has objects spread
across many revisions that have mixed-ABI.  For example, if struct stat were to
change on the host, some objects for a tool may have different ideas of that
struct's definition.  If just 1 source file were modified and rebuilt and
linked into the tool, then that toll will have mixed-ABI objects and crash.
This exact thing happened with the ino64 commit in r301467 followed by a
trivial update to libbfd in r318750.  The resulting binary would crash in
buildworld.

Sponsored by:	Dell EMC Isilon
2017-06-05 05:17:39 +00:00
bdrewery
e146f9c0bb META_MODE: Allow not ignoring host headers with NO_META_IGNORE_HOST_HEADERS.
See r301467 for more details on NO_META_IGNORE_HOST.  Usually the full
list of host ignores should have no real impact on the host tools.  The
headers however may reliably define what the ABI is for the host.  It
may be useful to allow using the headers for the build but still not
caring about things like /bin/sh, /lib/libedit.so, etc.

Sponsored by:	Dell EMC Isilon
2017-06-05 05:17:36 +00:00
bdrewery
f436c649ba META_MODE: Move ignoring of /usr/local/etc/libmap.d to proper place.
This was added in r318194 but local.meta.sys.mk is only used for
DIRDEPS_BUILD.
2017-06-01 23:04:44 +00:00
vangyzen
9a2a2c228b Fix INSTALL_AS_USER with external nsswitch databases
The INSTALL_AS_USER option tells "install" to use the current
user name as the owner of the installed file.  The "install"
command executed by the build is statically linked, so it does not
load nsswitch modules, such as nss_ldap.so, so it fails when
the user is only defined in such a database.

Fix it to use the current UID instead of user name.  This works
for all users.  I expect it is also slightly more efficient.

Reviewed by:	sjg
MFC after:	3 days
Sponsored by:	Dell EMC
Differential Revision:	https://reviews.freebsd.org/D10862
2017-05-28 00:45:28 +00:00
vangyzen
cafe808c23 Fix INSTALL_AS_USER
Move INSTALL_AS_USER into bsd.init.mk to maximize the chance that
it has final authority over fooOWN and fooGRP.

Reviewed by:	sjg
MFC after:	3 days
Sponsored by:	Dell EMC
Differential Revision:	https://reviews.freebsd.org/D10810
2017-05-28 00:43:12 +00:00
sjg
8dfef7b39f Tell bmake (meta mode) to ignore changes to /usr/local/etc/libmap.d/*
Differential Revision:	D10685
Reviewed by:	bdrewery
2017-05-11 19:49:22 +00:00
bdrewery
c427d5e8c2 PROGS+META_MODE: Avoid rebuilding common sources when recursing.
This could be seen in lib/libkvm/tests where kvm_test_common.o was
a common dependency, but one of the recursed progs had a special
CFLAGS+= -I that changed the build command.  This would cause
all recursed builds to rebuild while fighting over the meta file
and object file.

Reported by:	Mark Millard
MFC after:	1 week
Sponsored by:	Dell EMC Isilon
2017-05-09 18:45:34 +00:00
bdrewery
d5e00e118e Fix syntax error in parse_path after r316952.
Also fix bad whitespace in sort_unique after r314809.

The parse_path syntax error came up in DIRDEPS_BUILD as the following
and emptied out all Makefile.depend files due to it:

	# python share/mk/meta2deps.py
	  File "share/mk/meta2deps.py", line 538
	    rdir = os.path.realpath(dir)
	    ^
	IndentationError: unexpected indent

Sponsored by:	Dell EMC Isilon
2017-05-08 20:44:21 +00:00
bapt
7f77dcaf84 Build zstandard with threading enabled 2017-05-06 10:59:10 +00:00
bdrewery
329898f839 Fix some "don't build" optimizations from r308599 not working due to a typo.
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-05-06 05:37:36 +00:00
emaste
c787b852ae revert r313473 (Disable LLD_IS_LD option combinations that fail)
r316647 corrected the build of tblgen and libllvm as dependencies for
LLD so undo the temporary seat-belt.

We still want to extend the build infrastructure to automatically detect
the case where the host LLD can be used instead of building a bootstrap
LLD, and likely extend libllvmminimal to meet LLD's needs for cases
where the build includes LLD but not Clang.

Sponsored by:	The FreeBSD Foundation
2017-04-30 16:12:23 +00:00
jhb
54f1738f42 Add a new GDB_LIBEXEC option to install gdb and kgdb to /usr/libexec.
When this option is enabled, only gdb and kgdb are installed to
/usr/libexec for use by crashinfo(8). Other bits of GDB such as
gdbserver and gdbtui are not installed. For this option to be
effective, GDB must be enabled.

Rework r317094 to re-enable GDB on all platforms but enable
GDB_LIBEXEC on platforms for which the GDB in ports is a superset of
functionality.

Reviewed by:	emaste, kib
Suggested by:	kib
Relnotes:	yes
Differential Revision:	https://reviews.freebsd.org/D10449
2017-04-25 18:08:56 +00:00
emaste
af85d5feea bsdgrep: disable GNU_GREP_COMPAT by default
The GNU extension bits in the base system are old, no longer faithful
to upstream, and surprising in some regards. Switch to documenting
WITH_GNU_GREP_COMPAT and default GNU_GREP_COMPAT to OFF in the name of
good behavior.

According to http://www.regular-expressions.info, GNU extensions:

-  Add missing quantifiers to BREs: \?, \+

-  Add branching to BREs: \|

-  Add backreferences (\1 through \9) to EREs

-  Add \w, \W, \s, and \S corresponding to :alnum:, [^[:alnum:]],
   :space:, and [^[:space:]] respectively

-  Add word boundaries and anchors:
   \b: word boundary
   \B: not word boundary
   \<: Strt of word
   \>: End of word
   \`: Start of subject string
    \': End of subject string

These extensions are still available in /usr/bin/grep by default today,
as it is still GNU grep.  As part of the bsdgrep migration plan these
extensions may be added to bsdgrep's regex support if necessary.

Submitted by:	Kyle Evans <kevans91 at ksu.edu>
Reviewed by:	cem
Differential Revision:	https://reviews.freebsd.org/D10114
2017-04-21 14:50:29 +00:00
emaste
dd5e4bc8dd bsdgrep: add BSD_GREP_FASTMATCH knob for built-in fastmatch
Bugs have been found in the fastmatch implementation as used in bsdgrep.
Some have been fixed (r316495) while fixes for others are in review
(D10098).

In comparison with the fastmatch implementation, Kyle Evans found that:

- regex(3)'s performance with literal expressions offers a speed
  improvement over fastmatch

- regex(3)'s performance, both with simple BREs and EREs, seems to be
  comparable

The regex implementation was imported in r226035, and the commit message
reports:

    This is a temporary solution until the whole regex library is
    not replaced so that BSD grep development can continue and the
    backported code gets some review and testing. This change only
    improves scalability slightly, there is no big performance boost
    yet but several minor bugs have been found and fixed.

Introduce a WITH_/WITHOUT_BSD_GREP_FASTMATCH knob to support testing
of both approaches.

PR:		175314, 194823
Submitted by:	Kyle Evans <kevans91 at ksu.edu>
Reviewed by:	bdrewery (in part)
Differential Revision:	https://reviews.freebsd.org/D10282
2017-04-21 14:36:09 +00:00
ngie
1fa93003ee Add a knob, WITH*_RPCBIND_WARMSTART_SUPPORT, to allow the end-user to build
rpcbind(8) with/without warmstart support.

The knob defaults to off to preserve POLA for the feature.

See rpcbind(8) for more details about the warmstart feature.

MFC after:	7 weeks
Relnotes:	yes
Sponsored by:	Dell EMC Isilon
2017-04-19 20:45:46 +00:00
emaste
ae274c92ce Remove WITHOUT_GNU and WITHOUT_GNU_SUPPORT src.conf.knobs
These have no effect (and WITHOUT_GNU is documented as having no
effect). I intend to later introduce a WITHOUT_GPL knob to serve a
similar purpose as WITHOUT_GNU's previously documented intent, but with
a more accurate name. To avoid confusion over the transition though just
remove the existing, nonfunctional ones.
2017-04-19 18:59:38 +00:00