Commit Graph

1671 Commits

Author SHA1 Message Date
Dimitry Andric
54ff5d7323 Add a SUBDIR_PARALLEL option to bsd.subdir.mk, to allow make to process
all the SUBDIR entries in parallel, instead of serially.  Apply this
option to a selected number of Makefiles, which can greatly speed up the
build on multi-core machines, when using make -j.

This can be extended to more Makefiles later on, whenever they are
verified to work correctly with parallel building.

I tested this on a 24-core machine, with make -j48 buildworld (N = 6):

                before    stddev       after    stddev
                =======   ======       =======  ======
real time        1741.1     16.5         959.8     2.7
user time       12468.7     16.4       14393.0    16.8
sys  time        1825.0     54.8        2110.6    22.8

(user+sys)/real     8.2                   17.1

E.g. the build was approximately 45% faster in real time.  On machines
with less cores, or with lower -j settings, the speedup will not be as
impressive.  But at least you can now almost max out a machine with
buildworld!

Submitted by:	jilles
MFC after:	2 weeks
2014-03-26 22:30:38 +00:00
Andrew Turner
73279d4113 Add a new ARM TARGET_ARCH, armv6hf. This is considered experimental.
This targets the existing ARMv6 and ARMv7 SoCs that contain a VFP unit.
This is an optional coprocessors may not be present in all devices, however
it appears to be in all current SoCs we support.

armv6hf targets the VFP variant of the ARM EABI and our copy of gcc is too
old to support this. Because of this there are a number of WITH/WITHOUT
options that are unsupported and must be left as the default value. The
options and their required value are:
 * WITH_ARM_EABI
 * WITHOUT_GCC
 * WITHOUT_GNUCXX

In addition, without an external toolchain, the following need to be left
as their default:
 * WITH_CLANG
 * WITH_CLANG_IS_CC

As there is a different method of passing float and double values to
functions the ABI is incompatible with existing armv6 binaries. To use
this a full rebuild of world is required. Because no floating point values
are passed into the kernel an armv6 kernel with VFP enabled will work with
an armv6hf userland and vice versa.
2014-03-23 12:49:25 +00:00
Jung-uk Kim
6619822598 Add new CPUTYPEs supported by Clang 3.4, i.e., AMD Steamroller (bdver3) and
Intel Silvermont (slm) processors.
2014-03-20 19:17:46 +00:00
Ian Lepore
44653522f6 Add a way to apply CFLAGS only when building the given architecture. This
is useful primarily on a system used for cross-building, when you have a
set of flags to apply to the TARGET_ARCH being cross-built but don't want
those settings applied to building the cross-tools or other components that
run on the build host machine.
2014-03-19 18:54:53 +00:00
Julio Merino
4692f5ba26 Document support for TAP-compliant Perl test programs. 2014-03-16 01:22:23 +00:00
Julio Merino
a5bf6b6d21 Add some documentation for bsd.test.mk. 2014-03-15 09:27:59 +00:00
Julio Merino
a784098c10 Move FreeBSD Test Suite-specific code to a suite.test.mk file.
The new suite.test.mk file contains all the logic needed to install test
programs under /usr/tests/ and to support Kyua as the run-time engine.
This file is included by default by bsd.test.mk so Makefiles do not need
to care about its existence.

Specific Makefiles can define NOT_FOR_TEST_SUITE to indicate that whatever
test programs they are building are not supposed to be installed under
/usr/tests/ nor run by Kyua.  (The effect of passing this setting is that
suite.test.mk is simply not included.)

NOT_FOR_TEST_SUITE should never be used by Makefiles in the base system.
This functionality is provided so that third-parties can hook in their
own test code, with different semantics, if they wish.  This was asked
for by sjg@.
2014-03-14 12:52:55 +00:00
Julio Merino
f5fd950e35 Make bsd.test.mk the only public mk fragment for the building of tests.
Change {atf,plain,tap}.test.mk to be internal implementation details of
bsd.test.mk.  Makefiles that build tests should now only include bsd.test.mk
and declaratively specify what they want to build, without worrying about
the internal implementation of the mk files.

The reason for this change is to permit building test programs of different
interfaces from a single directory, which is something I had a need for
while porting tests over from src/tools/regression/.

Additionally, this change makes it possible to perform some other requested
changes to bsd.test.mk in an easier manner.  Coming soon.
2014-03-14 08:56:19 +00:00
Gleb Smirnoff
2c284d9395 Remove IPX support.
IPX was a network transport protocol in Novell's NetWare network operating
system from late 80s and then 90s. The NetWare itself switched to TCP/IP
as default transport in 1998. Later, in this century the Novell Open
Enterprise Server became successor of Novell NetWare. The last release
that claimed to still support IPX was OES 2 in 2007. Routing equipment
vendors (e.g. Cisco) discontinued support for IPX in 2011.

Thus, IPX won't be supported in FreeBSD 11.0-RELEASE.
2014-03-14 02:58:48 +00:00
Andrey A. Chernov
c34a97a05d Add fetch and patch to .PHONY and .NOTMAIN only if PORTNAME is defined.
Unconditional addition cause real fetch(1) and patch(1) programs are
builded every time.

MFC after:      1 week
2014-03-02 13:12:06 +00:00
Warner Losh
eeb913c99f Integrate device-tree upstream files into the build process:
(1) Invoke cpp to bring in files via #include (although the old
    /include/ stuff is supported still).
(2) bring in files from either vendor tree or freebsd-custom files
    when building.
(3) move all dts* files from sys/boot/fdt/dts to
    sys/boot/fdt/dts/${MACHINE} as appropriate.
(4) encode all the magic to do the build in sys/tools/fdt/make_dtb.sh
    so that the different places in the tree use the exact same logic.
(5) switch back to gpl dtc by default. the bsdl one in the tree has
    significant issues not easily addressed by those unfamiliar with
    the code.
2014-02-28 18:29:09 +00:00
Dimitry Andric
892620150f Merge from head up to r262415. 2014-02-23 23:33:11 +00:00
Baptiste Daroussin
d0bfcaf83e Remove libyaml, it has been replaced by libucl 2014-02-23 22:27:14 +00:00
Baptiste Daroussin
c99fb5f907 Import libucl into head
UCL is heavily infused by nginx configuration as the example of a convenient
configuration system. However, UCL is fully compatible with JSON format and is
able to parse json files.

UCL is used by pkg(8) for its configuration file as well for the manifest format
in packages, it will be used in base for the pkg boostrap (signature checking
and configuration file parsing.)

libucl has been developped and is maintained by vsevolod@
2014-02-23 21:49:21 +00:00
Baptiste Daroussin
6a54f620e5 Rename WITHOUT_DMA into WITHOUT_DMAGENT to avoid confusion
Requested by:	ian
2014-02-22 13:05:23 +00:00
Dimitry Andric
9bc21aae69 Merge from head up to r262311. 2014-02-21 22:54:35 +00:00
Dimitry Andric
2879baf24b Move the part in bsd.own.mk that sets -Wno-c++11-extensions for clang to
bsd.sys.mk, where it really belongs.  This also causes the flag to get
added when clang is *not* the default system compiler, but is still
used, e.g. by setting WITH_CLANG_IS_CC manually.

MFC after:	3 days
2014-02-21 22:38:25 +00:00
Dimitry Andric
7e4113e2e1 In case source files are compiled with -g, tell clang not to emit .cfi
directives, since on sparc64 we must still GNU as, which does not
support those directives.

Note there are several programs and libraries in our tree, which are
always compiled with -g, even if DEBUG_FLAGS is not set by the user!
2014-02-21 19:58:45 +00:00
Baptiste Daroussin
a9e8641da9 Import Dragonfly Mail Agent into base system
It is a small and lightweight Mail Transport Agent.
It accepts mails from locally installed Mail User Agents (MUA) and delivers the
mails either locally or to a remote destination. Remote delivery includes
several features like TLS/SSL support, SMTP authentication and NULLCLIENT.

Make dma conditional to new WITHOUT_DMA option and make it respect WITHOUT_MAIL

Reviewed by:	peter
Discussed with:	emaste, bz, peter
2014-02-21 07:26:49 +00:00
Dimitry Andric
f785676f2a Upgrade our copy of llvm/clang to 3.4 release. This version supports
all of the features in the current working draft of the upcoming C++
standard, provisionally named C++1y.

The code generator's performance is greatly increased, and the loop
auto-vectorizer is now enabled at -Os and -O2 in addition to -O3.  The
PowerPC backend has made several major improvements to code generation
quality and compile time, and the X86, SPARC, ARM32, Aarch64 and SystemZ
backends have all seen major feature work.

Release notes for llvm and clang can be found here:
<http://llvm.org/releases/3.4/docs/ReleaseNotes.html>
<http://llvm.org/releases/3.4/tools/clang/docs/ReleaseNotes.html>

MFC after:	1 month
2014-02-16 19:44:07 +00:00
Brooks Davis
8f26887dfb Merge from CheriBSD:
commit 2d581e8caf79d7a0f5a24590eccd06da90cccb74
Author: Brooks Davis <brooks@one-eyed-alien.net>
Date:   Mon Jan 27 22:57:51 2014 +0000

    Add WITHOUT_FMTREE to disable building fmtree.

MFC after:	4 weeks
Sponsored by:	DARPA, AFRL
2014-01-30 21:37:43 +00:00
Brooks Davis
6adfbbbf16 Merge from CheriBSD:
commit 6b569451b92c48ccf1768da32e7e89189e1aa253
Author: Brooks Davis <brooks@one-eyed-alien.net>
Date:   Mon Jan 27 22:50:46 2014 +0000

    Always install nmtree as mtree.
    For compability, link mtree to nmtree.

X-MFC after:	never
Sponsored by:	DARPA, AFRL
2014-01-30 21:25:01 +00:00
Brooks Davis
38e233371c Merge from CheriBSD:
commit c1acf022c533c5ae27e0cd556977eafe3f5959eb
Author: Brooks Davis <brooks@one-eyed-alien.net>
Date:   Fri Jan 17 21:46:44 2014 +0000

    Add an option WITHOUT_NCURSESW to suppress building and linking to
    libncursesw.  While wide character support it useful we'd like to
    only need one ncurses library on embedded systems.

MFC after:	4 weeks
Sponsored by:	DARPA, AFRL
2014-01-30 21:08:36 +00:00
Steven Kreuzer
c904339f2f Remove WITHOUT_ATF as it has been replaced by WITH_TESTS
PR:		conf/185326
Reported by:	Igor Mozolevsky <igor@hybrid-lab.co.uk>
Approved by:	hrs (mentor)
2014-01-23 03:14:48 +00:00
Ed Schouten
46fec22881 Correct value of LIBCOMPILER_RT.
Caught by:	Luca Bayer
2014-01-18 14:47:34 +00:00
Ed Schouten
37656b872b Replace LIBGCC by LIBCOMPILER_RT.
We now use libcompiler_rt on all platforms now. Instead of referring
directly to -lgcc and LIBGCC, use -lcompiler_rt and LIBCOMPILER_RT.
2014-01-18 14:22:56 +00:00
Julio Merino
e8de5c5235 Support perl-based TAP-compliant test programs.
Introduce a TAP_TESTS_PERL primitive to list test programs written in perl.
Only do this in tap.test.mk because I only expect perl-based test programs
with this interface.

This is very similar to TAP_TESTS_SH but the difference is that we record
in the Kyuafile that the test program requires a perl interpreter.  This
in turn makes Kyua mark the test as skipped if the perl package is not yet
installed, instead of mysteriously failing to run the program.

MFC after:	5 days
2014-01-14 18:35:56 +00:00
Julio Merino
8c465f5880 Support defining test program metadata from the Makefiles.
Introduce a new, per-test-program TEST_METADATA.<program> variable that
contains a list of key/value paris describing metadata properties for
that test program.  These properties are later written into the
auto-generated Kyuafile when using the KYUAFILE=auto functionality.

This is to avoid having to supply hand-crafted Kyuafiles when the needs
for metadata overrides are trivial.

While doing this, and because I am documenting TEST_METADATA, take the
chance to document the TEST_INTERFACE setting as well.

MFC after:	5 days
2014-01-14 18:32:47 +00:00
Julio Merino
d7efee230d Allow tests to provide a Kyuafile when they relied on auto-generation.
When generating a Kyuafile in the KYUAFILE=auto case, use a filename
that is unlikely to clash with the filename used by explicitly-provided
Kyuafiles.

This allows a Makefile to set KYUAFILE=yes and provide a Kyuafile in
the same directory when such Makefile was previously relying on
KYUAFILE=auto.

Fixes issues with new Kyuafiles not being picked up in NO_CLEAN builds
(although manual intervention is required once, unfortunately, as
described in UPDATING).

Reviewed by:	sjg
MFC after:	1 week
2014-01-10 10:36:14 +00:00
Dimitry Andric
f66aed321e Apply band-aid for 32-bit compat libs failures after r260334: put back
-Qunused-arguments for clang for now, until I can figure out a way to
make it unneeded in all scenarios.  Sorry about the breakage.
2014-01-06 14:39:10 +00:00
Dimitry Andric
4abd7edcbd Split the last gcc-specific flags off into CFLAGS.gcc. This also
removes the need to use -Qunused-arguments for clang throughout the
tree.

MFC after:	3 days
2014-01-05 21:03:49 +00:00
Dimitry Andric
20dcbdb6be In addition to r260102, also define GCC_MS_EXTENSIONS in bsd.sys.mk,
since kernel module builds do not use kern.pre.mk.

MFC after:	3 days
X-MFC-With:	r260102
2014-01-05 16:40:41 +00:00
Dimitry Andric
b294993d63 To avoid having to explicitly test COMPILER_TYPE for setting
clang-specific or gcc-specific flags, introduce the following new
variables for use in Makefiles:

CFLAGS.clang
CFLAGS.gcc
CXXFLAGS.clang
CXXFLAGS.gcc

In bsd.sys.mk, these get appended to the regular CFLAGS or CXXFLAGS for
the right compiler.

MFC after:	1 week
2013-12-22 17:51:33 +00:00
Ed Maste
7c27d02398 Require a C++11 compiler to build LLDB
In practice the old test (using MK_CLANG_IS_CC) is similar, but
COMPILER_FEATURES provides the information we actually want to test.

Sponsored by:	DARPA, AFRL
2013-12-17 18:30:37 +00:00
Julio Merino
f4a4cdd166 Make bsd.progs.mk work in directories with SCRIPTS but no PROGS.
This change fixes some cases where bsd.progs.mk would fail to handle
directories with SCRIPTS but no PROGS.  In particular, "install" did
not handle such scripts nor dependent files when bsd.subdir.mk was
added to the mix.

This is "make tinderbox" clean.

Reviewed by:	freebsd-testing
Approved by:	rpaulo (mentor)
2013-12-11 03:41:07 +00:00
Julio Merino
d8278f57f8 Add tap.test.mk.
This file provides support to build test programs that comply with the
Test Anything Protocol.  Its main goal is to support the painless
integration of existing tests from tools/regression/ into the Kyua-based
test suite.

Approved by:	rpaulo (mentor)
2013-12-11 03:39:50 +00:00
Dimitry Andric
e33625bfdc For WARNS <= 3, change the clang warning flag -Wno-conversion to
-Wno-enum-conversion.  In earlier clang versions (before 3.2), the
latter did not exist, and suppressing enum conversion warnings was
really the goal of this warning suppression flag.

This should enable the same kind of warning again as was fixed by
r259072 ("incompatible integer to pointer conversion passing 'Elf_Addr'
(aka 'unsigned int') to parameter of type 'void *'"), and which was only
emitted by gcc.

Noticed by:	kib
MFC after:	3 days
2013-12-07 22:30:07 +00:00
Pawel Jakub Dawidek
42a8595256 Please welcome casperd daemon. It (and its services) will be responsible for
giving access to functionality that is not available in capability mode
sandbox. The functionality can be precisely restricted.

Start with the following services:
- system.dns - provides API compatible to:
	- gethostbyname(3),
	- gethostbyname2(3),
	- gethostbyaddr(3),
	- getaddrinfo(3),
	- getnameinfo(3),
- system.grp - provides getgrent(3)-compatible API,
- system.pwd - provides getpwent(3)-compatible API,
- system.random - allows to obtain entropy from /dev/random,
- system.sysctl - provides sysctlbyname(3-compatible API.

Sponsored by:	The FreeBSD Foundation
2013-12-02 08:21:28 +00:00
Pawel Jakub Dawidek
2f02600abf Move my simple logging API to a separate library. It is now already used
by hastctl(8), hastd(8) and auditdistd(8) and will soon be also used
by casperd(8) and its services. There is no documentation and pjdlog.h
header file is not installed in /usr/include/ to keep it private.
Unfortunately we don't have /lib/private/ at this point, only
/usr/lib/private/, so the library is installed in /lib/.

Sponsored by:	The FreeBSD Foundation
2013-12-01 09:41:06 +00:00
Peter Wemm
17e27327de Revert r258672. We clearly don't compile libraries enough times. 2013-11-27 19:51:50 +00:00
Peter Wemm
ff678e6c81 At great personal risk, change the default for LIB32 from yes to no. As
mentioned in UPDATING, you can even do it as an as-needed operation after
doing a buildworld/installworld.  You can set WITH_LIB32=yes in make.conf
or src.conf.
2013-11-27 04:54:23 +00:00
Julio Merino
9e684a3a79 Generate plain sh test programs from a source file.
Instead of assuming that plain sh test programs exist in the source
tree in their final form and are marked as executable, generate them
from a list of sources.

By default, just assume that the source file for a program P is P.sh
but allow the caller to customize the inputs.  Similarly, also allow
the caller to apply sed(1) replacements on the output.  These will
both be useful in hooking existing test code from tools/regression/
into the test suite.

Approved by:	rpaulo (mentor)
2013-11-25 13:30:06 +00:00
Julio Merino
f67f6c4e3b Install plain.test.mk.
This was missed when this file was first imported.  Its atf.test.mk
counterpart is already being installed and these are necessary if we
want "make" within the source tree (not via "buildworld") to work.

Approved by:	rpaulo (mentor)
2013-11-25 13:28:40 +00:00
Simon J. Gerraty
3bd40bb188 Need to also test for defined(${v}_${PROG}) 2013-11-19 00:34:59 +00:00
Julio Merino
a2b5b37cb7 Fix the build of plain test programs.
SRCS.<prog> must be explicitly defined when using the PROGS* functionality
for each program to be built.

As there are no plain test programs in the system yet, this was not
detected.

Approved by:	rpaulo (mentor)
2013-11-18 12:47:35 +00:00
Julio Merino
4e6b339397 Remove registration of C++ test programs into PROGS.
C++ programs need to be added to PROGS_CXX, not PROGS, and the code was
actually doing both.  Just keep the registration into PROGS_CXX to
prevent possible obscure build problems.

Approved by:	rpaulo (mentor)
2013-11-18 12:44:05 +00:00
Simon J. Gerraty
b4ffe103d4 Allow this to work with fmake
Based on patch from Garret Cooper.
2013-11-13 15:10:32 +00:00
Pawel Jakub Dawidek
36da5199bb Bring in libnv library for managing name/value pairs. The following types
are currently supported:

- NV_TYPE_NULL - only name, no data;
- NV_TYPE_BOOL - boolean (true or false);
- NV_TYPE_NUMBER - 64bit unsigned integer;
- NV_TYPE_STRING - C string;
- NV_TYPE_NVLIST - nested nvlist;
- NV_TYPE_DESCRIPTOR - file descriptor;
- NV_TYPE_BINARY - binary data.

For detailed documentation and examples see nv(3) manual page.

Sponsored by:	The FreeBSD Foundation
2013-11-12 19:39:14 +00:00
Glen Barber
7efdb5bc68 Remove stray MK_PKGTOOLS leftover from r257444.
Sponsored by:	The FreeBSD Foundation
2013-11-11 21:19:18 +00:00
Julio Merino
e01d128a42 Subsume the functionality of MK_ATF into MK_TESTS.
There is no reason to keep the two knobs separate: if tests are
enabled, the ATF libraries are required; and if tests are disabled,
the ATF libraries are not necessary.  Keeping the two just serves
to complicate the build.

Reviewed by:	freebsd-testing
Approved by:	rpaulo (mentor)
2013-11-08 14:24:47 +00:00