Commit Graph

37 Commits

Author SHA1 Message Date
Enji Cooper
2740c3ea4a Add MK_TESTS to kern.opts.mk to support r31800
MFC after:	3 weeks
MFC with:	r318008
Pointyhat to:	ngie
Sponsored by:	Dell EMC Isilon
2017-05-09 05:03:35 +00:00
Ruslan Bukin
f13cb8ed2b Disable fformat-extensions for RISC-V target as GCC 6.1 external compiler
has no support for that.

Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D10348
2017-04-12 10:40:30 +00:00
Warner Losh
08356e1450 Remove EISA build option 2017-02-17 06:13:49 +00:00
Andrey V. Elsukov
fcf596178b Merge projects/ipsec into head/.
Small summary
 -------------

o Almost all IPsec releated code was moved into sys/netipsec.
o New kernel modules added: ipsec.ko and tcpmd5.ko. New kernel
  option IPSEC_SUPPORT added. It enables support for loading
  and unloading of ipsec.ko and tcpmd5.ko kernel modules.
o IPSEC_NAT_T option was removed. Now NAT-T support is enabled by
  default. The UDP_ENCAP_ESPINUDP_NON_IKE encapsulation type
  support was removed. Added TCP/UDP checksum handling for
  inbound packets that were decapsulated by transport mode SAs.
  setkey(8) modified to show run-time NAT-T configuration of SA.
o New network pseudo interface if_ipsec(4) added. For now it is
  build as part of ipsec.ko module (or with IPSEC kernel).
  It implements IPsec virtual tunnels to create route-based VPNs.
o The network stack now invokes IPsec functions using special
  methods. The only one header file <netipsec/ipsec_support.h>
  should be included to declare all the needed things to work
  with IPsec.
o All IPsec protocols handlers (ESP/AH/IPCOMP protosw) were removed.
  Now these protocols are handled directly via IPsec methods.
o TCP_SIGNATURE support was reworked to be more close to RFC.
o PF_KEY SADB was reworked:
  - now all security associations stored in the single SPI namespace,
    and all SAs MUST have unique SPI.
  - several hash tables added to speed up lookups in SADB.
  - SADB now uses rmlock to protect access, and concurrent threads
    can do SA lookups in the same time.
  - many PF_KEY message handlers were reworked to reflect changes
    in SADB.
  - SADB_UPDATE message was extended to support new PF_KEY headers:
    SADB_X_EXT_NEW_ADDRESS_SRC and SADB_X_EXT_NEW_ADDRESS_DST. They
    can be used by IKE daemon to change SA addresses.
o ipsecrequest and secpolicy structures were cardinally changed to
  avoid locking protection for ipsecrequest. Now we support
  only limited number (4) of bundled SAs, but they are supported
  for both INET and INET6.
o INPCB security policy cache was introduced. Each PCB now caches
  used security policies to avoid SP lookup for each packet.
o For inbound security policies added the mode, when the kernel does
  check for full history of applied IPsec transforms.
o References counting rules for security policies and security
  associations were changed. The proper SA locking added into xform
  code.
o xform code was also changed. Now it is possible to unregister xforms.
  tdb_xxx structures were changed and renamed to reflect changes in
  SADB/SPDB, and changed rules for locking and refcounting.

Reviewed by:	gnn, wblock
Obtained from:	Yandex LLC
Relnotes:	yes
Sponsored by:	Yandex LLC
Differential Revision:	https://reviews.freebsd.org/D9352
2017-02-06 08:49:57 +00:00
Hans Petter Selasky
f3e7afe2d7 Implement kernel support for hardware rate limited sockets.
- Add RATELIMIT kernel configuration keyword which must be set to
enable the new functionality.

- Add support for hardware driven, Receive Side Scaling, RSS aware, rate
limited sendqueues and expose the functionality through the already
established SO_MAX_PACING_RATE setsockopt(). The API support rates in
the range from 1 to 4Gbytes/s which are suitable for regular TCP and
UDP streams. The setsockopt(2) manual page has been updated.

- Add rate limit function callback API to "struct ifnet" which supports
the following operations: if_snd_tag_alloc(), if_snd_tag_modify(),
if_snd_tag_query() and if_snd_tag_free().

- Add support to ifconfig to view, set and clear the IFCAP_TXRTLMT
flag, which tells if a network driver supports rate limiting or not.

- This patch also adds support for rate limiting through VLAN and LAGG
intermediate network devices.

- How rate limiting works:

1) The userspace application calls setsockopt() after accepting or
making a new connection to set the rate which is then stored in the
socket structure in the kernel. Later on when packets are transmitted
a check is made in the transmit path for rate changes. A rate change
implies a non-blocking ifp->if_snd_tag_alloc() call will be made to the
destination network interface, which then sets up a custom sendqueue
with the given rate limitation parameter. A "struct m_snd_tag" pointer is
returned which serves as a "snd_tag" hint in the m_pkthdr for the
subsequently transmitted mbufs.

2) When the network driver sees the "m->m_pkthdr.snd_tag" different
from NULL, it will move the packets into a designated rate limited sendqueue
given by the snd_tag pointer. It is up to the individual drivers how the rate
limited traffic will be rate limited.

3) Route changes are detected by the NIC drivers in the ifp->if_transmit()
routine when the ifnet pointer in the incoming snd_tag mismatches the
one of the network interface. The network adapter frees the mbuf and
returns EAGAIN which causes the ip_output() to release and clear the send
tag. Upon next ip_output() a new "snd_tag" will be tried allocated.

4) When the PCB is detached the custom sendqueue will be released by a
non-blocking ifp->if_snd_tag_free() call to the currently bound network
interface.

Reviewed by:		wblock (manpages), adrian, gallatin, scottl (network)
Differential Revision:	https://reviews.freebsd.org/D3687
Sponsored by:		Mellanox Technologies
MFC after:		3 months
2017-01-18 13:31:17 +00:00
Ed Maste
602f95d4c4 Add WITH_REPRODUCIBLE_BUILD src.conf(5) knob to disable kernel metadata
The kernel builds reproducibly, except for the time, date, user, and
hostname baked into the kernel (reported at startup and via the
kern.version sysctl for uname).  Add a build knob to disable the
inclusion of this metadata.

Reviewed by:	jhb
MFC after:	1 month
Relnotes:	Yes
Sponsored by:	Reproducible Builds World Summit 2, Berlin
Differential Revision:	https://reviews.freebsd.org/D8809
2016-12-15 21:26:58 +00:00
Ruslan Bukin
11dc8730a6 Keep in-sync MK_SSP=no option both with kernel and userspace.
Pointed out by:	emaste
Sponsored by:	DARPA, AFRL
Sponsored by:	HEIF5
2016-10-12 13:51:41 +00:00
Jonathan T. Looney
ad8874fd21 Change the default build behavior so we don't compile extra TCP modules by
default. At least initially, the feature to support multiple TCP stacks is
aimed at supporting advanced use cases and TCP development, but it is not
necessarily aimed at a wide audience. Therefore, there is no need to build
and install the extra TCP stacks by default. Instead, the people who are
using or developing this functionality can add the extra option to build/
install the extra TCP stacks.

However, we do want to build the extra TCP stacks as part of test builds
(e.g. LINT or tinderbox) to ensure that developers who are testing their
changes will know that their changes do not break the additional TCP
stack modules.

After this change, a user will need to add WITH_EXTRA_TCP_STACKS=1 to
make.conf or the kernel config in order to build the extra TCP modules.

Differential Revision:	https://reviews.freebsd.org/D6795
Reviewed by:	sjg
Approved by:	re (kib)
2016-06-10 19:06:11 +00:00
Ian Lepore
99819ca00c When building modules, define make variable OPT_FDT if the kernel config
includes the FDT option.  Use OPT_FDT to conditionally compile modules
that require FDT support.

In the past we've gotten away with using the arch name as a proxy for FDT
support in makefile conditional logic, but now mips has some platforms with
fdt support and some without and we need a more direct test.
2016-05-27 17:40:29 +00:00
Bryan Drewery
497e80911e Remove the old depend (mkdep) code and make FAST_DEPEND the one true way.
Reviewed by:	emaste, hselasky (partial), brooks (brief)
Discussed on:	arch@
Sponsored by:	EMC / Isilon Storage Division
Differential Revision:	https://reviews.freebsd.org/D5742
2016-03-30 23:50:23 +00:00
Ian Lepore
c6fa17beae Build the zfs module on armv6, but not on older arm.
There have been reports in the past that zfs works on armv6, and now people
are wanting to test it again, so let's give them something to work with.
2016-03-21 23:32:13 +00:00
Bryan Drewery
1329d41cb6 Enable FAST_DEPEND by default.
Missed in r296668.

Discussed on:	arch
Sponsored by:	EMC / Isilon Storage Division
2016-03-12 22:25:10 +00:00
Bryan Drewery
ebf5587eca Stop looking up these values in every subdir on install.
This was slowing down installkernel since it was rerunning this in
every module directory.

Sponsored by:	EMC / Isilon Storage Division
2016-03-12 22:21:02 +00:00
Bryan Drewery
70ca9ec4b9 Hook the meta/nofilemon build into using FAST_DEPEND.
FAST_DEPEND is intended to be the "skip 'make depend' and mkdep"
feature.  Since DIRDEPS_BUILD does this already with some of its own
hacks, and filemon doesn't need this, and nofilemon does, teach it how
to handle each of these cases.

In meta+filemon mode filemon will handle dependencies itself via the
meta mode logic in bmake.  We still want to set MK_FAST_DEPEND=yes to
enable some logic that indicates that 'make depend' is skipped in the
traditional sense.  The actual .depend.* files will be skipped.

When nofilemon is set though we still need to track and generate dependencies.

Sponsored by:	EMC / Isilon Storage Division
2016-02-24 17:19:13 +00:00
Andrew Turner
c92ef78663 DTrace is known to work on armv6, enable building it as a module.
Reviewed by:	imp
Sponsored by:	ABT Systems Ltd
Differential Revision:	https://reviews.freebsd.org/D4221
2015-11-20 16:18:27 +00:00
Bryan Drewery
cf1eeb33be Add a FAST_DEPEND option, off by default, which speeds up the build significantly.
This speeds up buildworld by 16% on my system and buildkernel by 35%.

Rather than calling mkdep(1), which is just a wrapper around 'cc -E',
use the modern -MD -MT -MF flags to gather and generate dependencies during
compilation.  This flag was introduced in GCC "a long time ago", in GCC 3.0,
and is also supported by Clang.  (It appears that ICC also supports this but I
do not have access to test it).  This avoids running the preprocessor *twice*
for every build, in both 'make depend' and 'make all'.  This is especially
noticeable when using ccache since it does not cache preprocessor results from
mkdep(1) / 'cc -E', but still speeds up compilation with the -MD flags.

For 'make depend' a tree-walk is still done to ensure that all DPSRCS
are generated when expected, and that beforedepend/afterdepend and
_EXTRADEPEND are all still respected.  In time this may change but for now
I've been conservative.  The time for a tree-walk with -j combined with
SUBDIR_PARALLEL is not significant.  For example, it takes about 9 seconds
with -j15 to walk all of src/ for 'make depend' now on my system.

A .depend file is still generated with the various rules that apply to
the final target, or custom rules.  Otherwise there are now
per-built-object-file .depend files, such as .depend.filename.o.  These
are included directly by make rather than populating .depend with a loop
and .depend lines, which only added overhead to the now almost-NOP 'make
depend' phase.

Before this I experimented with having mkdep(1) called in parallel per-file.
While this improved the kernel and lib/libc 'make depend' phase, it resulted
in slower build times overall.

The -M flags are removed from CFLAGS when linking since they have no effect.

Enabling this by default, for src or out-of-src, can be done once more testing
has been done, such as a ports exp-run, and with more compilers.

The system I used for testing was:
  WITNESS
  Build options: -j20 WITH_LLDB=yes WITH_DEBUG_FILES=yes WITH_FAST_DEPEND=yes
  DISK: ZFS 3-way mirror with very slow disks using SSD l2arc/log.
        The arc was fully populated with src tree files.
  RAM: 76GiB
  CPU: Intel(R) Xeon(R) CPU L5520 @2.27GHz
       2 package(s) x 4 core(s) x 2 SMT threads = hw.ncpu=16

buildworld:
  x buildworld-before
  + buildworld-fastdep
  +-------------------------------------------------------------------------------+
  |+                                                                              |
  |+                                                                              |
  |+                                                                       xx    x|
  |                                                                       |_MA___||
  |A                                                                              |
  +-------------------------------------------------------------------------------+
      N           Min           Max        Median           Avg        Stddev
  x   3       3744.13       3794.31       3752.25     3763.5633     26.935139
  +   3       3153.34       3155.16        3154.2     3154.2333    0.91045776
  Difference at 95.0% confidence
          -609.33 +/- 43.1943
          -16.1902% +/- 1.1477%
          (Student's t, pooled s = 19.0569)

buildkernel:
  x buildkernel-before
  + buildkernel-fastdep
  +-------------------------------------------------------------------------------+
  |+                                                                            x |
  |++                                                                           xx|
  |                                                                             A||
  |A|                                                                             |
  +-------------------------------------------------------------------------------+
      N           Min           Max        Median           Avg        Stddev
  x   3        571.57        573.94        571.79     572.43333     1.3094401
  +   3        369.12        370.57         369.3     369.66333    0.79033748
  Difference at 95.0% confidence
          -202.77 +/- 2.45131
          -35.4225% +/- 0.428227%
          (Student's t, pooled s = 1.0815)

Sponsored by:	EMC / Isilon Storage Division
MFC after:	3 weeks
Relnotes:	yes
2015-11-06 04:45:29 +00:00
Warner Losh
cd9142c718 Add a note of clarification. MK_* variables only control what modules
are built by default. You can still override that with MODULES_EXTRA
for experimental features like ZFS and dtrace on some
architectures. Also note that kernel config files are not affected by
MK_ options listed, though some targets might be.
2015-03-27 17:36:22 +00:00
Warner Losh
0c46f3f950 Categorize certain kernel builds as being broken in certain places.
Differential Revision: https://reviews.freebsd.org/D2011
2015-03-27 02:35:25 +00:00
Warner Losh
3fe1e58de1 Add support for specifying unsupported / broken options that override
any defaults or user specified actions on the command line. This would
be useful for specifying features that are always broken or that
cannot make sense on a specific architecture, like ACPI on pc98 or
EISA on !i386 (!x86 usage of EISA is broken and there's no supported
hardware that could have it in any event). Any items in
__ALWAYS_NO_OPTIONS are forced to "no" regardless of other settings.

Differential Revision: https://reviews.freebsd.org/D2011
2015-03-27 02:35:11 +00:00
Warner Losh
5101927696 Categorize certain kernel builds as being broken in certain places.
Differential Revision: https://reviews.freebsd.org/D2010
2015-03-12 03:57:19 +00:00
Warner Losh
06d4e2ab2a Add support for specifying unsupported / broken options that override
any defaults or user specified actions on the command line. This would
be useful for specifying features that are always broken or that
cannot make sense on a specific architecture, like ACPI on pc98 or
EISA on !i386 (!x86 usage of EISA is broken and there's no supported
hardware that could have it in any event). Any items in
BROKEN_OPTIONS are forced to "no" regardless of other settings.
Clients are expected change BROKEN_OPTIONS with +=. It will not
be unset, so other parts of the build system can have visibility
into the options that are broken on this platform, though this
should be very rare.

Differential Revision: https://reviews.freebsd.org/D2009
2015-03-12 03:57:00 +00:00
Warner Losh
a469e551a4 Enforce that MK_foo options shall be either "yes" or "no" and nothing
else.
2015-02-09 16:03:55 +00:00
Enji Cooper
b359042310 Add MK_AUTOFS knob for building and installing autofs(4), et al
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
2015-01-26 07:15:49 +00:00
Enji Cooper
02629e469f Add MK_BHYVE knob for building and installing bhyve(4), et al
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
2015-01-26 06:44:48 +00:00
Enji Cooper
e3d1e0f6ab Build cuse(4) if MK_CUSE != no
MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division
2015-01-25 05:13:15 +00:00
Enji Cooper
18cc317233 Add MK_CCD knob for building and installing ccd(4), ccdconfig, etc
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
2015-01-25 04:52:48 +00:00
Enji Cooper
b29d6977f3 Add MK_ISCSI knob for building the iscsi initiator, iscsi daemon, kernel
modules, etc

MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
2015-01-25 04:20:11 +00:00
Warner Losh
8a3e6d90e5 Automate options INET/INET6 turning into MK_$opt_SUPPORT
correctly. Fixes if_gfe when building without INET6 and
simplifies if_gif's Makefile.

Sponsored by: Netflix, Inc
2014-11-14 21:10:45 +00:00
Andrew Turner
6d4766c1b8 Remove MK_ARM_EABI, the armeb issues have been fixed. The code to support
the oabi is still in the tree, but it is expected this will be removed
as developers work on surrounding code.

With this commit the ARM EABI is the only supported supported ABI by
FreeBSD on ARMa 32-bit processors.

X-MFC after:	never
Relnotes:	yes
Differential Revision: https://reviews.freebsd.org/D876
2014-10-01 08:26:51 +00:00
Warner Losh
df3394b3de Remove dependence on source tree options. Move all kernel module
options into kern.opts.mk and change all the places where we use
src.opts.mk to pull in the options. Conditionally define SYSDIR and
use SYSDIR/conf/kern.opts.mk instead of a CURDIR path. Replace all
instances of CURDIR/../../etc with STSDIR, but only in the affected
files.

As a special compatibility hack, include bsd.owm.mk at the top of
kern.opts.mk to allow the bare build of sys/modules to work on older
systems. If the defaults ever change between 9.x, 10.x and current for
these options, however, you'll wind up with the host OS' defaults
rather than the -current defaults. This hack will be removed when
we no longer need to support this build scenario.

Reviewed by: jhb
Differential Revision: https://phabric.freebsd.org/D529
2014-08-11 14:50:49 +00:00
Warner Losh
aeaed50898 Move most of the 15 variations on generating opt_inet.h and
opt_inet6.h into kmod.mk by forcing almost everybody to eat the same
dogfood. While at it, consolidate the opt_bpf.h and opt_mroute.h
targets here too.
2014-08-04 22:37:02 +00:00
Warner Losh
d93a1c8a02 Add ARM_EABI to the list, since arm kernels need it
# Note: MK_ARM_EABI likely is going to die soon.
2014-05-19 16:13:40 +00:00
Warner Losh
1f4e0ed969 The time is not yet ripe to break the lack of dependencies between
src/sys and the rest of the tree for builds.
o eliminate including bsd.mkopts.mk for the moment in kern.opts.mk
o No need to include src.opts.mk at all anymore. The reasons for it
  are now coverted in sys.mk and src.sys.mk.
2014-05-17 20:31:34 +00:00
Warner Losh
018c8b7687 s/JIRA/Jenkins/g in comments. I was confused. 2014-05-12 01:47:45 +00:00
Warner Losh
8783345b6c Attempt to walk a fine line between current usage (/usr/ports which
does an out-of-tree build without setting MAKESYSPATH) and recently
added requirements (JIRA's building the modules in a non-standard
layout). So, when MAKESYSPATH is defined, trust that it will do the
right thing (to catch the JIRA use case). When it isn't defined,
assume a standard FreeBSD tree and reach over to grab bsd.mkopt.mk (to
fix the /usr/ports use case). Both camps cannot be appeased otherwise,
so we have this kludge until it can be sorted out.
2014-05-11 23:22:32 +00:00
Warner Losh
5a701e9953 Fix typo in FORMAT_EXTENSIONS which breaks universe. 2014-05-10 00:42:43 +00:00
Warner Losh
22cac7546c Introduce kern.opts.mk to hold all the options for kernel module
builds. Include this in the right places. Make src.opts.mk optional so
that modules can be built outside of the tree in the ports system.

PR: 189520
2014-05-09 21:11:27 +00:00