Commit Graph

258795 Commits

Author SHA1 Message Date
asomers
8e5e4ff2ad googletest: backport GTEST_SKIP to googletest 1.8.1
This commit backports revisions 00938b2b228f3b70d3d9e51f29a1505bdad43f1e and
59f90a338bce2376b540ee239cf4e269bf6d68ad from googletest's master branch to
our included version of googletest, which is based on 1.8.1.  It adds the
GTEST_SKIP feature, which is very useful for a project like FreeBSD where
some tests depend on particular system configurations.

Obtained from:	github.com/google/googletest
Sponsored by:	The FreeBSD Foundation
2019-03-20 17:11:25 +00:00
asomers
0fdf4eed76 fuse(4): fix a race condition in the tests
Sometimes the fuse daemon doesn't die as soon as its /dev/fuse file
descriptor is closed; it needs to be unmounted first.

Sponsored by:	The FreeBSD Foundation
2019-03-20 16:08:07 +00:00
asomers
b1397d8fd5 fuse(4): remove more debugging printfs
I missed these in r344664.  They're basically useless because they can only
be controlled at compile-time.  Also, de-inline fuse_internal_cache_attrs.
It's big enough to be a regular function, and this way it gets a dtrace FBT
probe.

Sponsored by:	The FreeBSD Foundation
2019-03-19 17:49:15 +00:00
asomers
1b3d8ebb21 fuse(4): build the tests with the new googletest in base
Sponsored by:	The FreeBSD Foundation
2019-03-19 03:10:13 +00:00
asomers
ca3f058a08 MFHead@r345275 2019-03-18 19:21:53 +00:00
asomers
be65471ae9 fuse(4): add tests for some mount options.
This commit adds tests for the default_permissions and push_symlinks_in
mount options.  It doesn't add tests for allow_other, because I'm not sure
how that will interact with Kyua (the test will need to drop privileges).
All of the other mount options are undocumented.

PR:		216391
Sponsored by:	The FreeBSD Foundation
2019-03-18 18:05:19 +00:00
ae
e171491f01 Revert r345274. It appears that not all 32-bit architectures have
necessary CK primitives.
2019-03-18 14:00:19 +00:00
ae
f13ac20eb6 Update NAT64LSN implementation:
o most of data structures and relations were modified to be able support
  large number of translation states. Now each supported protocol can
  use full ports range. Ports groups now are belongs to IPv4 alias
  addresses, not hosts. Each ports group can keep several states chunks.
  This is controlled with new `states_chunks` config option. States
  chunks allow to have several translation states for single alias address
  and port, but for different destination addresses.
o by default all hash tables now use jenkins hash.
o ConcurrencyKit and epoch(9) is used to make NAT64LSN lockless on fast path.
o one NAT64LSN instance now can be used to handle several IPv6 prefixes,
  special prefix "::" value should be used for this purpose when instance
  is created.
o due to modified internal data structures relations, the socket opcode
  that does states listing was changed.

Obtained from:	Yandex LLC
MFC after:	1 month
Sponsored by:	Yandex LLC
2019-03-18 12:59:08 +00:00
gallatin
b9125f9ffc Fix a typo introduced in r344133
The line was misedited to change tt to st instead of
changing ut to st.

The use of st as the denominator in mul64_by_fraction() will lead
to an integer divide fault in the intr proc (the process holding
ithreads) where st will be 0.  This divide by 0 happens after
the total runtime for all ithreads exceeds 76 hours.

Submitted by: bde
2019-03-18 12:41:42 +00:00
vmaffione
c056f1d6c1 netmap: add support for multiple host rings
Some applications forward from/to host rings most or all the
traffic received or sent on a physical interface. In this
cases it is desirable to have more than a pair of RX/TX host
rings, and use multiple threads to speed up forwarding.
This change adds support for multiple host rings. On registering
a netmap port, the user can specify the number of desired receive
and transmit host rings in the nr_host_tx_rings and nr_host_rx_rings
fields of the nmreq_register structure.

MFC after:	2 weeks
2019-03-18 12:22:23 +00:00
ae
93a7173b74 Add NAT64 CLAT implementation as defined in RFC6877.
CLAT is customer-side translator that algorithmically translates 1:1
private IPv4 addresses to global IPv6 addresses, and vice versa.
It is implemented as part of ipfw_nat64 kernel module. When module
is loaded or compiled into the kernel, it registers "nat64clat" external
action. External action named instance can be created using `create`
command and then used in ipfw rules. The create command accepts two
IPv6 prefixes `plat_prefix` and `clat_prefix`. If plat_prefix is ommitted,
IPv6 NAT64 Well-Known prefix 64:ff9b::/96 will be used.

  # ipfw nat64clat CLAT create clat_prefix SRC_PFX plat_prefix DST_PFX
  # ipfw add nat64clat CLAT ip4 from IPv4_PFX to any out
  # ipfw add nat64clat CLAT ip6 from DST_PFX to SRC_PFX in

Obtained from:	Yandex LLC
Submitted by:	Boris N. Lytochkin
MFC after:	1 month
Relnotes:	yes
Sponsored by:	Yandex LLC
2019-03-18 11:44:53 +00:00
ae
2770fa04e1 Add SPDX-License-Identifier and update year in copyright.
MFC after:	1 month
2019-03-18 10:50:32 +00:00
ae
6b7a62da46 Modify struct nat64_config.
Add second IPv6 prefix to generic config structure and rename another
fields to conform to RFC6877. Now it contains two prefixes and length:
PLAT is provider-side translator that translates N:1 global IPv6 addresses
to global IPv4 addresses. CLAT is customer-side translator (XLAT) that
algorithmically translates 1:1 IPv4 addresses to global IPv6 addresses.
Use PLAT prefix in stateless (nat64stl) and stateful (nat64lsn)
translators.

Modify nat64_extract_ip4() and nat64_embed_ip4() functions to accept
prefix length and use plat_plen to specify prefix length.

Retire net.inet.ip.fw.nat64_allow_private sysctl variable.
Add NAT64_ALLOW_PRIVATE flag and use "allow_private" config option to
configure this ability separately for each NAT64 instance.

Obtained from:	Yandex LLC
MFC after:	1 month
Sponsored by:	Yandex LLC
2019-03-18 10:39:14 +00:00
markj
b24a98cb7e Revert r345244 for now.
The code which advances the block number is simplistic and is not
correct when the starting offset is non-zero.  Revert the change until
this is fixed.
2019-03-18 05:03:55 +00:00
avos
c67db9c243 net80211: correct check for SMPS node flags updates
Update node flags when driver supports SMPS, not when it is disabled or
in dynamic mode ((iv_htcaps & HTCAP_SMPS) != 0).

Checked with RTL8188EE (1T1R), STA mode - 'smps' word should disappear
from 'ifconfig wlan0' output.

MFC after:	2 weeks
2019-03-18 02:40:22 +00:00
grog
8b17fbc25c Change date of Canberra Day, now on second Monday in March.
https://en.wikipedia.org/wiki/Canberra_Day
2019-03-17 22:26:50 +00:00
kib
ea004a70f9 i386: improve detection of the fast page fault assist.
In particular, check that we are assisting the page fault from kernel mode.

Sponsored by:	The FreeBSD Foundation
MFC after:	3 days
2019-03-17 18:31:48 +00:00
markj
72039f2089 Fix the gcc build (-Wstrict-prototypes) after r345244.
Reported by:	jenkins
MFC with:	r345244
2019-03-17 18:06:13 +00:00
markj
3139ccc80b Optimize lseek(SEEK_DATA) on UFS.
The old implementation, at the VFS layer, would map the entire range of
logical blocks between the starting offset and the first data block
following that offset.  With large sparse files this is very
inefficient.  The VFS currently doesn't provide an interface to improve
upon the current implementation in a generic way.

Add ufs_bmap_seekdata(), which uses the obvious algorithm of scanning
indirect blocks to look for data blocks.  Use it instead of
vn_bmap_seekhole() to implement SEEK_DATA.

Reviewed by:	kib, mckusick
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D19598
2019-03-17 17:34:06 +00:00
dim
ad625eb486 Explicitly link libomp.so against -lpthread, as it depends on pthread
functionality.  This should make example OpenMP programs work out of the
box.

Reported by:	jbeich
PR:		236062, 236581
MFC after:	1 month
X-MFC-With:	r344779
2019-03-17 11:27:27 +00:00
bz
8f6fafa869 Fix legacy IP autoconfiguration.
It seems my subconcious plan in r345088 to not only prefer IPv6 autoconf
but to also slowly deteriorate legacy IP auto-configuration was uncovered
way too early.

In case IPv6 is a thing yet ipv6_autoconfif was not true, we would not
bring up the interface yet tell the follow-up DHCPv4 configuration in
ifconfig_up() that we did.  So unless you were doing SYNCDHCP or IPv6
you would not get legacy-IP DHCPv4 configuration.

I see multiple problems here: (a) people not yet using IPv6 (obviously a
problem), and (b) the dhclient startup script not running dhclient in
that case despite configured to do so (needs to be investigated seperately).

Reported by:	Pawel Biernacki (pawel.biernacki gmail.com)
Tested by:	Pawel Biernacki
Differential Revision:	https://reviews.freebsd.org/D19488
Pointyhat to:	bz (not sure if it is for breaking or
		for letting them notice it so easily)
2019-03-17 09:31:09 +00:00
jhibbits
985f00a61c fdt: Explicitly mark fdt_slicer as dependent on geom_flashmap
Without this dependency relationship, the linker doesn't find the
flash_register_slicer() function, so kldload fails to load fdt_slicer.ko.

Discussed with:	ian@
2019-03-17 04:33:17 +00:00
wosch
310a121be6 `make buildkernel' should display the build time in seconds
PR:		224433
Approved by:	cem
Differential Revision:	https://reviews.freebsd.org/D13910
2019-03-16 20:02:57 +00:00
dim
9bc552afe9 Disable lib/libomp build for the 32-bit part of amd64 buildworld, as it
is not supported for that target.

Reported by:	Michael Butler <imb@protected-networks.net>
PR:		236062
MFC after:	1 month
X-MFC-With:	r344779
2019-03-16 17:55:22 +00:00
dim
4c7b570576 Connect lib/libomp to the build.
* Set MK_OPENMP to yes by default only on amd64, for now.
* Bump __FreeBSD_version to signal this addition.
* Ensure gcc's conflicting omp.h is not installed if MK_OPENMP is yes.
* Update OptionalObsoleteFiles.inc to cope with the conflicting omp.h.
* Regenerate src.conf(5) with new WITH/WITHOUT fragments.

Relnotes:	yes
PR:		236062
MFC after:	1 month
X-MFC-With:	r344779
2019-03-16 15:45:15 +00:00
dim
ac90eb90b3 Add lib/libomp, with a Makefile, and generated configuration headers.
Not connected to the main build yet, as there is still the issue of the
GNU omp.h header conflicting with the LLVM one.  (That is, if MK_GCC is
enabled.)

PR:		236062
MFC after:	1 month
X-MFC-With:	r344779
2019-03-16 15:01:36 +00:00
dim
4ccae8165c Add openmp __kmp_gettid() wrapper, using pthread_getthreadid_np(3).
This has also been submitted upstream.

PR:           236062
MFC after:    1 month
X-MFC-With:   r344779
2019-03-16 13:45:14 +00:00
dim
efddb4ed9b Merge openmp release_80 branch r356034 (effectively, 8.0.0 rc5).
PR:		236062
MFC after:	1 month
X-MFC-With:	r344779
2019-03-16 13:43:07 +00:00
dim
69c3de6092 Add LLVM openmp trunk r351319 (just before the release_80 branch point)
to contrib/llvm.  This is not yet connected to the build, the glue for
that will come in a follow-up commit.

PR:		236062
MFC after:	1 month
X-MFC-With:	r344779
2019-03-16 13:40:27 +00:00
dim
2b7278f31e Update maintainers for libunwind and lldb. 2019-03-16 13:26:42 +00:00
kib
4ed72fff9d proccontrol(1): Add kpti control mode.
Requested by:	jhb
Reviewed by:	jhb, markj (previous version)
Tested by:	pho
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D19514
2019-03-16 11:46:48 +00:00
kib
71140c5be4 amd64 KPTI: add control from procctl(2).
Add the infrastructure to allow MD procctl(2) commands, and use it to
introduce amd64 PTI control and reporting.  PTI mode cannot be
modified for existing pmap, the knob controls PTI of the new vmspace
created on exec.

Requested by:	jhb
Reviewed by:	jhb, markj (previous version)
Tested by:	pho
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D19514
2019-03-16 11:44:33 +00:00
kib
9b5673f116 amd64: Add md process flags and first P_MD_PTI flag.
PTI mode for the process pmap on exec is activated iff P_MD_PTI is set.

On exec, the existing vmspace can be reused only if pti mode of the
pmap matches the P_MD_PTI flag of the process.  Add MD
cpu_exec_vmspace_reuse() callback for exec_new_vmspace() which can
vetoed reuse of the existing vmspace.

MFC note: md_flags change struct proc KBI.

Reviewed by:	jhb, markj
Tested by:	pho
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D19514
2019-03-16 11:31:01 +00:00
kib
ac023f52e4 amd64: fix switching to the pmap with pti disabled.
When the pmap with pti disabled (i.e. pm_ucr3 == PMAP_NO_CR3) is
activated, tss.rsp0 was not updated.  Any interrupt that happen before
next context switch would use pti trampoline stack for hardware frame
but fault and interrupt handlers are not prepared to this.  Correctly
update tss.rsp0 for both PMAP_NO_CR3 and pti pmaps.

Note that this case, pti = 1 but pmap->pm_ucr3 == PMAP_NO_CR3 is not
used at the moment.

Reviewed by:	markj
Tested by:	pho
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D19514
2019-03-16 11:16:09 +00:00
kib
a83c13f174 amd64: rewrite cpu_switch.S fragment to reload tss.rsp0 on context switch.
New code avoids jumps.

Reviewed by:	markj
Tested by:	pho
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D19514
2019-03-16 11:12:02 +00:00
kp
de68b872a7 pf: Rename pfsync bucket lock
Previously the main pfsync lock and the bucket locks shared the same name.
This lead to spurious warnings from WITNESS like this:

    acquiring duplicate lock of same type: "pfsync"
     1st pfsync @ /usr/src/sys/netpfil/pf/if_pfsync.c:1402
     2nd pfsync @ /usr/src/sys/netpfil/pf/if_pfsync.c:1429

It's perfectly okay to grab both the main pfsync lock and a bucket lock at the
same time.

We don't need different names for each bucket lock, because we should always
only acquire a single one of those at a time.

MFC after:	1 week
2019-03-16 10:14:03 +00:00
jmallett
d2e46ebc0d Remove empty directory left by r342255. 2019-03-16 06:12:28 +00:00
jmallett
cd45d9e953 Remove obsolete wrappers for 64-bit loads/stores which were only used by the
removed (r342255) SiByte port.

Reviewed by:	imp
2019-03-16 06:09:45 +00:00
gonzo
67c1ff30c9 Fix build for KLD dyn_sysctl example
Looks like the example was broken by change of SYSCTL_STATIC_CHILDREN
definition in r267992. Fix build by switching to using SYSCTL_ADD_ROOT_NODE

PR:		236139
Submitted by:	Andrew Reiter <arr@watson.org>
MFC after:	1 week
2019-03-16 04:24:02 +00:00
ngie
1e528cab57 Remove duplicate ${PACKAGE}FILES+= cleanup.ksh line
This mutes the duplicate target warning emitted via bsd.files.mk each build.

MFC after:	1 week
Reviewed by:	asomers
Approved by:	emaste (mentor)
Differential Revision: https://reviews.freebsd.org/D19603
2019-03-16 03:37:47 +00:00
asomers
1f9aeb4e2f fuse(4): add tests for the FOPEN_KEEP_CACHE option
PR:		236560
Sponsored by:	The FreeBSD Foundation
2019-03-15 22:47:20 +00:00
cem
4d504c5762 stack(9): Drop unused API mode and comment that referenced it
Reviewed by:	markj
Differential Revision:	https://reviews.freebsd.org/D19601
2019-03-15 22:39:55 +00:00
ngie
192cdbcab2 Integrate cddl/usr.sbin/zfds/tests into the FreeBSD test suite
This change integrates the unit tests for zfsd into the test suite using the
integration method described in r345203.

This change removes the `LOCALBASE` includes added for the port version of
googlemock/googletest, as well as unnecessary `LIBADD`/`DPADD` and `CXXFLAGS`
defines, which are included in the `GTEST_CXXFLAGS` variable, as part of
r345203.

Reviewed by:	asomers
Approved by:	emaste (mentor)
MFC after:	2 months
MFC with:	r345203
Differential Revision: https://reviews.freebsd.org/D19552
2019-03-15 21:49:19 +00:00
ngie
4b24254622 Regenerate src.conf(5) after r345203 (MK_GOOGLETEST addition)
MFC after:	2 months
MFC with:	r345203
Approved by:	emaste (mentor; implicit: https://reviews.freebsd.org/D19551)
2019-03-15 21:46:54 +00:00
ngie
3c7fe6eece Initial googlemock/googletest integration into the build/FreeBSD test suite
This initial integration takes googlemock/googletest release 1.8.1, integrates
the library, tests, and sample unit tests into the build.

googlemock/googletest's inclusion is optionally available via `MK_GOOGLETEST`.
`MK_GOOGLETEST` is dependent on `MK_TESTS` and is enabled by default when
built with a C++11 capable toolchain.

Google tests can be specified via the `GTESTS` variable, which, in comparison
with the other test drivers, is more simplified/streamlined, as Googletest only
supports C++ tests; not raw C or shell tests (C tests can be written in C++
using the standard embedding methods).

No dependent libraries are assumed for the tests. One must specify `gmock`,
`gmock_main`, `gtest`, or `gtest_main`, via `LIBADD` for the program.

More information about googlemock and googletest can be found on the
Googletest [project page](https://github.com/google/googletest), and the
[GoogleMock](https://github.com/google/googletest/blob/v1.8.x/googlemock/docs/Documentation.md)
and
[GoogleTest](https://github.com/google/googletest/tree/v1.8.x/googletest/docs)
docs.

These tests are originally integrated into the build as plain driver tests, but
will be natively integrated into Kyua in a later version.

Known issues/Errata:
* [WhenDynamicCastToTest.AmbiguousCast fails on FreeBSD](https://github.com/google/googletest/issues/2172)

Reviewed by:	asomers
Approved by:	emaste (mentor)
MFC after:	2 months
Differential Revision: https://reviews.freebsd.org/D19551
2019-03-15 21:43:52 +00:00
asomers
17edbdf9f7 VOP_INACTIVE(9): clarify wording
Reviewed by:	kib, 0mp
MFC after: 	2 weeks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D19596
2019-03-15 21:11:47 +00:00
asomers
7048cf91dc fuse(4): add tests for the FUSE_ASYNC_READ option
Sponsored by:	The FreeBSD Foundation
2019-03-15 20:16:35 +00:00
mav
892774d4dc MFV r336930: 9284 arc_reclaim_thread has 2 jobs
`arc_reclaim_thread()` calls `arc_adjust()` after calling
`arc_kmem_reap_now()`; `arc_adjust()` signals `arc_get_data_buf()` to
indicate that we may no longer be `arc_is_overflowing()`.

The problem is, `arc_kmem_reap_now()` can take several seconds to
complete, has no impact on `arc_is_overflowing()`, but due to how the
code is structured, can impact how long the ARC will remain in the
`arc_is_overflowing()` state.

The fix is to use seperate threads to:

1. keep `arc_size` under `arc_c`, by calling `arc_adjust()`, which
    improves `arc_is_overflowing()`

2. keep enough free memory in the system, by calling
 `arc_kmem_reap_now()` plus `arc_shrink()`, which improves
 `arc_available_memory()`.

illumos/illumos-gate@de753e34f9

Reviewed by: Matt Ahrens <mahrens@delphix.com>
Reviewed by: Serapheim Dimitropoulos <serapheim@delphix.com>
Reviewed by: Pavel Zakharov <pavel.zakharov@delphix.com>
Reviewed by: Dan Kimmel <dan.kimmel@delphix.com>
Reviewed by: Paul Dagnelie <pcd@delphix.com>
Reviewed by: Dan McDonald <danmcd@joyent.com>
Reviewed by: Tim Kordas <tim.kordas@joyent.com>
Approved by: Garrett D'Amore <garrett@damore.org>
Author: Brad Lewis <brad.lewis@delphix.com>
2019-03-15 18:59:04 +00:00
asomers
197621b57e VOP_INACTIVE(9): clarify wording
Reviewed by:	kib, 0mp
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D19596
2019-03-15 18:50:36 +00:00
glebius
7c045aad76 Deanonymize thread and proc state enums, so that a userland app can
use them without redefining the value names. New clang no longer
allows to redefine a enum value name to the same value.

Bump __FreeBSD_version, since ports depend on that.

Discussed with:	jhb
2019-03-15 18:18:05 +00:00