Commit Graph

266759 Commits

Author SHA1 Message Date
Kristof Provost
be70c7a50d libpfctl: migrate to DIOCGETSTATESV2
Stop using the *NV version to retrieve states, as its performance is
unacceptably bad.

For 1,000,000 states the nvlist version needed ~100 seconds to retrieve
the states, the new version needs ~3 seconds.

Reviewed by:	mjg
MFC after:	1 week
Sponsored by:	Rubicon Communications, LLC ("Netgate")
Differential Revision:	https://reviews.freebsd.org/D31098
2021-07-09 10:29:57 +02:00
Kristof Provost
c6bf20a2a4 pf: add DIOCGETSTATESV2
Add a new version of the DIOCGETSTATES call, which extends the struct to
include the original interface information.

MFC after:	1 week
Sponsored by:	Rubicon Communications, LLC ("Netgate")
Differential Revision:	https://reviews.freebsd.org/D31097
2021-07-09 10:29:53 +02:00
Warner Losh
a65fe39d54 UPDATING: fix typo
The correct version is 20210215, not 20120215 :(

Reported by:		Yasuhiro Kimura
Fixes:			3f7b23177f
Sponsored by:		Netflix
2021-07-08 23:42:15 -06:00
Warner Losh
3f7b23177f UPDATING: Note awk upgraded to upstream one-true-awk 20210215.
The updating entry can be used for release notes. I'll merge this to
stable/12 and stable/13 to make sure it gets into thier release notes.

Relnotes:		yes
Sponsored by:		Netflix
2021-07-08 23:16:06 -06:00
Warner Losh
f68a53dba9 awk: Reduce diffs with upstream to almost nothing.
In the merge of 20210215, I left two merge conflicts #if 0'd by mistake
to check later rather than resolve them as part of the merge.  This code
turns out to be from the original one-true-awk import and not FreeBSD
specific, so remove them.

Remove a extra definition of HAT.

Remove a stylistic change that also appears to be a mismerge along the
way.

Remove FREEBSD-upgrade. Nobody has updated it since the original 2007
cvs import. It talks about old CVS branches that never made it into svn,
let alone git. New imports will follow the standard practices now, so
there's nothing left to document.

Move README to README.md and copy the README.md from upstream over.

This leaves just the $FreeBSD$ lines (which remain for the stable/12
merge) and the strcoll part of ru@'s r201989/d98dd8e5f94c as the only
diffs with upstream. FreeBSD also still has its own man page, which I
don't plan on changing. Once this commit is merged to stable/12, I plan
no further merges to stable/12. Sometime after that I'll remove the
$FreeBSD$ lines to reduce the diffs even more (though i want to make
sure plans won't change first). I also plan to talk to upstream about
this change...

MFC After:		2 weeks
Sponsored by:		Netflix
2021-07-08 23:05:13 -06:00
Rick Macklem
bd597b8149 UPDATING: Add an entry for commit 1e0a518d65 2021-07-08 18:30:24 -07:00
Rick Macklem
1e0a518d65 nfscl: Add a Linux compatible "nconnect" mount option
Linux has had an "nconnect" NFS mount option for some time.
It specifies that N (up to 16) TCP connections are to created for a mount,
instead of just one TCP connection.

A discussion on freebsd-net@ indicated that this could improve
client<-->server network bandwidth, if either the client or server
have one of the following:
- multiple network ports aggregated to-gether with lagg/lacp.
- a fast NIC that is using multiple queues
It does result in using more IP port#s and might increase server
peak load for a client.

One difference from the Linux implementation is that this implementation
uses the first TCP connection for all RPCs composed of small messages
and uses the additional TCP connections for RPCs that normally have
large messages (Read/Readdir/Write).  The Linux implementation spreads
all RPCs across all TCP connections in a round robin fashion, whereas
this implementation spreads Read/Readdir/Write across the additional
TCP connections in a round robin fashion.

Reviewed by:	markj
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D30970
2021-07-08 17:39:04 -07:00
Warner Losh
25a66f1fb1 nanobsd: remove sparc64 embedded example
Remove the qemu sparc64 example. It was only ever compile tested since
qemu had issues booting FreeBSD/sparc64. Also remove obsolete info about
armv5 configs removed long ago.

Sponsored by:		Netflix
2021-07-08 17:56:54 -06:00
Warner Losh
5549c6a62f devmatch: don't announce autoloading so much
devmatch rc script would announce it was loading a module multiple
times. It used kldload -n so it really wasn't loading it that many
times, but the message is confusing. Use kldstat to see if we need to
load the module before saying we do. This fixes the vast majority of the
problems. It may be possible to race devmatch with a user invocation and
devd, though quite hard. In that case we'll announce things twice, but
still only load it once. No attempt is made to fix this.

PR:			232782
MFC After:		2 weeks
Sponsored by:		Netflix
2021-07-08 15:22:22 -06:00
Warner Losh
b29ebb9c65 devmatch: Be tolerant of .ko being present.
We document that we did not need .ko on the module names in
devmatch_blocklist, but we really needed them. Keep the documentation
the same, but strip the .ko when we need to use the names so you can
specify either.

PR:			256240
MFC After:		2 weeks
Sponsored by:		Netflix
2021-07-08 15:22:22 -06:00
Bjoern A. Zeeb
bf3a385217 fan53555: attach to build and switch from syr827
Rather than extending syr827 for syr828 (as initially done in D31103)
switch to the Fairchild Semiconductor Corporation fan53555 implementation
which is in-tree but was not attached to the build.  The fan53555
implementation also supports syr827/syr8278 already. [1]
Update NOTES and the arm64 GENERIC configuration for the switch.
syr827 for now stays in the tree but is not used by any
kernel configuration.

Suggested by:	mmel [1]
Reviewed by:	mmel, manu
Differential Revision: https://reviews.freebsd.org/D31112
2021-07-08 20:17:45 +00:00
Luiz Otavio O Souza
c5dd8bac0b dummynet: reduce console spam
Only print this warning when boot verbose is enabled.
This can get pretty annoying (and useless) in some systems.

Reviewed by:	kp
MFC after:	1 week
Sponsored by:	Rubicon Communications, LLC ("Netgate")
2021-07-08 20:25:05 +02:00
Kristof Provost
3464105282 pf: pf_killstates() never fails, so remove the return value
Suggested by:	mjg
MFC after:	1 week
Sponsored by:	Rubicon Communications, LLC ("Netgate")
2021-07-08 18:56:29 +02:00
Alan Somers
61631b24a1 iostat: fix rounding errors in iostat -x
Better to round numbers instead of flooring them.

MFC after:	3 weeks
Sponsored by:	Axcient
2021-07-08 10:16:32 -06:00
Thomas Steen Rasmussen
28f47a199c pf: fallback if $pf_rules fails to load
Support loading a default pf ruleset in case of invalid pf.conf.

If no pf rules are loaded pf will pass/allow all traffic, assuming the
kernel is compiled without PF_DEFAULT_TO_DROP, as is the case in
GENERIC.

In other words: if there's a typo in the main pf_rules we would allow
all traffic. The new default rules minimise the impact of this.

If $pf_program (i.e. pfctl) fails to set $pf_fules and
$pf_fallback_rules_enable is YES we will load $pf_fallback_rules_file if
set, or $pf_fallback_rules.

$pf_fallback_rules can include multiple rules, for example to permit
traffic on a management interface.

$pf_fallback_rules_enable defaults to "NO", preserving historic behaviour.

man page changes by ceri@.

PR:		256410
Reviewed by:	donner, kp
Sponsored by:	semaphor.dk
Differential Revision:	https://reviews.freebsd.org/D30791
2021-07-08 14:22:04 +02:00
Mateusz Guzik
19d6e29b87 pf: add pf_find_state_all_exists
Reviewed by:	kp
Sponsored by:	Rubicon Communications, LLC ("Netgate")
2021-07-08 14:00:55 +00:00
Mateusz Guzik
1e67e3109d setkey: drop an unused argument from postproc 2021-07-08 14:00:55 +00:00
Mateusz Guzik
b29817874d setkey: fix several warnings popping up when compiling without WARNS=1 2021-07-08 14:00:55 +00:00
Mateusz Guzik
e0c0612d9b setkey: ansify 2021-07-08 14:00:55 +00:00
Andrew Turner
c0edde3021 Fix the name of the arm64 SCTLR_E0E register
The character between the E's was the letter O, however in the Arm
Documentation and XML the character is the number 0 (zero).

Sponsored by:	The FreeBSD Foundation
2021-07-07 23:18:04 +00:00
Randall Stewart
7312e4e5cf tcp: Fix 32 bit platform breakage
This fixes the incorrect use of a sysctl add to u64. It
was for a useconds time, but on 32 bit platforms its
not a u64. Instead use the long directive.

Reviewed by: tuexen
Sponsored by: Netflix Inc.
Differential Revision: https://reviews.freebsd.org/D31107
2021-07-08 08:16:45 -04:00
Kristof Provost
fa96701c8a pf: Handle errors returned by pf_killstates()
Happily this wasn't a real bug, because pf_killstates() never fails, but
we should check the return value anyway, in case it does ever start
returning errors.

Reported by:	clang --analyze
MFC after:	1 week
Sponsored by:	Rubicon Communications, LLC ("Netgate")
2021-07-08 10:32:42 +02:00
Kristof Provost
8cceacc0f1 pf: Remove unneeded NULL check
pidx is never NULL, and is used unconditionally later on in the
function.
Add an assertion, as documentation for the requirement to provide an idx
pointer.

Reported by:	clang --analyze
MFC after:	1 week
Sponsored by:	Rubicon Communications, LLC ("Netgate")
2021-07-08 10:32:33 +02:00
Kristof Provost
211cddf9e3 pf: rename pf_state to pf_kstate
Indicate that this is a kernel-only structure, and make it easier to
distinguish from others used to communicate with userspace.

Reviewed by:	mjg
MFC after:	1 week
Sponsored by:	Rubicon Communications, LLC ("Netgate")
Differential Revision:	https://reviews.freebsd.org/D31096
2021-07-08 10:31:43 +02:00
Michal Meloun
e88c3b1b02 intrng: remove now redundant shadow variable.
Should not be a functional change.

Submitted by: 	ehem_freebsd@m5p.com
Discussed in:	https://reviews.freebsd.org/D29310
MFC after:	4 weeks
2021-07-08 08:46:41 +02:00
Michal Meloun
a49f208d94 intrng: Releasing interrupt source should clear interrupt table full state.
The first release of an interrupt in a situation where the interrupt table
is full should schedule a full table check the next time an interrupt is
allocated. A full check is necessary to ensure maximum separation between
the order of allocation and the order of release.

Submitted by:	ehem_freebsd@m5p.com (initial version)
Discussed in:	https://reviews.freebsd.org/D29310
MFC after:	4 weeks
2021-07-08 08:16:46 +02:00
Warner Losh
f39dd6a978 one-true-awk: import 20210221 (1e4bc42c53a1) which fixes a number of bugs
Import the latest bsd-features branch of the one-true-awk upstream:

o Move to bison for $YACC
o Set close-on-exec flag for file and pipe redirects that aren't std*
o lots of little fixes to modernize ocde base
o free sval member before setting it
o fix a bug where a{0,3} could match aaaa
o pull in systime and strftime from NetBSD awk
o pull in fixes from {Net,Free,Open}BSD (normalized our code with them)
o add BSD extensions and, or, xor, compl, lsheift, rshift (mostly a nop)

Also revert a few of the trivial FreeBSD changes that were done slightly
differently in the upstreaming process. Also, our PR database may have
been mined by upstream for these fixes, and Mikolaj Golub may deserve
credit for some of the fixes in this update.

Suggested by:		Mikolaj Golub <to.my.trociny@gmail.com>
PR:			143363,143365,143368,143369,143373,143375,214782
Sponsored by:		Netflix
2021-07-07 19:25:43 -06:00
Warner Losh
746b7396bb one-true-awk: import 20210221 (1e4bc42c53a1) which fixes a number of bugs
Import the latest bsd-features branch of the one-true-awk upstream:

o Move to bison for $YACC
o Set close-on-exec flag for file and pipe redirects that aren't std*
o lots of little fixes to modernize ocde base
o free sval member before setting it
o fix a bug where a{0,3} could match aaaa
o pull in systime and strftime from NetBSD awk
o pull in fixes from {Net,Free,Open}BSD
o add BSD extensions and, or, xor, compl, lsheift, rshift

Sponsored by:		Netflix
2021-07-07 17:09:41 -06:00
Martin Matuska
7cd22ac434 zfs: merge openzfs/zfs@bdd11cbb9 (master) into main
Notable upstream pull request merges:
  #12274 Optimize txg_kick() process
  #12281 Move gethrtime() calls out of vdev queue lock
  #12287 Remove refcount from spa_config_*(
  #12289 Compact dbuf/buf hashes and lock arrays
  #12290 Remove avl_size field from struct avl_tree
  #12294 Upstream: dmu_zfetch_stream_fini leaks refcount
  #12295 Fix abd leak, kmem_free correct size of abd_t
  #12328 FreeBSD: Hardcode abd_chunk_size to PAGE_SIZE

Obtained from:	OpenZFS
OpenZFS commit:	bdd11cbb90
2021-07-07 23:31:52 +02:00
Helge Oldach
f68e3ea831 devmatch: defer until after kld
devmatch loads a number of things automatically. Allow the list of
things to load to happen first in case those drivers affect what would
be loaded. Normally, this will produce the same results, but there's
some special cases that may not when drivers are loaded that report
other drivers missing, like virtio_pci.

PR:		253287
Reviewed by:	imp
MFC After:	2 weeks
2021-07-07 14:48:40 -06:00
Fernando Apesteguía
f6043a6721 ObsoleteFiles.inc: Remove manpages from OLD_FILES
In 0a0f748641 sconfig.8.gz and apmd.8.gz
were moved back to the parent directory share/man/man8.

This conflicts with a previous entry in OLD_FILES so running 'make
delete-old' would delete the new installed files.

Reported by:	marklmi@yahoo.com
Approved by:	emaste@
Fixes:	0a0f748641 - Build manpages for all architectures
MFH:	4 weeks
Differential Revision:	https://reviews.freebsd.org/D31092
2021-07-07 20:09:02 +02:00
Alan Somers
3874c0abb0 [skip ci] correct a few SPDX license tags
These were all incorrectly labeled as 2-clause BSD licenses by a
semi-automated process, when in fact they are 3-clause.

Discussed with:	pfg, imp
MFC after:	2 weeks
Sponsored by:	Axcient
2021-07-07 13:52:20 -06:00
Andrew Gallatin
0756bdf19c ktls: make ktls_disable_ifnet() shim static
A user reported that when compiling without KERN_TLS, and
with -O0, the kernel failed to link due to ktls_disable_ifnet()
being undefined.   Making the shim static works around this issue.

Reported by: Gary Jennejohn
Sponsored by: Netflix
2021-07-07 15:08:13 -04:00
Alan Cox
0add3c9945 arm64: Simplify fcmpset failure in pmap_promote_l2()
When the initial fcmpset in pmap_promote_l2() fails, there is no need
to repeat the check for the physical address being 2MB aligned or for
the accessed bit being set.  While the pmap is locked the hardware can
only transition the accessed bit from 0 to 1, and we have already
determined that it is 1 when the fcmpset fails.

MFC after:	1 week
2021-07-07 13:34:11 -05:00
Martin Matuska
48b4fe0503 zfs: attach zpool_influxdb to build
From the zpool_influxdb.8 manual page:
  zpool_influxdb produces InfluxDB-line-protocol-compatible metrics from
  zpools.  Like the zpool command, zpool_influxdb reads the current pool
  status and statistics.  Unlike the zpool command which is intended for
  humans, zpool_influxdb formats the output in the InfluxDB line protocol.
  The expected use is as a plugin to a metrics collector or aggregator,
  such as Telegraf.

zpool_influxdb is installed into /usr/libexec/zfs/

Differential revision:	https://reviews.freebsd.org/D31094
MFC after:	3 days
2021-07-07 20:15:12 +02:00
Andrew Gallatin
b1e806c0ed tcp: fix alternate stack build with LINT-NO{INET,INET6,IP}
When fixing another bug, I noticed that the alternate
TCP stacks do not build when various combinations of
ipv4 and ipv6 are disabled.

Reviewed by:	rrs, tuexen
Differential Revision:	https://reviews.freebsd.org/D31094
Sponsored by: Netflix
2021-07-07 13:02:08 -04:00
Andrew Gallatin
4150a5a87e ktls: fix NOINET build
Reported by: mjguzik
Sponsored by: Netflix
2021-07-07 10:40:02 -04:00
Randall Stewart
d7955cc0ff tcp: HPTS performance enhancements
HPTS drives both rack and bbr, and yet there have been many complaints
about performance. This bit of work restructures hpts to help reduce CPU
overhead. It does this by now instead of relying on the timer/callout to
drive it instead use user return from a system call as well as lro flushes
to drive hpts. The timer becomes a backstop that dynamically adjusts
based on how "late" we are.

Reviewed by: tuexen, glebius
Sponsored by: Netflix Inc.
Differential Revision: https://reviews.freebsd.org/D31083
2021-07-07 07:22:35 -04:00
Konstantin Belousov
747a6b7ace cloudabi and linux ABIs: do not call umtx_thread_cleanup() from thr_exit syscall
These ABIs do not use umtx at all, so there is nothing to clean.
Cloudabi references to umtx keys do not require any cleanups anyway.

Requested by:	dchagin
Reviewed by:	dchagin, markj
Sponsored by:	The FreeBSD Foundation
MFC after:	2 weeks
Differential revision:	https://reviews.freebsd.org/D30987
2021-07-07 14:12:14 +03:00
Konstantin Belousov
28a66fc3da Do not call FreeBSD-ABI specific code for all ABIs
Use sysentvec hooks to only call umtx_thread_exit/umtx_exec, which handle
robust mutexes, for native FreeBSD ABI.  Similarly, there is no sense
in calling sigfastblock_clear() for non-native ABIs.

Requested by:	dchagin
Reviewed by:	dchagin, markj (previous version)
Sponsored by:	The FreeBSD Foundation
MFC after:	2 weeks
Differential revision:	https://reviews.freebsd.org/D30987
2021-07-07 14:12:07 +03:00
Konstantin Belousov
55976ce11a Move sv_onexit() sysentvec hook slightly later
after itimers are stopped.  This makes it more usable for e.g. native FreeBSD
ABI sysentvecs.

Reviewed by:	dchagin, markj
Sponsored by:	The FreeBSD Foundation
MFC after:	2 weeks
Differential revision:	https://reviews.freebsd.org/D30987
2021-07-07 14:12:07 +03:00
Konstantin Belousov
71ab344524 Add sv_onexec_old() sysent hook for exec event
Unlike sv_onexec(), it is called from the old (pre-exec) sysentvec structure.
The old vmspace for the process is still intact during the call.

Reviewed by:	dchagin, markj
Sponsored by:	The FreeBSD Foundation
MFC after:	2 weeks
Differential revision:	https://reviews.freebsd.org/D30987
2021-07-07 14:12:07 +03:00
Mateusz Guzik
edcf1054d3 cxgb: use m_gethdr_raw
Sponsored by:	Rubicon Communications, LLC ("Netgate")
2021-07-07 11:05:46 +00:00
Mateusz Guzik
a56888534d iflib: use m_gethdr_raw
Reviewed by:	gallatin
Sponsored by:	Rubicon Communications, LLC ("Netgate")
Differential Revision:	https://reviews.freebsd.org/D31081
2021-07-07 11:05:46 +00:00
Mateusz Guzik
c2c34ee540 mbuf: add m_get_raw and m_gethdr_raw
The intent is to eliminate the MT_NOINIT flag and consequently a branch
from the constructor.

Reviewed by:	gallatin
Sponsored by:	Rubicon Communications, LLC ("Netgate")
Differential Revision:	https://reviews.freebsd.org/D31080
2021-07-07 11:05:46 +00:00
Mateusz Guzik
0a718a6e6e mbuf: replace all direct uma_zfree(zone_mbuf) calls with m_free_raw
Reviewed by:	donner
Sponsored by:	Rubicon Communications, LLC ("Netgate")
Differential Revision:	https://reviews.freebsd.org/D31082
2021-07-07 11:05:46 +00:00
Alexander Motin
bdd11cbb90
FreeBSD: Hardcode abd_chunk_size to PAGE_SIZE
It makes no sense to set it below PAGE_SIZE, since it increases all
overheads and makes returning memory to OS problematic.  It makes no
sense to set it above PAGE_SIZE, since such allocations and especially
frees are too expensive and cause KVA fragmentation to benefit from
fewer chunks.  After that it makes no sense to keep more complicated
math here.

What may have sense though is just a tunable border between linear and
scatter ABDs, previously also controlled by this tunable.  Retain that
functionality by taking abd_scatter_min_size tunable from Linux, just
with different default value.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Brian Atkinson <batkinson@lanl.gov>
Reviewed-by: Ryan Moeller <ryan@iXsystems.com>
Signed-off-by: Alexander Motin <mav@FreeBSD.org>
Closes #12328
2021-07-06 17:39:23 -07:00
Alexander Motin
97752ba22a
Move gethrtime() calls out of vdev queue lock
This dramatically reduces the lock contention on systems with slower
(non-TSC) timecounters.  With TSC the difference is minimal, but since
this lock is pretty congested, any improvement counts.  Plus I don't
see any reason to do it under the lock other than the latency of the
lock itself, which this change actually reduces.

Reviewed-by: Matthew Ahrens <mahrens@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Alexander Motin <mav@FreeBSD.org>
Sponsored-By: iXsystems, Inc.
Closes #12281
2021-07-06 14:38:00 -07:00
Andrew Turner
a7b05eb16c Sync the arm64 special registers with the Armv8.5 XML
Add the missing macros and decode all the fields as described in the
Arm Architecture System Registers XML corresponding to Armv8.5.

Sponsored by:	The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D30983
2021-07-06 20:46:55 +00:00
Edward Tomasz Napierala
6f147a0734 cam: enable kern.cam.ada.enable_uma_ccbs by default
This makes the ada(4) driver use UMA for its CCBs.  While it's
da(4) counterpart needs some more testing, this one seems to be
safe now.

Please let me know via email if you notice any suspicious kernel
messages,

Reviewed By:	imp
Sponsored by:	NetApp, Inc.
Sponsored by:	Klara, Inc.
Differential Revision:	https://reviews.freebsd.org/D30567
2021-07-07 09:40:34 +01:00