The last two drivers that required sppp are cp(4) and ce(4).
These devices are still produced and can be purchased
at Cronyx <http://cronyx.ru/hardware/wan.html>.
Since Roman Kurakin <rik@FreeBSD.org> has quit them, they no
longer support FreeBSD officially. Later they have dropped
support for Linux drivers to. As of mid-2020 they don't even
have a developer to maintain their Windows driver. However,
their support verbally told me that they could provide aid to
a FreeBSD developer with documentaion in case if there appears
a new customer for their devices.
These drivers have a feature to not use sppp(4) and create an
interface, but instead expose the device as netgraph(4) node.
Then, you can attach ng_ppp(4) with help of ports/net/mpd5 on
top of the node and get your synchronous PPP. Alternatively
you can attach ng_frame_relay(4) or ng_cisco(4) for HDLC.
Actually, last time I used cp(4) back in 2004, using netgraph(4)
instead of sppp(4) was already the right way to do.
Thus, remove the sppp(4) related part of the drivers and enable
by default the negraph(4) part. Further maintenance of these
drivers in the tree shouldn't be a big deal.
While doing that, remove some cruft and enable cp(4) compilation
on amd64. The ce(4) for some unknown reason marks its internal
DDK functions with __attribute__ fastcall, which most likely is
safe to remove, but without hardware I'm not going to do that, so
ce(4) remains i386-only.
Reviewed by: emaste, imp, donner
Differential Revision: https://reviews.freebsd.org/D32590
See also: https://reviews.freebsd.org/D23928
The ng_h4 module was disconnected 13 years ago when the tty later was
locked by Ed. It completely fails to compile, and has a number of false
positives for Giant use. Remove it for lack of interest. Bluetooth has
largely (completely?) moved on from bluetooth over UART transport.
OK'd by: emax
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D31846
Create netgraph nodes for bridge interfaces when the ng_ether module
is loaded. If a bridge interface is created after loading the ng_ether
module, a netgraph node is created via ether_ifattach().
MFC after: 1 week
Apparently e62e4b8594 wasn't enough to close the race between
a queue being flushed by a packet and callout executing, because
the callouts used without a lock aren't 100% bulletproof. To close
the race use callout_init_mtx() for L2TP timers, and make sure that
all calls to ng_callout()/ng_uncallout() are done under the seq lock.
If used properly, a locked callout can be used transparently with
old netgraph KPI of ng_callout/ng_uncallout which predates locked
callouts.
While here, utilize ng_uncallout_drain() instead of ng_uncallout()
on the node shutdown.
PR: 241133
Reviewed by: mjg, markj
Differential Revision: https://reviews.freebsd.org/D31476
Cover few cases of access to seq without lock missed in 702f98951d.
There are no known bugs fixed with this change, however. With INVARIANTS
embed ng_l2tp_seq_check() into lock/unlock macros. Slightly reduce number
of locks/unlocks per packet keeping the lock between functions.
Reviewed by: mjg, markj
Differential Revision: https://reviews.freebsd.org/D31476
Move shared code into ng_uncallout_internal(). While here add a comment
mentioning a problem with scheduled+executing callout.
Reviewed by: mjg, markj
Differential Revision: https://reviews.freebsd.org/D31476
ng_uncallout. Most of them do not check it anyway, so very little node
changes are required.
Reviewed by: mjg, markj
Differential Revision: https://reviews.freebsd.org/D31476
Currently, most protocols implement pru_listen with something like the
following:
SOCK_LOCK(so);
error = solisten_proto_check(so);
if (error) {
SOCK_UNLOCK(so);
return (error);
}
solisten_proto(so);
SOCK_UNLOCK(so);
solisten_proto_check() fails if the socket is connected or connecting.
However, the socket lock is not used during I/O, so this pattern is
racy.
The change modifies solisten_proto_check() to additionally acquire
socket buffer locks, and the calling thread holds them until
solisten_proto() or solisten_proto_abort() is called. Now that the
socket buffer locks are preserved across a listen(2), this change allows
socket I/O paths to properly interlock with listen(2).
This fixes a large number of syzbot reports, only one is listed below
and the rest will be dup'ed to it.
Reported by: syzbot+9fece8a63c0e27273821@syzkaller.appspotmail.com
Reviewed by: tuexen, gallatin
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D31659
Implement kernel support for RFC 5549/8950.
* Relax control plane restrictions and allow specifying IPv6 gateways
for IPv4 routes. This behavior is controlled by the
net.route.rib_route_ipv6_nexthop sysctl (on by default).
* Always pass final destination in ro->ro_dst in ip_forward().
* Use ro->ro_dst to exract packet family inside if_output() routines.
Consistently use RO_GET_FAMILY() macro to handle ro=NULL case.
* Pass extracted family to nd6_resolve() to get the LLE with proper encap.
It leverages recent lltable changes committed in c541bd368f.
Presence of the functionality can be checked using ipv4_rfc5549_support feature(3).
Example usage:
route add -net 192.0.0.0/24 -inet6 fe80::5054:ff:fe14:e319%vtnet0
Differential Revision: https://reviews.freebsd.org/D30398
MFC after: 2 weeks
newhook can be invoked by ngthread, which runs in a network epoch
section and is thus not permitted to perform M_WAITOK allocations.
Reported by: Jenkins
Reviewed by: donner, afedorov
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D31486
SO_RERROR indicates that receive buffer overflows should be handled as
errors. Historically receive buffer overflows have been ignored and
programs could not tell if they missed messages or messages had been
truncated because of overflows. Since programs historically do not
expect to get receive overflow errors, this behavior is not the
default.
This is really really important for programs that use route(4) to keep
in sync with the system. If we loose a message then we need to reload
the full system state, otherwise the behaviour from that point is
undefined and can lead to chasing bogus bug reports.
Reviewed by: philip (network), kbowling (transport), gbe (manpages)
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D26652
Some code was using it already, but in many places we were testing
SO_ACCEPTCONN directly. As a small step towards fixing some bugs
involving synchronization with listen(2), make the kernel consistently
use SOLISTENING(). No functional change intended.
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
During tests an assert was triggered and pointed to missing flags in
the newlink function of ng_bridge(4).
Reported by: markj
Reviewed by: markj
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D30759
Detailed analysis in https://github.com/genneko/freebsd-vimage-jails/issues/2
brought the problem down to a double call of ng_node_name() before and
after a vnet move. Because the name of the node is already known
(occupied by itself), the second call fails.
PR: 241954
Reported by: Paul Armstrong
MFC: 1 week
Differential Revision: https://reviews.freebsd.org/D30110
Each packet is counted as 128 bytes by the code, not 125. Not sure
what I was thinking about here 14 years ago. May be just a typo.
Reported by: Dmitry Luhtionov <dmitryluhtionov@gmail.com>
MFC after: 2 weeks
Once the final component of the IP address has been parsed, the offset
on the input must not be advanced, as this would remove an unparsed
character from the input.
Submitted by: Markus Stoff
Reviewed by: donner
MFC after: 3 weeks
Differential Revision: https://reviews.freebsd.org/D26489
m_pullup(9) frees the mbuf(9) chain in the case of an allocation error.
The mbuf chain must not be freed again in this case.
PR: 255874
Submitted by: <lylgood@foxmail.com>
Approved by: markj
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D30273
If sending out a packet fails during the loop over all links, the
allocated memory is leaked and not all links receive a copy. This
patch fixes those problems, clarifies a premature abort of the loop,
and fixes a minory style(9) bug.
PR: 255430
Submitted by: Dancho Penev
Tested by: Dancho Penev
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D30008
Hint the compiler, that this update is needed at most once per second.
Only in this case the memory line needs to be written. This will
reduce the amount of cache trashing during forward of most frames.
Suggested by: zec
Approved by: zec
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D28601
The node ng_bridge underwent a lot of changes in the last few months.
All those steps were necessary to distinguish between structure
modifying and read-only data transport paths. Now it's done, the node
can perform frame forwarding on multiple cores in parallel.
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D28123
Use the new control message to move ethernet addresses from a link to
a new link in ng_bridge(4). Send this message instead of doing the
work directly requires to move the loop detection into the control
message processing. This will delay the loop detection by a few
frames.
This decouples the read-only activity from the modification under a
more strict writer lock.
Reviewed by: manpages (gbe)
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D28559
Add a new control message to move ethernet addresses to a given link
in ng_bridge(4). Send this message instead of doing the work directly.
This decouples the read-only activity from the modification under a
more strict writer lock.
Decoupling the work is a prerequisite for multithreaded operation.
Approved by: manpages (bcr), kp (earlier version)
MFC: 3 weeks
Differential Revision: https://reviews.freebsd.org/D28516
Several protocol methods take a sockaddr as input. In some cases the
sockaddr lengths were not being validated, or were validated after some
out-of-bounds accesses could occur. Add requisite checking to various
protocol entry points, and convert some existing checks to assertions
where appropriate.
Reported by: syzkaller+KASAN
Reviewed by: tuexen, melifaro
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D29519
As this controller requires firmware patch downloading to operate.
"Intel Wireless 7265" support in iwmbtfw(8) is yet to be done.
Tested by: arrowd et al
PR: 228787
MFC after: 2 weeks
Unconditional execution of "clear feature" request at SETUP stage was
workaround for probe failures on ng_ubt.ko re-kldloading which is
unnecessary now.
Reviewed by: hselasky
MFC after: 2 weeks
Differential revision: https://reviews.freebsd.org/D29775
For broadcast, multicast and unknown unicast, the replication loop
sends a copy of the packet to each link, beside the first one. This
special path is handled later, but the counters are not updated.
Factor out the common send and count actions as a function.
Reviewed by: kp
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D28537
In the data path of ng_bridge(4), the only value of the host struct,
which needs to be modified, is the staleness, which is reset every
time a frame is received. It's save to leave the code as it is.
This patch is part of a series to make ng_bridge(4) multithreaded.
Reviewed by: kp
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D28546
In a earlier version of ng_bridge(4) the exernal visible host entry
structure was a strict subset of the internal one. So internal view
was a direct annotation of the external structure. This strict
inheritance was lost many versions ago. There is no need to
encapsulate a part of the internal represntation as a separate
structure.
This patch is a preparation to make the internal structure read only
in the data path in order to make ng_bridge(4) multithreaded.
Reviewed by: kp
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D28545
Historically receive buffer overflows have been ignored and programs
could not tell if they missed messages or messages had been truncated
because of overflows. Since programs historically do not expect to get
receive overflow errors, this behavior is not the default.
This is really really important for programs that use route(4) to keep in sync
with the system. If we loose a message then we need to reload the full system
state, otherwise the behaviour from that point is undefined and can lead
to chasing bogus bug reports.
Allocate the necessary memory for the conversion dynamically starting
with a value which is sufficient for almost all normal cases.
PR: 187835
Reviewed by: kp
Differential Revision: https://reviews.freebsd.org/D23840
The data path in netgraph is designed to work on an read only state of
the whole netgraph network. Currently this is achived by convention,
there is no technical enforcment. In the case of NETGRAPH_DEBUG all
nodes can be annotated for debugging purposes, so the strict
enforcment needs to be lifted for this purpose.
This patch is part of a series to make ng_bridge multithreaded, which
is done by rewrite the data path to operate on const.
Reviewed By: kp
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D28141
The data path in netgraph is designed to work on an read only state of
the whole netgraph network. Currently this is achived by convetion,
there is no technical enforcment. This patch is part of a series to
make ng_brigde multithreaded, which is done by rewrite the data path
to const handling.
Reviewed By: kp
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D28141
This was announced to happen after the 12 relases.
Remove a depeciated ABI.
The complete removal is for HEAD only. I'll remove the #define in
stable/13 as MFC, so the code will still exist in 13.x, but will not
included by default. Earlier versions will not be affected.
Reviewed by: kp
MFC after: 5 days
Differential Revision: https://reviews.freebsd.org/D28518
This is the first patch of a series of necessary steps
to make ng_bridge(4) multithreaded.
Reviewed by: melifaro (network), afedorov
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D28125
Handling of unknown MACs on an bridge with incomplete learning
capabilites (aka uplink ports) can be defined in different ways.
The classical approach is to broadcast unicast frames send to an
unknown MAC, because the unknown devices can be everywhere. This mode
is default for ng_bridge(4).
In the case of dedicated uplink ports, which prohibit learning of MAC
addresses in order to save memory and CPU cycles, the broadcast
approach is dangerous. All traffic to the uplink port is broadcasted
to every downlink port, too. In this case, it's better to restrict the
distribution of frames to unknown MAC to the uplink ports only.
In order to keep the chance small and the handling as natural as
possible, the first attached link is used to determine the behaviour
of the bridge: If it is an "uplink" port, then the bridge switch from
classical mode to restricted mode.
Reviewed By: kp
Approved by: kp (mentor)
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D28487
The ng_bridge(4) node is designed to work in moderately small
environments. Connecting such a node to a larger network rapidly fills
the MAC table for no reason. It even become complicated to obtain data
from the gettable message, because the result is too large to
transmit.
This patch introduces, two new functionality bits on the hooks:
- Allow or disallow MAC address learning for incoming patckets.
- Allow or disallow sending unknown MACs through this hook.
Uplinks are characterized by denied learing while sending out
unknowns. Normal links are charaterized by allowed learning and
sending out unknowns.
Reviewed by: kp
Approved by: kp (mentor)
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D23963
Chained policing should be able to reuse the classification of
traffic. A new mbuf_tag type is defined to handle gereral QoS
marking. A new subtype is defined to track the color marking.
Reviewed by: manpages (bcr), melifaro, kp
Approved by: kp (mentor)
Sponsored by: IKS Service GmbH
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D22110
This node is part of an A10-NSP (L2-BSA) development.
Carrier networks tend to stack three or more tags for internal
purposes and therefore hiding the service tags deep inside of the
stack. When decomposing such an access network frame, the processing
order is typically reversed: First distinguish by service, than by
other means.
This new netgragh node allows to bring the relevant VLAN in front (to
the out-most position). This way other netgraph nodes (like ng_vlan)
can operate on this specific type.
Reviewed by: manpages (gbe), brueffer (manpages), kp
Approved by: kp (mentor)
MFC after: 1 month
Relnotes: yes
Sponsored by: IKS Service GmbH
Differential Revision: https://reviews.freebsd.org/D22076
This extends upon the RFC 6598 support to libalias/ipfw in r357092.
Reviewed By: manpages (bcr), donner, adrian, kp
Approved by: kp (mentor)
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D23461
The original comment suggests an optimization, which was proven wrong.
Reported by: nc
Reviewed by: kp, nc
Approved by: kp (mentor)
Differential Revision: https://reviews.freebsd.org/D23727
ng_tag(4) operate on arbitrary data of mbuf_tags(9). Those structures
are padded to the next multiple of the alignment by the compiler.
Hence a valid argument has be at most as long as the data received.
PR: 241462
Reviewed by: kp
Approved by: kp (mentor)
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D22140
pccard is being removed, so remove bt3c driver since it only has PC
Card attachment. Also remove bt3cfw(8) since it's the firmware for this
driver.
Relnotes: Yes
Use recently-added combination of `fib[46]_lookup_rt()` which
returns rtentry & raw nexthop with `rt_get_inet[6]_plen()` which
returns address/prefix length of prefix inside `rt`.
Add `nhop_select_func()` wrapper around inlined `nhop_select()` to
allow callers external to the routing subsystem select the proper
nexthop from the multipath group without including internal headers.
New calls does not require reference counting objects and reduce
the amount of copied/processed rtentry data.
Differential Revision: https://reviews.freebsd.org/D27675
When tearing down a VNET, netgraph sends shutdown messages to all of the
nodes before detaching interfaces (SI_SUB_NETGRAPH comes before
SI_SUB_INIT_IF in teardown order). ng_ether nodes handle this by
destroying themselves without detaching from the parent ifnet. Then,
when ifnets go away they detach their ng_ether nodes again, triggering a
use-after-free.
Handle this by modifying ng_ether_shutdown() to detach from the ifnet.
If the shutdown was triggered by an ifnet being destroyed, we will clear
priv->ifp in the ng_ether detach callback, so priv->ifp may be NULL.
Also get rid of the printf in vnet_netgraph_uninit(). It can be
triggered trivially by ng_ether since ng_ether_shutdown() persists the
node unless NG_REALLY_DIE is set.
PR: 233622
Reviewed by: afedorov, kp, Lutz Donnerhacke
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D27662
This is just a minor optimization, but it's sensitive. This gives an improvement of 30-50 kpps.
Reviewed by: kp, markj, glebius, lutz_donnerhacke.de
Approved by: vmaffione (mentor)
Sponsored by: vstack.com
Differential Revision: https://reviews.freebsd.org/D27382
Two issues:
- The DEBUG macro defined is in direct conflict with the DEBUG kernel
option, which broke the -LINT build[0]
- Building with NG_MACFILTER_DEBUG did not compile on LP64 systems due to
using %d for sizeof().
Reported by: Jenkins[0]
Macfilter to route packets through different hooks based on sender MAC address.
Based on ng_macfilter written by Pekka Nikander
Sponsered by Retina b.v.
Reviewed by: afedorov
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D27268
The revision r342168 broke ABI of ng_nat needlessly and
the change was merged to stable branches breaking ABI there, too.
Unbreak it.
PR: 250722
MFC after: 1 week
A received control packet may cause the transmit queue to be flushed, in
which case ng_l2tp_seq_recv_nr() cancels the transmit timeout handler.
The handler checks to see if it was cancelled before doing anything, but
did so before acquiring the node lock, so a small race window could
cause ng_l2tp_seq_rack_timeout() to attempt to flush an empty queue,
ultimately causing a null pointer dereference.
PR: 241133
Reviewed by: bz, glebius, Lutz Donnerhacke
MFC after: 3 days
Sponsored by: Rubicon Communications, LLC (Netgate)
Differential Revision: https://reviews.freebsd.org/D26548
We must enter NET_EPOCH before calling ether_output_frame(). Several of the
functions it calls (pfil_run_hooks, if_transmit) expect to be running in the
NET_EPOCH.
While here remove an unneeded EPOCH entry (which wasn't wide enough to cover
BRIDGE_INPUT).
PR: 248958
Reviewed by: glebius, bz (previous version), melifaro (previous version)
Tested by: manu
Differential Revision: https://reviews.freebsd.org/D26226
Update the ng_iface documentation and hooks to reflect the fact that the
node currently only supports IPv4 and v6 packets.
Reviewed by: Lutz Donnerhacke
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D25862
This is normally handled by a netgraph thread, but netgraph messages may
be dispatched directly to a node, in which case no VNET is set before
ng_iface calls into the network stack. Netgraph could probably handle
this more generally, but for now just be sure to set the current VNET in
ng_iface.
PR: 242406
Tested by: Michael Muenz <m.muenz@gmail.com>
Reviewed by: Lutz Donnerhacke
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D25788
ACL packet boundary flag should be 0 instead of 2 for LE PDU.
Some HCI will drop LE packet with PB flag is 2, and if sent,
some target may reject the packet.
PR: 248024
Reported by: Greg V
Reviewed by: Greg V, emax
Differential Revision: https://reviews.freebsd.org/D25704
Refer to bluetooth core v5.2 specifications Vol4. Part E. 7.8.27.
PR: 245763
Submitted by: Marc Veldman <marc@bumblingdork.com>
MFC after: 1 week
Sponsored by: Mellanox Technologies
A later change, currently being iterated on in D24459, will in-fact change
the lock type to an sx so that TTY drivers can sleep on it if they need to.
Committing this ahead of time to make the review in question a little more
palatable.
tty_lock_assert() is unfortunately still needed for now in two places to
make sure that the tty lock has not been recursed upon, for those scenarios
where it's supplied by the TTY driver and possibly a mutex that is allowed
to recurse.
Suggested by: markj
r357614 added CTLFLAG_NEEDGIANT to make it easier to find nodes that are
still not MPSAFE (or already are but aren’t properly marked).
Use it in preparation for a general review of all nodes.
This is non-functional change that adds annotations to SYSCTL_NODE and
SYSCTL_PROC nodes using one of the soon-to-be-required flags.
Mark all obvious cases as MPSAFE. All entries that haven't been marked
as MPSAFE before are by default marked as NEEDGIANT
Approved by: kib (mentor, blanket)
Commented by: kib, gallatin, melifaro
Differential Revision: https://reviews.freebsd.org/D23718
From the beginning, ng_nat safely assumed cleansed traffic
because of limited ways it could be attached to NETGRAPH:
ng_ipfw or ng_ppp only.
Now as it may be attached with ng_ether too, the assumption proven wrong.
Add needed check to the ng_nat. Thanks for markj for debugging this.
PR: 243096
Submitted by: Lutz Donnerhacke <lutz@donnerhacke.de>
Reported by: Robert James Hernandez <rob@sarcasticadmin.com>
Reviewed by: markj and others
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D23091
ng_nat implements NAT for IPv4 traffic only. When connected to an
ng_ether node it erroneously handled IPv6 packets as well.
This change is not sufficient: ng_nat does not do any validation of IP
packets in this mode, even though they have not yet passed through
ip_input().
PR: 243096
Reported by: Robert James Hernandez <rob@sarcasticadmin.com>
Reviewed by: julian
Differential Revision: https://reviews.freebsd.org/D23080
In order to be able to merge r353026 bring back support for the old
cookie API for a transition period in 12.x releases (and possibly 13)
before the old API can be removed again entirely.
Suggested by: julian
Submitted by: Lutz Donnerhacke (lutz donnerhacke.de)
PR: 240787
Reviewed by: julian
MFC after: 2 weeks
X-MFC with: r353026
Differential Revision: https://reviews.freebsd.org/D21961
When epoch(9) was introduced to network stack, it was basically
dropped in place of existing locking, which was mutexes and
rwlocks. For the sake of performance mutex covered areas were
as small as possible, so became epoch covered areas.
However, epoch doesn't introduce any contention, it just delays
memory reclaim. So, there is no point to minimise epoch covered
areas in sense of performance. Meanwhile entering/exiting epoch
also has non-zero CPU usage, so doing this less often is a win.
Not the least is also code maintainability. In the new paradigm
we can assume that at any stage of processing a packet, we are
inside network epoch. This makes coding both input and output
path way easier.
On output path we already enter epoch quite early - in the
ip_output(), in the ip6_output().
This patch does the same for the input path. All ISR processing,
network related callouts, other ways of packet injection to the
network stack shall be performed in net_epoch. Any leaf function
that walks network configuration now asserts epoch.
Tricky part is configuration code paths - ioctls, sysctls. They
also call into leaf functions, so some need to be changed.
This patch would introduce more epoch recursions (see EPOCH_TRACE)
than we had before. They will be cleaned up separately, as several
of them aren't trivial. Note, that unlike a lock recursion the
epoch recursion is safe and just wastes a bit of resources.
Reviewed by: gallatin, hselasky, cy, adrian, kristof
Differential Revision: https://reviews.freebsd.org/D19111
