Commit Graph

4470 Commits

Author SHA1 Message Date
Konstantin Belousov
80ba361b2f if_media.c SIOCGMEDIAX handler: improve loop
Stop advancing counter past the current iteration number at the start
of iteration.  This removes the need of subtracting one when
calculating index for copyout, and arguably fixes off-by-one reporting
of copied out elements when copyout failed.

Reviewed by:	hselasky
Sponsored by:	Mellanox Technologies / NVidia Networking
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D27073
2020-11-03 14:33:04 +00:00
Konstantin Belousov
1fbbe9dbf5 net/if_media.c: improve IFMEDIA_DEBUG output.
Use consistent output format for hex.
Print both media and mask where relevant.

Reviewed by:	hselasky
Sponsored by:	Mellanox Technologies/NVidia Networking
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D27034
2020-11-01 16:38:30 +00:00
Konstantin Belousov
e399f19dba Cleanup of net/if_media.c: simplify cleanup loop in ifmedia_removeall().
Reviewed by:	hselasky
Sponsored by:	Mellanox Technologies/NVidia Networking
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D27034
2020-11-01 16:36:21 +00:00
Konstantin Belousov
899322fdfa Cleanup of net/if_media.c: some style.
Reviewed by:	hselasky
Sponsored by:	Mellanox Technologies/NVidia Networking
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D27034
2020-11-01 16:30:17 +00:00
Konstantin Belousov
2193fb16b5 Cleanup of net/if_media.c: switch to ANSI C function definitions.
Reviewed by:	hselasky
Sponsored by:	Mellanox Technologies/NVidia Networking
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D27034
2020-11-01 16:25:35 +00:00
Mitchell Horne
ced0f52457 net: add ETHER_IS_IPV6_MULTICAST
This can be used to detect if an ethernet address is specifically an
IPv6 multicast address, defined in accordance to RFC 2464.

ETHER_IS_MULTICAST is still preferred in the general case.

Reviewed by:	ae
Sponsored by:	NetApp, Inc.
Sponsored by:	Klara, Inc.
Differential Revision:	https://reviews.freebsd.org/D26611
2020-10-30 13:32:58 +00:00
John Baldwin
36e0a362ac Add m_snd_tag_alloc() as a wrapper around if_snd_tag_alloc().
This gives a more uniform API for send tag life cycle management.

Reviewed by:	gallatin, hselasky
Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D27000
2020-10-29 23:28:39 +00:00
John Baldwin
521eac97f3 Support hardware rate limiting (pacing) with TLS offload.
- Add a new send tag type for a send tag that supports both rate
  limiting (packet pacing) and TLS offload (mostly similar to D22669
  but adds a separate structure when allocating the new tag type).

- When allocating a send tag for TLS offload, check to see if the
  connection already has a pacing rate.  If so, allocate a tag that
  supports both rate limiting and TLS offload rather than a plain TLS
  offload tag.

- When setting an initial rate on an existing ifnet KTLS connection,
  set the rate in the TCP control block inp and then reset the TLS
  send tag (via ktls_output_eagain) to reallocate a TLS + ratelimit
  send tag.  This allocates the TLS send tag asynchronously from a
  task queue, so the TLS rate limit tag alloc is always sleepable.

- When modifying a rate on a connection using KTLS, look for a TLS
  send tag.  If the send tag is only a plain TLS send tag, assume we
  failed to allocate a TLS ratelimit tag (either during the
  TCP_TXTLS_ENABLE socket option, or during the send tag reset
  triggered by ktls_output_eagain) and ignore the new rate.  If the
  send tag is a ratelimit TLS send tag, change the rate on the TLS tag
  and leave the inp tag alone.

- Lock the inp lock when setting sb_tls_info for a socket send buffer
  so that the routines in tcp_ratelimit can safely dereference the
  pointer without needing to grab the socket buffer lock.

- Add an IFCAP_TXTLS_RTLMT capability flag and associated
  administrative controls in ifconfig(8).  TLS rate limit tags are
  only allocated if this capability is enabled.  Note that TLS offload
  (whether unlimited or rate limited) always requires IFCAP_TXTLS[46].

Reviewed by:	gallatin, hselasky
Relnotes:	yes
Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D26691
2020-10-29 00:23:16 +00:00
Vincenzo Maffione
be7a6b3d84 iflib: fix typo bug introduced by r367093
Code was supposed to call callout_reset_sbt_on() rather than
callout_reset_sbt(). This resulted into passing a "cpu" value
to a "flag" argument. A recipe for subtle errors.

PR:	248652
Reported by:	sg@efficientip.com
MFC with: r367093
2020-10-28 21:06:17 +00:00
Vincenzo Maffione
17cec474c0 iflib: add per-tx-queue netmap timer
The way netmap TX is handled in iflib when TX interrupts are not
used (IFC_NETMAP_TX_IRQ not set) has some issues:
  - The netmap_tx_irq() function gets called by iflib_timer(), which
    gets scheduled with tick granularity (hz). This is not frequent
    enough for 10Gbps NICs and beyond (e.g., ixgbe or ixl). The end
    result is that the transmitting netmap application is not woken
    up fast enough to saturate the link with small packets.
  - The iflib_timer() functions also calls isc_txd_credits_update()
    to ask for more TX completion updates. However, this violates
    the netmap requirement that only txsync can access the TX queue
    for datapath operations. Only netmap_tx_irq() may be called out
    of the txsync context.

This change introduces per-tx-queue netmap timers, using microsecond
granularity to ensure that netmap_tx_irq() can be called often enough
to allow for maximum packet rate. The timer routine simply calls
netmap_tx_irq() to wake up the netmap application. The latter will
wake up and call txsync to collect TX completion updates.

This change brings back line rate speed with small packets for ixgbe.
For the time being, timer expiration is hardcoded to 90 microseconds,
in order to avoid introducing a new sysctl.
We may eventually implement an adaptive expiration period or use another
deferred work mechanism in place of timers.

Also, fix the timers usage to make sure that each queue is serviced
by a different CPU.

PR:	248652
Reported by:	sg@efficientip.com
MFC after:	2 weeks
2020-10-27 21:53:33 +00:00
Hans Petter Selasky
1355e2dc4f More style fixes (partial revert of r366994).
Suggested by:		danfe@
Differential Revision:	https://reviews.freebsd.org/D26254
MFC after:		1 week
Sponsored by:		Mellanox Technologies // NVIDIA Networking
2020-10-24 13:07:50 +00:00
Hans Petter Selasky
1d3a22e765 Fix order of header files:
sys/systm.h should come right after sys/param.h

Suggested by:		kib@
Differential Revision:	https://reviews.freebsd.org/D26254
MFC after:		1 week
Sponsored by:		Mellanox Technologies // NVIDIA Networking
2020-10-24 10:52:09 +00:00
Hans Petter Selasky
01630a496b Run code through "clang-format -style=file" with some additional fixes.
No functional change.

Suggested by:		kib@ and emaste@
Differential Revision:	https://reviews.freebsd.org/D26254
MFC after:		1 week
Sponsored by:		Mellanox Technologies // NVIDIA Networking
2020-10-24 10:23:21 +00:00
Navdeep Parhar
610d345953 if_vxlan(4): csum_flags_to_inner_flags takes the tunnel protocol as a parameter.
No functional change.
2020-10-22 17:05:55 +00:00
Hans Petter Selasky
ce329aa256 Compile fix for MIPS, MIPS64, POWERPC and POWERPC64.
Add missing include files.

Differential Revision:	https://reviews.freebsd.org/D26254
Reviewed by:		melifaro@
MFC after:		1 week
Sponsored by:		Mellanox Technologies // NVIDIA Networking
2020-10-22 12:22:08 +00:00
Hans Petter Selasky
a92c4bb62a Add support for IP over infiniband, IPoIB, to lagg(4). Currently only
the failover protocol is supported due to limitations in the IPoIB
architecture. Refer to the lagg(4) manual page for how to configure
and use this new feature. A new network interface type,
IFT_INFINIBANDLAG, has been added, similar to the existing
IFT_IEEE8023ADLAG .

ifconfig(8) has been updated to accept a new laggtype argument when
creating lagg(4) network interfaces. This new argument is used to
distinguish between ethernet and infiniband type of lagg(4) network
interface. The laggtype argument is optional and defaults to
ethernet. The lagg(4) command line syntax is backwards compatible.

Differential Revision:	https://reviews.freebsd.org/D26254
Reviewed by:		melifaro@
MFC after:		1 week
Sponsored by:		Mellanox Technologies // NVIDIA Networking
2020-10-22 09:47:12 +00:00
Hans Petter Selasky
9d40cf60d6 Factor out generic IP over infiniband, IPoIB, definitions and code
into net/if_infiniband.c and net/infiniband.h . No functional change
intended.

Differential Revision:	https://reviews.freebsd.org/D26254
Reviewed by:		melifaro@
MFC after:		1 week
Sponsored by:		Mellanox Technologies // NVIDIA Networking
2020-10-22 09:09:53 +00:00
Alexander V. Chernikov
c7cffd65c5 Add support for stacked VLANs (IEEE 802.1ad, AKA Q-in-Q).
802.1ad interfaces are created with ifconfig using the "vlanproto" parameter.
Eg., the following creates a 802.1Q VLAN (id #42) over a 802.1ad S-VLAN
(id #5) over a physical Ethernet interface (em0).

ifconfig vlan5 create vlandev em0 vlan 5 vlanproto 802.1ad up
ifconfig vlan42 create vlandev vlan5 vlan 42 inet 10.5.42.1/24

VLAN_MTU, VLAN_HWCSUM and VLAN_TSO capabilities should be properly
supported. VLAN_HWTAGGING is only partially supported, as there is
currently no IFCAP_VLAN_* denoting the possibility to set the VLAN
EtherType to anything else than 0x8100 (802.1ad uses 0x88A8).

Submitted by:	Olivier Piras
Sponsored by:	RG Nets
Differential Revision:	https://reviews.freebsd.org/D26436
2020-10-21 21:28:20 +00:00
Alexander V. Chernikov
0c325f53f1 Implement flowid calculation for outbound connections to balance
connections over multiple paths.

Multipath routing relies on mbuf flowid data for both transit
 and outbound traffic. Current code fills mbuf flowid from inp_flowid
 for connection-oriented sockets. However, inp_flowid is currently
 not calculated for outbound connections.

This change creates simple hashing functions and starts calculating hashes
 for TCP,UDP/UDP-Lite and raw IP if multipath routes are present in the
 system.

Reviewed by:	glebius (previous version),ae
Differential Revision:	https://reviews.freebsd.org/D26523
2020-10-18 17:15:47 +00:00
Marcin Wojtas
1148702e43 Add SADB_SAFLAGS_ESN flag
This flag is going to be used by IKE daemon to signal if
Extended Sequence Number feature is going to be used.

Value for this flag was taken from OpenBSD source code
6b4cbaf181

Submitted by:           Patryk Duda <pdk@semihalf.com>
Reviewed by:            ae
Differential revision:  https://reviews.freebsd.org/D22366
Obtained from:          Semihalf
Sponsored by:           Stormshield
2020-10-16 11:22:29 +00:00
Richard Scheffenegger
868aabb470 Add IP(V6)_VLAN_PCP to set 802.1 priority per-flow.
This adds a new IP_PROTO / IPV6_PROTO setsockopt (getsockopt)
option IP(V6)_VLAN_PCP, which can be set to -1 (interface
default), or explicitly to any priority between 0 and 7.

Note that for untagged traffic, explicitly adding a
priority will insert a special 801.1Q vlan header with
vlan ID = 0 to carry the priority setting

Reviewed by:	gallatin, rrs
MFC after:	2 weeks
Sponsored by:	NetApp, Inc.
Differential Revision:	https://reviews.freebsd.org/D26409
2020-10-09 12:06:43 +00:00
Konstantin Belousov
cefdb89514 Fix typo.
Sponsored by:	Mellanox Technologies/NVIDIA Networking
MFC after:	3 days
2020-10-07 10:58:56 +00:00
Kristof Provost
4af1bd8157 bridge: call member interface ioctl() without NET_EPOCH
We're not allowed to hold NET_EPOCH while sleeping, so when we call ioctl()
handlers for member interfaces we cannot be in NET_EPOCH.  We still need some
protection of our CK_LISTs, so hold BRIDGE_LOCK instead.

That requires changing BRIDGE_LOCK into a sleepable lock, and separating the
BRIDGE_RT_LOCK, to protect bridge_rtnode lists. That lock is taken in the data
path (while in NET_EPOCH), so it cannot be a sleepable lock.

While here document the locking strategy.

MFC after:	1 month
Differential Revision:	https://reviews.freebsd.org/D26418
2020-10-06 19:19:56 +00:00
John Baldwin
56fb710f1b Store the send tag type in the common send tag header.
Both cxgbe(4) and mlx5(4) wrapped the existing send tag header with
their own identical headers that stored the type that the
type-specific tag structures inherited from, so in practice it seems
drivers need this in the tag anyway.  This permits removing these
extra header indirections (struct cxgbe_snd_tag and struct
mlx5e_snd_tag).

In addition, this permits driver-independent code to query the type of
a tag, e.g. to know what type of tag is being queried via
if_snd_query.

Reviewed by:	gallatin, hselasky, np, kib
Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D26689
2020-10-06 17:58:56 +00:00
Alexander V. Chernikov
1b95005e95 Fix route flags update during RTM_CHANGE.
Nexthop lookup was not consireding rt_flags when doing
 structure comparison, which lead to an original nexthop
 selection when changing flags. Fix the case by adding
 rt_flags field into comparison and rearranging nhop_priv
 fields to allow for efficient matching.
Fix `route change X/Y flags` case - recent changes
 disallowed specifying RTF_GATEWAY flag without actual gateway.
 It turns out, route(8) fills in RTF_GATEWAY by default, unless
 -interface flag is specified. Fix regression by clearing
 RTF_GATEWAY flag instead of failing.
Fix route flag reporting in RTM_CHANGE messages by explicitly
 updating rtm_flags after operation competion.
Add IPv4/IPv6 tests for flag-only route changes.
2020-10-04 13:24:58 +00:00
Alexander V. Chernikov
9c584fa4bc Remove ROUTE_MPATH-related warnings introduced in r366390.
Reported by:	mjg
2020-10-03 14:37:54 +00:00
Alexander V. Chernikov
fedeb08b6a Introduce scalable route multipath.
This change is based on the nexthop objects landed in D24232.

The change introduces the concept of nexthop groups.
Each group contains the collection of nexthops with their
 relative weights and a dataplane-optimized structure to enable
 efficient nexthop selection.

Simular to the nexthops, nexthop groups are immutable. Dataplane part
 gets compiled during group creation and is basically an array of
 nexthop pointers, compiled w.r.t their weights.

With this change, `rt_nhop` field of `struct rtentry` contains either
 nexthop or nexthop group. They are distinguished by the presense of
 NHF_MULTIPATH flag.
All dataplane lookup functions returns pointer to the nexthop object,
leaving nexhop groups details inside routing subsystem.

User-visible changes:

The change is intended to be backward-compatible: all non-mpath operations
 should work as before with ROUTE_MPATH and net.route.multipath=1.

All routes now comes with weight, default weight is 1, maximum is 2^24-1.

Current maximum multipath group width is statically set to 64.
 This will become sysctl-tunable in the followup changes.

Using functionality:
* Recompile kernel with ROUTE_MPATH
* set net.route.multipath to 1

route add -6 2001:db8::/32 2001:db8::2 -weight 10
route add -6 2001:db8::/32 2001:db8::3 -weight 20

netstat -6On

Nexthop groups data

Internet6:
GrpIdx  NhIdx     Weight   Slots                                 Gateway     Netif  Refcnt
1         ------- ------- ------- --------------------------------------- ---------       1
              13      10       1                             2001:db8::2     vlan2
              14      20       2                             2001:db8::3     vlan2

Next steps:
* Land outbound hashing for locally-originated routes ( D26523 ).
* Fix net/bird multipath (net/frr seems to work fine)
* Add ROUTE_MPATH to GENERIC
* Set net.route.multipath=1 by default

Tested by:	olivier
Reviewed by:	glebius
Relnotes:	yes
Differential Revision:	https://reviews.freebsd.org/D26449
2020-10-03 10:47:17 +00:00
Vincenzo Maffione
adf41f0788 netmap: fix constness warnings generated by "-Wcast-qual"
Submitted by:	milosz.kaniewski@gmail.com
MFC after:	3 days
2020-10-03 09:33:29 +00:00
Ed Maste
c1aedfcbd9 add SIOCGIFDATA ioctl
For interfaces that do not support SIOCGIFMEDIA (for which there are
quite a few) the only fallback is to query the interface for
if_data->ifi_link_state.  While it's possible to get at if_data for an
interface via getifaddrs(3) or sysctl, both are heavy weight mechanisms.

SIOCGIFDATA is a simple ioctl to retrieve this fast with very little
resource use in comparison.  This implementation mirrors that of other
similar ioctls in FreeBSD.

Submitted by:	Roy Marples <roy@marples.name>
Reviewed by:	markj
MFC after:	1 month
Differential Revision:	https://reviews.freebsd.org/D26538
2020-09-28 16:54:39 +00:00
Alexander V. Chernikov
2259a03020 Rework part of routing code to reduce difference to D26449.
* Split rt_setmetrics into get_info_weight() and rt_set_expire_info(),
 as these two can be applied at different entities and at different times.
* Start filling route weight in route change notifications
* Pass flowid to UDP/raw IP route lookups
* Rework nd6_subscription_cb() and sysctl_dumpentry() to prepare for the fact
 that rtentry can contain multiple nexthops.

Differential Revision:	https://reviews.freebsd.org/D26497
2020-09-21 20:02:26 +00:00
Alexander V. Chernikov
1440f62266 Remove unused nhop_ref_any() function.
Remove "opt_mpath.h" header where not needed.

No functional changes.
2020-09-20 21:32:52 +00:00
Alexander V. Chernikov
c4bcfe98e2 Fix gw updates / flag updates during route changes.
* Zero gw_sdl if switching to interface route - the assumption
 that underlying storage is zeroed is incorrect with route changes.
* Apply proper flag mask to rte.

Reported by:	vangyzen
2020-09-20 12:31:48 +00:00
Navdeep Parhar
b092fd6c97 if_vxlan(4): add support for hardware assisted checksumming, TSO, and RSS.
This lets a VXLAN pseudo-interface take advantage of hardware checksumming (tx
and rx), TSO, and RSS if the NIC is capable of performing these operations on
inner VXLAN traffic.

A VXLAN interface inherits the capabilities of its vxlandev interface if one is
specified or of the interface that hosts the vxlanlocal address. If other
interfaces will carry traffic for that VXLAN then they must have the same
hardware capabilities.

On transmit, if_vxlan verifies that the outbound interface has the required
capabilities and then translates the CSUM_ flags to their inner equivalents.
This tells the hardware ifnet that it needs to operate on the inner frame and
not the outer VXLAN headers.

An event is generated when a VXLAN ifnet starts. This allows hardware drivers to
configure their devices to expect VXLAN traffic on the specified incoming port.

On receive, the hardware does RSS and checksum verification on the inner frame.
if_vxlan now does a direct netisr dispatch to take full advantage of RSS. It is
not very clear why it didn't do this already.

Future work:
Rx: it should be possible to avoid the first trip up the protocol stack to get
the frame to if_vxlan just so it can decapsulate and requeue for a second trip
up the stack. The hardware NIC driver could directly call an if_vxlan receive
routine for VXLAN traffic instead.

Rx: LRO. depends on what happens with the previous item. There will have to to
be a mechanism to indicate that it's time for if_vxlan to flush its LRO state.

Reviewed by:	kib@
Relnotes:	Yes
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D25873
2020-09-18 02:37:57 +00:00
Navdeep Parhar
830edb4561 Add two new ifnet capabilities for hw checksumming and TSO for VXLAN traffic.
These are similar to the existing VLAN capabilities.

Reviewed by:	kib@
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D25873
2020-09-18 02:10:28 +00:00
Mitchell Horne
ceff9b9d25 if_media: definitions for 40GE LM4 ethernet media type
Reviewed by:	erj
Sponsored by:	NetApp, Inc.
Sponsored by:	Klara, Inc.
Differential Revision:	https://reviews.freebsd.org/D26276
2020-09-16 14:45:16 +00:00
Alexander V. Chernikov
2b32d93e55 Fix RADIX_MPATH build broken by r365521.
Reported by:	jenkins, Hartmann, O. <ohartmann at walstatt.org>
2020-09-10 07:05:31 +00:00
Alexander V. Chernikov
aa8f9f90ff Update nexthop handling for route addition/deletion in preparation for mpath.
Currently kernel requests deletion for the certain routes with specified gateway,
 but this gateway is not actually checked. With multipath routes, internal gateway
 checking becomes mandatory. Add the logic performing this check.

Generalise RTF_PINNED routes to the generic route priorities, simplifying the logic.

Add lookup_prefix() function to perform exact match search based on data in @info.

Differential Revision:	https://reviews.freebsd.org/D26356
2020-09-09 22:07:54 +00:00
Alexander V. Chernikov
cd6298d5c5 Retain marking net.fibs sysctl as a tunable.
Suggested by:	avg
2020-09-09 21:45:18 +00:00
Alexander V. Chernikov
4a8201c13a Fix panic with net.fibs tunable set in loader.conf.
Fix by removing forgotten CTLFLAG_RWTUN flag from the sysctl,
 loader variable will be read later in vnet_rtables_init().

Reported by:	mav
2020-09-08 21:39:34 +00:00
Kristof Provost
a969635b83 net: mitigate vnet / epair cleanup races
There's a race where dying vnets move their interfaces back to their original
vnet, and if_epair cleanup (where deleting one interface also deletes the other
end of the epair). This is commonly triggered by the pf tests, but also by
cleanup of vnet jails.

As we've not yet been able to fix the root cause of the issue work around the
panic by not dereferencing a NULL softc in epair_qflush() and by not
re-attaching DYING interfaces.

This isn't a full fix, but makes a very common panic far less likely.

PR:		244703, 238870
Reviewed by:	lutz_donnerhacke.de
MFC after:	4 days
Differential Revision:	https://reviews.freebsd.org/D26324
2020-09-08 14:54:10 +00:00
Alexander V. Chernikov
05aca418f4 Consistently use the same gateway when adding/deleting interface routes.
Use the same link-level gateway when adding or deleting interface routes.
This helps nexthop checking in the upcoming multipath changes.

Differential Revision:	https://reviews.freebsd.org/D26317
2020-09-07 10:13:54 +00:00
Ed Maste
4fa9815a3d rtsock.c: remove extraneous space
Submitted by:	Neel Chauhan <neel AT neelc DOT org>
Differential Revision:	https://reviews.freebsd.org/D26249
2020-09-05 16:13:36 +00:00
Alexander V. Chernikov
8f07963360 Fix regression for IPv6 loopback routes.
After nexthop introduction, loopback routes for the interface addresses
 were created without embedding actual interface index in the gateway.
 The latter is needed to pass the IPv6 scope during transmission via loopback..

Fix the regression by actually using passed gateway data with interface index.

Differential Revision:	https://reviews.freebsd.org/D26306
2020-09-03 22:24:52 +00:00
Mateusz Guzik
662c13053f net: clean up empty lines in .c and .h files 2020-09-01 21:19:14 +00:00
Vincenzo Maffione
35d8a463e8 iflib: leave only 1 receive descriptor unused
The pidx argument of isc_rxd_flush() indicates which is the last valid
receive descriptor to be used by the NIC. However, current code has
multiple issues:
  - Intel drivers write pidx to their RDT register, which means that
    NICs will only use the descriptors up to pidx-1 (modulo ring size N),
    and won't actually use the one pointed by pidx. This does not break
    reception, but it is anyway confusing and suboptimal (the NIC will
    actually see only N-2 descriptors as available, rather than N-1).
    Other drivers (if_vmx, if_bnxt, if_mgb) adhere to this semantic).
  - The semantic used by Intel (RDT is one descriptor past the last
     valid one) is used by most (if not all) NICs, and it is also used
     on the TX side (also in iflib). Since iflib is not currently
     using this semantic for RX, it must decrement fl->ifl_pidx
     (modulo N) before calling isc_rxd_flush(), and then the
     per-driver callback implementation must increment the index
     again (to match the real semantic). This is confusing and suboptimal.
  -  The iflib refill function is also called at initialization.
     However, in case the ring size is smaller than 128 (e.g. if_mgb),
     the refill function will actually prepare all the receive
     descriptors (N), without leaving one unused, as most of NICs assume
     (e.g. to avoid RDT to overrun RDH). I can speculate that the code
     looks like this right now because this issue showed up during
     testing (e.g. with if_mgb), and it was easy to workaround by
     decrementing pidx before isc_rxd_flush().

The goal of this change is to simplify the code (removing a bunch
of instructions from the RX fast path), and to make the semantic of
isc_rxd_flush() consistent across drivers. To achieve this, we:
  - change the semantics of the pidx argument to the usual one (that
    is the index one past the last valid one), so that both iflib and
    drivers avoid the decrement/increment dance.
  - fix the initialization code to prepare at most N-1 descriptors.

Reviewed by:	markj
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D26191
2020-09-01 20:41:47 +00:00
Alexander V. Chernikov
b8d2d479cd Revert uma zone alignemnt cache unadvertenly committed in r364950. 2020-08-29 12:04:13 +00:00
Alexander V. Chernikov
6498f66f7c Fix build with RADIX_MPATH.
Reported by:	Hartmann, O <ohartmann@walstatt.org>
2020-08-29 11:04:24 +00:00
Alexander V. Chernikov
7c89a3b63f Move fib_rte_to_nh_flags() from net/route_var.h to net/route/nhop_ctl.c.
No functional changes.
Initially this function was created to perform runtime flag conversions
 for the previous incarnation of fib lookup functions. As these functions
 got deprecated, move the function to the file with the only remaining
 caller. Lastly, rename it to convert_rt_to_nh_flags() to follow the
 naming notation.
2020-08-28 23:01:56 +00:00
Alexander V. Chernikov
a624ca3dff Move net/route/shared.h definitions to net/route/route_var.h.
No functional changes.

net/route/shared.h was created in the inital phases of nexthop conversion.
It was intended to serve the same purpose as route_var.h - share definitions
 of functions and structures between the routing subsystem components. At
 that time route_var.h was included by many files external to the routing
 subsystem, which largerly defeats its purpose.

As currently this is not the case anymore and amount of route_var.h includes
 is roughly the same as shared.h, retire the latter in favour of the former.
2020-08-28 22:50:20 +00:00
Alexander V. Chernikov
b122304f6a Further split nhop creation and rtable operations.
As nexthops are immutable, some operations such as route attribute changes
 require nexthop fetching, forking, modification and route switching.
These operations are not atomic, so they may need to be retried multiple
 times in presence of multiple speakers changing the same route.

This change introduces "synchronisation" primitive: route_update_conditional(),
 simplifying logic for route changes and upcoming multipath operations.

Differential Revision:	https://reviews.freebsd.org/D26216
2020-08-28 21:59:10 +00:00