freebsd-skq/sys/netinet6
John Baldwin fb3bc59600 Restructure mbuf send tags to provide stronger guarantees.
- Perform ifp mismatch checks (to determine if a send tag is allocated
  for a different ifp than the one the packet is being output on), in
  ip_output() and ip6_output().  This avoids sending packets with send
  tags to ifnet drivers that don't support send tags.

  Since we are now checking for ifp mismatches before invoking
  if_output, we can now try to allocate a new tag before invoking
  if_output sending the original packet on the new tag if allocation
  succeeds.

  To avoid code duplication for the fragment and unfragmented cases,
  add ip_output_send() and ip6_output_send() as wrappers around
  if_output and nd6_output_ifp, respectively.  All of the logic for
  setting send tags and dealing with send tag-related errors is done
  in these wrapper functions.

  For pseudo interfaces that wrap other network interfaces (vlan and
  lagg), wrapper send tags are now allocated so that ip*_output see
  the wrapper ifp as the ifp in the send tag.  The if_transmit
  routines rewrite the send tags after performing an ifp mismatch
  check.  If an ifp mismatch is detected, the transmit routines fail
  with EAGAIN.

- To provide clearer life cycle management of send tags, especially
  in the presence of vlan and lagg wrapper tags, add a reference count
  to send tags managed via m_snd_tag_ref() and m_snd_tag_rele().
  Provide a helper function (m_snd_tag_init()) for use by drivers
  supporting send tags.  m_snd_tag_init() takes care of the if_ref
  on the ifp meaning that code alloating send tags via if_snd_tag_alloc
  no longer has to manage that manually.  Similarly, m_snd_tag_rele
  drops the refcount on the ifp after invoking if_snd_tag_free when
  the last reference to a send tag is dropped.

  This also closes use after free races if there are pending packets in
  driver tx rings after the socket is closed (e.g. from tcpdrop).

  In order for m_free to work reliably, add a new CSUM_SND_TAG flag in
  csum_flags to indicate 'snd_tag' is set (rather than 'rcvif').
  Drivers now also check this flag instead of checking snd_tag against
  NULL.  This avoids false positive matches when a forwarded packet
  has a non-NULL rcvif that was treated as a send tag.

- cxgbe was relying on snd_tag_free being called when the inp was
  detached so that it could kick the firmware to flush any pending
  work on the flow.  This is because the driver doesn't require ACK
  messages from the firmware for every request, but instead does a
  kind of manual interrupt coalescing by only setting a flag to
  request a completion on a subset of requests.  If all of the
  in-flight requests don't have the flag when the tag is detached from
  the inp, the flow might never return the credits.  The current
  snd_tag_free command issues a flush command to force the credits to
  return.  However, the credit return is what also frees the mbufs,
  and since those mbufs now hold references on the tag, this meant
  that snd_tag_free would never be called.

  To fix, explicitly drop the mbuf's reference on the snd tag when the
  mbuf is queued in the firmware work queue.  This means that once the
  inp's reference on the tag goes away and all in-flight mbufs have
  been queued to the firmware, tag's refcount will drop to zero and
  snd_tag_free will kick in and send the flush request.  Note that we
  need to avoid doing this in the middle of ethofld_tx(), so the
  driver grabs a temporary reference on the tag around that loop to
  defer the free to the end of the function in case it sends the last
  mbuf to the queue after the inp has dropped its reference on the
  tag.

- mlx5 preallocates send tags and was using the ifp pointer even when
  the send tag wasn't in use.  Explicitly use the ifp from other data
  structures instead.

- Sprinkle some assertions in various places to assert that received
  packets don't have a send tag, and that other places that overwrite
  rcvif (e.g. 802.11 transmit) don't clobber a send tag pointer.

Reviewed by:	gallatin, hselasky, rgrimes, ae
Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D20117
2019-05-24 22:30:40 +00:00
..
dest6.c Remove some unneccessary variable sets in IPv6 code, as detected by 2018-03-24 12:43:34 +00:00
frag6.c Revert r346530 until further. 2019-04-22 19:36:19 +00:00
icmp6.c Fix hostname to be returned in an ICMPv6 NI Reply message defined 2019-05-16 19:09:41 +00:00
icmp6.h
in6_cksum.c sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
in6_fib.c Switch RIB and RADIX_NODE_HEAD lock from rwlock(9) to rmlock(9). 2018-06-16 08:26:23 +00:00
in6_fib.h Renumber copyright clause 4 2017-02-28 23:42:47 +00:00
in6_gif.c Add the check that current VNET is ready and access to srchash is allowed. 2018-10-23 13:11:45 +00:00
in6_ifattach.c poib: assign link-local address according to RFC 2019-04-23 12:23:44 +00:00
in6_ifattach.h sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
in6_jail.c Move most of the contents of opt_compat.h to opt_global.h. 2018-04-06 17:35:35 +00:00
in6_mcast.c Revert r347582 for now. 2019-05-16 13:04:26 +00:00
in6_pcb.c Mechanical cleanup of epoch(9) usage in network stack. 2019-01-09 01:11:19 +00:00
in6_pcb.h sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
in6_pcbgroup.c sys: general adoption of SPDX licensing ID tags. 2017-11-27 15:23:17 +00:00
in6_proto.c Update for IETF draft-ietf-6man-ipv6only-flag. 2019-03-06 23:31:42 +00:00
in6_rmx.c Use the new VNET_DEFINE_STATIC macro when we are defining static VNET 2018-07-24 16:35:52 +00:00
in6_rss.c
in6_rss.h
in6_src.c Restore IPV6_NEXTHOP option support that seem was partially broken 2019-05-24 11:45:32 +00:00
in6_var.h Fix refcounting leaks in IPv6 MLD code leading to loss of IPv6 2019-01-24 08:34:13 +00:00
in6.c Do not perform DAD on stf(4) interfaces. 2019-03-30 18:00:44 +00:00
in6.h Implement a limit on on the number of IPv6 reassembly queues per bucket. 2018-08-14 17:27:41 +00:00
ip6_ecn.h sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
ip6_fastfwd.c New pfil(9) KPI together with newborn pfil API and control utility. 2019-01-31 23:01:03 +00:00
ip6_forward.c New pfil(9) KPI together with newborn pfil API and control utility. 2019-01-31 23:01:03 +00:00
ip6_gre.c Add GRE-in-UDP encapsulation support as defined in RFC8086. 2019-04-24 09:05:45 +00:00
ip6_id.c ip6_randomflowlabel: Avoid blocking if random(4) is not available 2019-04-23 17:18:20 +00:00
ip6_input.c New pfil(9) KPI together with newborn pfil API and control utility. 2019-01-31 23:01:03 +00:00
ip6_mroute.c Plug some networking sysctl leaks. 2018-11-22 20:49:41 +00:00
ip6_mroute.h sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
ip6_output.c Restructure mbuf send tags to provide stronger guarantees. 2019-05-24 22:30:40 +00:00
ip6_var.h Add stat counter for ipv6 atomic fragments 2019-04-19 17:06:43 +00:00
ip6.h
ip6protosw.h sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
ip_fw_nat64.h Reapply r345274 with build fixes for 32-bit architectures. 2019-03-19 10:57:03 +00:00
ip_fw_nptv6.h Add ability to use dynamic external prefix in ipfw_nptv6 module. 2018-11-12 11:20:59 +00:00
mld6_var.h Fix refcounting leaks in IPv6 MLD code leading to loss of IPv6 2019-01-24 08:34:13 +00:00
mld6.c In mld_v2_cancel_link_timers() check number of references and disconnect 2019-05-09 07:57:33 +00:00
mld6.h sys: general adoption of SPDX licensing ID tags. 2017-11-27 15:23:17 +00:00
nd6_nbr.c Extract eventfilter declarations to sys/_eventfilter.h 2019-05-20 00:38:23 +00:00
nd6_rtr.c Fix gateway setup for the interface routes. 2019-05-22 21:20:15 +00:00
nd6.c Extract eventfilter declarations to sys/_eventfilter.h 2019-05-20 00:38:23 +00:00
nd6.h Update for IETF draft-ietf-6man-ipv6only-flag. 2019-03-07 23:03:39 +00:00
pim6_var.h Rework IP encapsulation handling code. 2018-06-05 20:51:01 +00:00
pim6.h sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
raw_ip6.c When an IPv6 packet is received for a raw socket which has the 2019-04-19 18:09:37 +00:00
raw_ip6.h sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
route6.c sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
scope6_var.h Constify argument of in6_getscope(). 2018-06-05 20:54:29 +00:00
scope6.c Mechanical cleanup of epoch(9) usage in network stack. 2019-01-09 01:11:19 +00:00
sctp6_usrreq.c Use a macro to set the assoc state. I missed this in r337706. 2018-08-14 08:33:47 +00:00
sctp6_var.h Whitespace changes due to changes in ident. 2018-07-19 20:16:33 +00:00
send.c Use the new VNET_DEFINE_STATIC macro when we are defining static VNET 2018-07-24 16:35:52 +00:00
send.h sys: general adoption of SPDX licensing ID tags. 2017-11-27 15:23:17 +00:00
tcp6_var.h sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
udp6_usrreq.c There are three places where we return from a function which entered an 2018-10-09 13:26:06 +00:00
udp6_var.h sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00