2005-01-07 02:30:35 +00:00
|
|
|
/*-
|
2017-11-20 19:43:44 +00:00
|
|
|
* SPDX-License-Identifier: BSD-3-Clause
|
|
|
|
*
|
1999-12-07 17:39:16 +00:00
|
|
|
* Copyright (C) 1995, 1996, 1997, and 1998 WIDE Project.
|
Decompose the current single inpcbinfo lock into two locks:
- The existing ipi_lock continues to protect the global inpcb list and
inpcb counter. This lock is now relegated to a small number of
allocation and free operations, and occasional operations that walk
all connections (including, awkwardly, certain UDP multicast receive
operations -- something to revisit).
- A new ipi_hash_lock protects the two inpcbinfo hash tables for
looking up connections and bound sockets, manipulated using new
INP_HASH_*() macros. This lock, combined with inpcb locks, protects
the 4-tuple address space.
Unlike the current ipi_lock, ipi_hash_lock follows the individual inpcb
connection locks, so may be acquired while manipulating a connection on
which a lock is already held, avoiding the need to acquire the inpcbinfo
lock preemptively when a binding change might later be required. As a
result, however, lookup operations necessarily go through a reference
acquire while holding the lookup lock, later acquiring an inpcb lock --
if required.
A new function in_pcblookup() looks up connections, and accepts flags
indicating how to return the inpcb. Due to lock order changes, callers
no longer need acquire locks before performing a lookup: the lookup
routine will acquire the ipi_hash_lock as needed. In the future, it will
also be able to use alternative lookup and locking strategies
transparently to callers, such as pcbgroup lookup. New lookup flags are,
supplementing the existing INPLOOKUP_WILDCARD flag:
INPLOOKUP_RLOCKPCB - Acquire a read lock on the returned inpcb
INPLOOKUP_WLOCKPCB - Acquire a write lock on the returned inpcb
Callers must pass exactly one of these flags (for the time being).
Some notes:
- All protocols are updated to work within the new regime; especially,
TCP, UDPv4, and UDPv6. pcbinfo ipi_lock acquisitions are largely
eliminated, and global hash lock hold times are dramatically reduced
compared to previous locking.
- The TCP syncache still relies on the pcbinfo lock, something that we
may want to revisit.
- Support for reverting to the FreeBSD 7.x locking strategy in TCP input
is no longer available -- hash lookup locks are now held only very
briefly during inpcb lookup, rather than for potentially extended
periods. However, the pcbinfo ipi_lock will still be acquired if a
connection state might change such that a connection is added or
removed.
- Raw IP sockets continue to use the pcbinfo ipi_lock for protection,
due to maintaining their own hash tables.
- The interface in6_pcblookup_hash_locked() is maintained, which allows
callers to acquire hash locks and perform one or more lookups atomically
with 4-tuple allocation: this is required only for TCPv6, as there is no
in6_pcbconnect_setup(), which there should be.
- UDPv6 locking remains significantly more conservative than UDPv4
locking, which relates to source address selection. This needs
attention, as it likely significantly reduces parallelism in this code
for multithreaded socket use (such as in BIND).
- In the UDPv4 and UDPv6 multicast cases, we need to revisit locking
somewhat, as they relied on ipi_lock to stablise 4-tuple matches, which
is no longer sufficient. A second check once the inpcb lock is held
should do the trick, keeping the general case from requiring the inpcb
lock for every inpcb visited.
- This work reminds us that we need to revisit locking of the v4/v6 flags,
which may be accessed lock-free both before and after this change.
- Right now, a single lock name is used for the pcbhash lock -- this is
undesirable, and probably another argument is required to take care of
this (or a char array name field in the pcbinfo?).
This is not an MFC candidate for 8.x due to its impact on lookup and
locking semantics. It's possible some of these issues could be worked
around with compatibility wrappers, if necessary.
Reviewed by: bz
Sponsored by: Juniper Networks, Inc.
2011-05-30 09:43:55 +00:00
|
|
|
* Copyright (c) 2010-2011 Juniper Networks, Inc.
|
2014-04-07 01:53:03 +00:00
|
|
|
* Copyright (c) 2014 Kevin Lo
|
1999-12-07 17:39:16 +00:00
|
|
|
* All rights reserved.
|
|
|
|
*
|
Decompose the current single inpcbinfo lock into two locks:
- The existing ipi_lock continues to protect the global inpcb list and
inpcb counter. This lock is now relegated to a small number of
allocation and free operations, and occasional operations that walk
all connections (including, awkwardly, certain UDP multicast receive
operations -- something to revisit).
- A new ipi_hash_lock protects the two inpcbinfo hash tables for
looking up connections and bound sockets, manipulated using new
INP_HASH_*() macros. This lock, combined with inpcb locks, protects
the 4-tuple address space.
Unlike the current ipi_lock, ipi_hash_lock follows the individual inpcb
connection locks, so may be acquired while manipulating a connection on
which a lock is already held, avoiding the need to acquire the inpcbinfo
lock preemptively when a binding change might later be required. As a
result, however, lookup operations necessarily go through a reference
acquire while holding the lookup lock, later acquiring an inpcb lock --
if required.
A new function in_pcblookup() looks up connections, and accepts flags
indicating how to return the inpcb. Due to lock order changes, callers
no longer need acquire locks before performing a lookup: the lookup
routine will acquire the ipi_hash_lock as needed. In the future, it will
also be able to use alternative lookup and locking strategies
transparently to callers, such as pcbgroup lookup. New lookup flags are,
supplementing the existing INPLOOKUP_WILDCARD flag:
INPLOOKUP_RLOCKPCB - Acquire a read lock on the returned inpcb
INPLOOKUP_WLOCKPCB - Acquire a write lock on the returned inpcb
Callers must pass exactly one of these flags (for the time being).
Some notes:
- All protocols are updated to work within the new regime; especially,
TCP, UDPv4, and UDPv6. pcbinfo ipi_lock acquisitions are largely
eliminated, and global hash lock hold times are dramatically reduced
compared to previous locking.
- The TCP syncache still relies on the pcbinfo lock, something that we
may want to revisit.
- Support for reverting to the FreeBSD 7.x locking strategy in TCP input
is no longer available -- hash lookup locks are now held only very
briefly during inpcb lookup, rather than for potentially extended
periods. However, the pcbinfo ipi_lock will still be acquired if a
connection state might change such that a connection is added or
removed.
- Raw IP sockets continue to use the pcbinfo ipi_lock for protection,
due to maintaining their own hash tables.
- The interface in6_pcblookup_hash_locked() is maintained, which allows
callers to acquire hash locks and perform one or more lookups atomically
with 4-tuple allocation: this is required only for TCPv6, as there is no
in6_pcbconnect_setup(), which there should be.
- UDPv6 locking remains significantly more conservative than UDPv4
locking, which relates to source address selection. This needs
attention, as it likely significantly reduces parallelism in this code
for multithreaded socket use (such as in BIND).
- In the UDPv4 and UDPv6 multicast cases, we need to revisit locking
somewhat, as they relied on ipi_lock to stablise 4-tuple matches, which
is no longer sufficient. A second check once the inpcb lock is held
should do the trick, keeping the general case from requiring the inpcb
lock for every inpcb visited.
- This work reminds us that we need to revisit locking of the v4/v6 flags,
which may be accessed lock-free both before and after this change.
- Right now, a single lock name is used for the pcbhash lock -- this is
undesirable, and probably another argument is required to take care of
this (or a char array name field in the pcbinfo?).
This is not an MFC candidate for 8.x due to its impact on lookup and
locking semantics. It's possible some of these issues could be worked
around with compatibility wrappers, if necessary.
Reviewed by: bz
Sponsored by: Juniper Networks, Inc.
2011-05-30 09:43:55 +00:00
|
|
|
* Portions of this software were developed by Robert N. M. Watson under
|
|
|
|
* contract to Juniper Networks, Inc.
|
|
|
|
*
|
1999-12-07 17:39:16 +00:00
|
|
|
* Redistribution and use in source and binary forms, with or without
|
|
|
|
* modification, are permitted provided that the following conditions
|
|
|
|
* are met:
|
|
|
|
* 1. Redistributions of source code must retain the above copyright
|
|
|
|
* notice, this list of conditions and the following disclaimer.
|
|
|
|
* 2. Redistributions in binary form must reproduce the above copyright
|
|
|
|
* notice, this list of conditions and the following disclaimer in the
|
|
|
|
* documentation and/or other materials provided with the distribution.
|
|
|
|
* 3. Neither the name of the project nor the names of its contributors
|
|
|
|
* may be used to endorse or promote products derived from this software
|
|
|
|
* without specific prior written permission.
|
|
|
|
*
|
|
|
|
* THIS SOFTWARE IS PROVIDED BY THE PROJECT AND CONTRIBUTORS ``AS IS'' AND
|
|
|
|
* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
|
|
|
|
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
|
|
|
|
* ARE DISCLAIMED. IN NO EVENT SHALL THE PROJECT OR CONTRIBUTORS BE LIABLE
|
|
|
|
* FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
|
|
|
|
* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
|
|
|
|
* OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
|
|
|
|
* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
|
|
|
|
* LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
|
|
|
|
* OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
|
|
|
|
* SUCH DAMAGE.
|
2007-12-10 16:03:40 +00:00
|
|
|
*
|
|
|
|
* $KAME: udp6_usrreq.c,v 1.27 2001/05/21 05:45:10 jinmei Exp $
|
|
|
|
* $KAME: udp6_output.c,v 1.31 2001/05/21 16:39:15 jinmei Exp $
|
1999-12-07 17:39:16 +00:00
|
|
|
*/
|
|
|
|
|
2005-01-07 02:30:35 +00:00
|
|
|
/*-
|
2007-09-08 08:18:24 +00:00
|
|
|
* Copyright (c) 1982, 1986, 1988, 1990, 1993, 1995
|
2007-07-09 17:47:04 +00:00
|
|
|
* The Regents of the University of California.
|
|
|
|
* All rights reserved.
|
1999-12-07 17:39:16 +00:00
|
|
|
*
|
|
|
|
* Redistribution and use in source and binary forms, with or without
|
|
|
|
* modification, are permitted provided that the following conditions
|
|
|
|
* are met:
|
|
|
|
* 1. Redistributions of source code must retain the above copyright
|
|
|
|
* notice, this list of conditions and the following disclaimer.
|
|
|
|
* 2. Redistributions in binary form must reproduce the above copyright
|
|
|
|
* notice, this list of conditions and the following disclaimer in the
|
|
|
|
* documentation and/or other materials provided with the distribution.
|
2017-02-28 23:42:47 +00:00
|
|
|
* 3. Neither the name of the University nor the names of its contributors
|
1999-12-07 17:39:16 +00:00
|
|
|
* may be used to endorse or promote products derived from this software
|
|
|
|
* without specific prior written permission.
|
|
|
|
*
|
|
|
|
* THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND
|
|
|
|
* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
|
|
|
|
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
|
|
|
|
* ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE
|
|
|
|
* FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
|
|
|
|
* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
|
|
|
|
* OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
|
|
|
|
* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
|
|
|
|
* LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
|
|
|
|
* OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
|
|
|
|
* SUCH DAMAGE.
|
|
|
|
*
|
2007-09-08 08:18:24 +00:00
|
|
|
* @(#)udp_usrreq.c 8.6 (Berkeley) 5/23/95
|
1999-12-07 17:39:16 +00:00
|
|
|
*/
|
|
|
|
|
2007-12-10 16:03:40 +00:00
|
|
|
#include <sys/cdefs.h>
|
2000-07-04 16:35:15 +00:00
|
|
|
#include "opt_inet.h"
|
|
|
|
#include "opt_inet6.h"
|
1999-12-22 19:13:38 +00:00
|
|
|
#include "opt_ipsec.h"
|
2020-10-18 17:15:47 +00:00
|
|
|
#include "opt_route.h"
|
2014-09-09 04:20:53 +00:00
|
|
|
#include "opt_rss.h"
|
1999-12-22 19:13:38 +00:00
|
|
|
|
1999-12-07 17:39:16 +00:00
|
|
|
#include <sys/param.h>
|
MFp4:
Bring in updated jail support from bz_jail branch.
This enhances the current jail implementation to permit multiple
addresses per jail. In addtion to IPv4, IPv6 is supported as well.
Due to updated checks it is even possible to have jails without
an IP address at all, which basically gives one a chroot with
restricted process view, no networking,..
SCTP support was updated and supports IPv6 in jails as well.
Cpuset support permits jails to be bound to specific processor
sets after creation.
Jails can have an unrestricted (no duplicate protection, etc.) name
in addition to the hostname. The jail name cannot be changed from
within a jail and is considered to be used for management purposes
or as audit-token in the future.
DDB 'show jails' command was added to aid debugging.
Proper compat support permits 32bit jail binaries to be used on 64bit
systems to manage jails. Also backward compatibility was preserved where
possible: for jail v1 syscalls, as well as with user space management
utilities.
Both jail as well as prison version were updated for the new features.
A gap was intentionally left as the intermediate versions had been
used by various patches floating around the last years.
Bump __FreeBSD_version for the afore mentioned and in kernel changes.
Special thanks to:
- Pawel Jakub Dawidek (pjd) for his multi-IPv4 patches
and Olivier Houchard (cognet) for initial single-IPv6 patches.
- Jeff Roberson (jeff) and Randall Stewart (rrs) for their
help, ideas and review on cpuset and SCTP support.
- Robert Watson (rwatson) for lots and lots of help, discussions,
suggestions and review of most of the patch at various stages.
- John Baldwin (jhb) for his help.
- Simon L. Nielsen (simon) as early adopter testing changes
on cluster machines as well as all the testers and people
who provided feedback the last months on freebsd-jail and
other channels.
- My employer, CK Software GmbH, for the support so I could work on this.
Reviewed by: (see above)
MFC after: 3 months (this is just so that I get the mail)
X-MFC Before: 7.2-RELEASE if possible
2008-11-29 14:32:14 +00:00
|
|
|
#include <sys/jail.h>
|
1999-12-07 17:39:16 +00:00
|
|
|
#include <sys/kernel.h>
|
2002-04-30 01:54:54 +00:00
|
|
|
#include <sys/lock.h>
|
1999-12-07 17:39:16 +00:00
|
|
|
#include <sys/mbuf.h>
|
2006-11-06 13:42:10 +00:00
|
|
|
#include <sys/priv.h>
|
2002-04-30 01:54:54 +00:00
|
|
|
#include <sys/proc.h>
|
1999-12-07 17:39:16 +00:00
|
|
|
#include <sys/protosw.h>
|
2013-08-25 21:54:41 +00:00
|
|
|
#include <sys/sdt.h>
|
2002-04-30 01:54:54 +00:00
|
|
|
#include <sys/signalvar.h>
|
1999-12-07 17:39:16 +00:00
|
|
|
#include <sys/socket.h>
|
|
|
|
#include <sys/socketvar.h>
|
2002-04-30 01:54:54 +00:00
|
|
|
#include <sys/sx.h>
|
|
|
|
#include <sys/sysctl.h>
|
1999-12-07 17:39:16 +00:00
|
|
|
#include <sys/syslog.h>
|
2002-04-30 01:54:54 +00:00
|
|
|
#include <sys/systm.h>
|
1999-12-07 17:39:16 +00:00
|
|
|
|
|
|
|
#include <net/if.h>
|
2013-10-26 17:58:36 +00:00
|
|
|
#include <net/if_var.h>
|
1999-12-07 17:39:16 +00:00
|
|
|
#include <net/if_types.h>
|
2002-04-30 01:54:54 +00:00
|
|
|
#include <net/route.h>
|
2015-01-18 18:06:40 +00:00
|
|
|
#include <net/rss_config.h>
|
1999-12-07 17:39:16 +00:00
|
|
|
|
|
|
|
#include <netinet/in.h>
|
2013-08-25 21:54:41 +00:00
|
|
|
#include <netinet/in_kdtrace.h>
|
1999-12-07 17:39:16 +00:00
|
|
|
#include <netinet/in_pcb.h>
|
2002-04-30 01:54:54 +00:00
|
|
|
#include <netinet/in_systm.h>
|
1999-12-07 17:39:16 +00:00
|
|
|
#include <netinet/in_var.h>
|
2002-04-30 01:54:54 +00:00
|
|
|
#include <netinet/ip.h>
|
|
|
|
#include <netinet/ip6.h>
|
|
|
|
#include <netinet/icmp6.h>
|
1999-12-07 17:39:16 +00:00
|
|
|
#include <netinet/ip_var.h>
|
|
|
|
#include <netinet/udp.h>
|
|
|
|
#include <netinet/udp_var.h>
|
2014-04-07 01:53:03 +00:00
|
|
|
#include <netinet/udplite.h>
|
2008-12-02 21:37:28 +00:00
|
|
|
|
1999-12-07 17:39:16 +00:00
|
|
|
#include <netinet6/ip6_var.h>
|
2020-10-18 17:15:47 +00:00
|
|
|
#include <netinet6/in6_fib.h>
|
1999-12-07 17:39:16 +00:00
|
|
|
#include <netinet6/in6_pcb.h>
|
2015-01-18 18:06:40 +00:00
|
|
|
#include <netinet6/in6_rss.h>
|
1999-12-07 17:39:16 +00:00
|
|
|
#include <netinet6/udp6_var.h>
|
2005-07-25 12:31:43 +00:00
|
|
|
#include <netinet6/scope6_var.h>
|
1999-12-07 17:39:16 +00:00
|
|
|
|
2017-02-06 08:49:57 +00:00
|
|
|
#include <netipsec/ipsec_support.h>
|
2002-10-16 02:25:05 +00:00
|
|
|
|
2007-07-19 22:34:25 +00:00
|
|
|
#include <security/mac/mac_framework.h>
|
|
|
|
|
2020-09-18 02:21:15 +00:00
|
|
|
VNET_DEFINE(int, zero_checksum_port) = 0;
|
|
|
|
#define V_zero_checksum_port VNET(zero_checksum_port)
|
|
|
|
SYSCTL_INT(_net_inet6_udp6, OID_AUTO, rfc6935_port, CTLFLAG_VNET | CTLFLAG_RW,
|
|
|
|
&VNET_NAME(zero_checksum_port), 0,
|
|
|
|
"Zero UDP checksum allowed for traffic to/from this port.");
|
2022-08-16 19:40:36 +00:00
|
|
|
|
1999-12-07 17:39:16 +00:00
|
|
|
/*
|
2007-09-08 08:18:24 +00:00
|
|
|
* UDP protocol implementation.
|
1999-12-07 17:39:16 +00:00
|
|
|
* Per RFC 768, August, 1980.
|
|
|
|
*/
|
|
|
|
|
2007-07-09 17:47:04 +00:00
|
|
|
static void udp6_detach(struct socket *so);
|
1999-12-07 17:39:16 +00:00
|
|
|
|
2015-07-21 09:54:31 +00:00
|
|
|
static int
|
2007-09-08 08:18:24 +00:00
|
|
|
udp6_append(struct inpcb *inp, struct mbuf *n, int off,
|
2006-05-01 21:39:48 +00:00
|
|
|
struct sockaddr_in6 *fromsa)
|
|
|
|
{
|
|
|
|
struct socket *so;
|
2017-03-06 04:01:58 +00:00
|
|
|
struct mbuf *opts = NULL, *tmp_opts;
|
2014-10-09 05:42:07 +00:00
|
|
|
struct udpcb *up;
|
2022-04-22 14:11:31 +00:00
|
|
|
bool filtered;
|
2006-05-01 21:39:48 +00:00
|
|
|
|
2008-04-22 12:20:33 +00:00
|
|
|
INP_LOCK_ASSERT(inp);
|
2006-05-01 21:39:48 +00:00
|
|
|
|
2014-10-09 05:42:07 +00:00
|
|
|
/*
|
|
|
|
* Engage the tunneling protocol.
|
|
|
|
*/
|
|
|
|
up = intoudpcb(inp);
|
|
|
|
if (up->u_tun_func != NULL) {
|
2015-07-21 09:54:31 +00:00
|
|
|
in_pcbref(inp);
|
|
|
|
INP_RUNLOCK(inp);
|
2022-04-22 14:11:31 +00:00
|
|
|
filtered = (*up->u_tun_func)(n, off, inp,
|
|
|
|
(struct sockaddr *)&fromsa[0], up->u_tun_ctx);
|
2015-07-21 09:54:31 +00:00
|
|
|
INP_RLOCK(inp);
|
2022-04-22 14:11:31 +00:00
|
|
|
if (filtered)
|
|
|
|
return (in_pcbrele_rlocked(inp));
|
2014-10-09 05:42:07 +00:00
|
|
|
}
|
2017-02-06 08:49:57 +00:00
|
|
|
#if defined(IPSEC) || defined(IPSEC_SUPPORT)
|
2007-07-09 17:47:04 +00:00
|
|
|
/* Check AH/ESP integrity. */
|
2017-02-06 08:49:57 +00:00
|
|
|
if (IPSEC_ENABLED(ipv6)) {
|
|
|
|
if (IPSEC_CHECK_POLICY(ipv6, n, inp) != 0) {
|
|
|
|
m_freem(n);
|
|
|
|
return (0);
|
|
|
|
}
|
2006-05-01 21:39:48 +00:00
|
|
|
}
|
2007-07-03 12:13:45 +00:00
|
|
|
#endif /* IPSEC */
|
2007-07-19 22:34:25 +00:00
|
|
|
#ifdef MAC
|
2007-10-24 19:04:04 +00:00
|
|
|
if (mac_inpcb_check_deliver(inp, n) != 0) {
|
2007-07-19 22:34:25 +00:00
|
|
|
m_freem(n);
|
2015-07-21 09:54:31 +00:00
|
|
|
return (0);
|
2007-07-19 22:34:25 +00:00
|
|
|
}
|
|
|
|
#endif
|
2006-05-01 21:39:48 +00:00
|
|
|
opts = NULL;
|
2008-12-17 13:00:18 +00:00
|
|
|
if (inp->inp_flags & INP_CONTROLOPTS ||
|
2007-09-08 08:18:24 +00:00
|
|
|
inp->inp_socket->so_options & SO_TIMESTAMP)
|
|
|
|
ip6_savecontrol(inp, n, &opts);
|
2017-03-06 04:01:58 +00:00
|
|
|
if ((inp->inp_vflag & INP_IPV6) && (inp->inp_flags2 & INP_ORIGDSTADDR)) {
|
2022-05-17 17:10:42 +00:00
|
|
|
tmp_opts = sbcreatecontrol(&fromsa[1],
|
|
|
|
sizeof(struct sockaddr_in6), IPV6_ORIGDSTADDR,
|
|
|
|
IPPROTO_IPV6, M_NOWAIT);
|
2017-03-06 04:01:58 +00:00
|
|
|
if (tmp_opts) {
|
|
|
|
if (opts) {
|
|
|
|
tmp_opts->m_next = opts;
|
|
|
|
opts = tmp_opts;
|
|
|
|
} else
|
|
|
|
opts = tmp_opts;
|
|
|
|
}
|
|
|
|
}
|
2006-05-01 21:39:48 +00:00
|
|
|
m_adj(n, off + sizeof(struct udphdr));
|
|
|
|
|
2007-09-08 08:18:24 +00:00
|
|
|
so = inp->inp_socket;
|
2006-05-01 21:39:48 +00:00
|
|
|
SOCKBUF_LOCK(&so->so_rcv);
|
2017-03-06 04:01:58 +00:00
|
|
|
if (sbappendaddr_locked(&so->so_rcv, (struct sockaddr *)&fromsa[0], n,
|
2006-05-01 21:39:48 +00:00
|
|
|
opts) == 0) {
|
2021-07-28 15:46:59 +00:00
|
|
|
soroverflow_locked(so);
|
2006-05-01 21:39:48 +00:00
|
|
|
m_freem(n);
|
|
|
|
if (opts)
|
|
|
|
m_freem(opts);
|
2009-04-12 11:53:12 +00:00
|
|
|
UDPSTAT_INC(udps_fullsock);
|
2006-05-01 21:39:48 +00:00
|
|
|
} else
|
|
|
|
sorwakeup_locked(so);
|
2015-07-21 09:54:31 +00:00
|
|
|
return (0);
|
2006-05-01 21:39:48 +00:00
|
|
|
}
|
|
|
|
|
2021-12-02 22:45:04 +00:00
|
|
|
struct udp6_multi_match_ctx {
|
|
|
|
struct ip6_hdr *ip6;
|
|
|
|
struct udphdr *uh;
|
|
|
|
};
|
|
|
|
|
|
|
|
static bool
|
|
|
|
udp6_multi_match(const struct inpcb *inp, void *v)
|
|
|
|
{
|
|
|
|
struct udp6_multi_match_ctx *ctx = v;
|
|
|
|
|
|
|
|
if ((inp->inp_vflag & INP_IPV6) == 0)
|
|
|
|
return(false);
|
|
|
|
if (inp->inp_lport != ctx->uh->uh_dport)
|
|
|
|
return(false);
|
|
|
|
if (inp->inp_fport != 0 && inp->inp_fport != ctx->uh->uh_sport)
|
|
|
|
return(false);
|
|
|
|
if (!IN6_IS_ADDR_UNSPECIFIED(&inp->in6p_laddr) &&
|
|
|
|
!IN6_ARE_ADDR_EQUAL(&inp->in6p_laddr, &ctx->ip6->ip6_dst))
|
|
|
|
return (false);
|
|
|
|
if (!IN6_IS_ADDR_UNSPECIFIED(&inp->in6p_faddr) &&
|
|
|
|
(!IN6_ARE_ADDR_EQUAL(&inp->in6p_faddr, &ctx->ip6->ip6_src) ||
|
|
|
|
inp->inp_fport != ctx->uh->uh_sport))
|
|
|
|
return (false);
|
|
|
|
|
|
|
|
return (true);
|
|
|
|
}
|
|
|
|
|
|
|
|
static int
|
|
|
|
udp6_multi_input(struct mbuf *m, int off, int proto,
|
|
|
|
struct sockaddr_in6 *fromsa)
|
|
|
|
{
|
|
|
|
struct udp6_multi_match_ctx ctx;
|
|
|
|
struct inpcb_iterator inpi = INP_ITERATOR(udp_get_inpcbinfo(proto),
|
|
|
|
INPLOOKUP_RLOCKPCB, udp6_multi_match, &ctx);
|
|
|
|
struct inpcb *inp;
|
|
|
|
struct ip6_moptions *imo;
|
|
|
|
struct mbuf *n;
|
|
|
|
int appends = 0;
|
|
|
|
|
|
|
|
/*
|
|
|
|
* In the event that laddr should be set to the link-local
|
|
|
|
* address (this happens in RIPng), the multicast address
|
|
|
|
* specified in the received packet will not match laddr. To
|
|
|
|
* handle this situation, matching is relaxed if the
|
|
|
|
* receiving interface is the same as one specified in the
|
|
|
|
* socket and if the destination multicast address matches
|
|
|
|
* one of the multicast groups specified in the socket.
|
|
|
|
*/
|
|
|
|
|
|
|
|
/*
|
|
|
|
* KAME note: traditionally we dropped udpiphdr from mbuf
|
|
|
|
* here. We need udphdr for IPsec processing so we do that
|
|
|
|
* later.
|
|
|
|
*/
|
|
|
|
ctx.ip6 = mtod(m, struct ip6_hdr *);
|
|
|
|
ctx.uh = (struct udphdr *)((char *)ctx.ip6 + off);
|
|
|
|
while ((inp = inp_next(&inpi)) != NULL) {
|
|
|
|
INP_RLOCK_ASSERT(inp);
|
|
|
|
/*
|
|
|
|
* XXXRW: Because we weren't holding either the inpcb
|
|
|
|
* or the hash lock when we checked for a match
|
|
|
|
* before, we should probably recheck now that the
|
|
|
|
* inpcb lock is (supposed to be) held.
|
|
|
|
*/
|
|
|
|
/*
|
|
|
|
* Handle socket delivery policy for any-source
|
|
|
|
* and source-specific multicast. [RFC3678]
|
|
|
|
*/
|
|
|
|
if ((imo = inp->in6p_moptions) != NULL) {
|
|
|
|
struct sockaddr_in6 mcaddr;
|
|
|
|
int blocked;
|
|
|
|
|
|
|
|
bzero(&mcaddr, sizeof(struct sockaddr_in6));
|
|
|
|
mcaddr.sin6_len = sizeof(struct sockaddr_in6);
|
|
|
|
mcaddr.sin6_family = AF_INET6;
|
|
|
|
mcaddr.sin6_addr = ctx.ip6->ip6_dst;
|
|
|
|
|
|
|
|
blocked = im6o_mc_filter(imo, m->m_pkthdr.rcvif,
|
|
|
|
(struct sockaddr *)&mcaddr,
|
|
|
|
(struct sockaddr *)&fromsa[0]);
|
|
|
|
if (blocked != MCAST_PASS) {
|
|
|
|
if (blocked == MCAST_NOTGMEMBER)
|
|
|
|
IP6STAT_INC(ip6s_notmember);
|
|
|
|
if (blocked == MCAST_NOTSMEMBER ||
|
|
|
|
blocked == MCAST_MUTED)
|
|
|
|
UDPSTAT_INC(udps_filtermcast);
|
|
|
|
continue;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
if ((n = m_copym(m, 0, M_COPYALL, M_NOWAIT)) != NULL) {
|
|
|
|
if (proto == IPPROTO_UDPLITE)
|
|
|
|
UDPLITE_PROBE(receive, NULL, inp, ctx.ip6,
|
|
|
|
inp, ctx.uh);
|
|
|
|
else
|
|
|
|
UDP_PROBE(receive, NULL, inp, ctx.ip6, inp,
|
|
|
|
ctx.uh);
|
|
|
|
if (udp6_append(inp, n, off, fromsa)) {
|
|
|
|
break;
|
|
|
|
} else
|
|
|
|
appends++;
|
|
|
|
}
|
|
|
|
/*
|
|
|
|
* Don't look for additional matches if this one does
|
|
|
|
* not have either the SO_REUSEPORT or SO_REUSEADDR
|
|
|
|
* socket options set. This heuristic avoids
|
|
|
|
* searching through all pcbs in the common case of a
|
|
|
|
* non-shared port. It assumes that an application
|
|
|
|
* will never clear these options after setting them.
|
|
|
|
*/
|
|
|
|
if ((inp->inp_socket->so_options &
|
|
|
|
(SO_REUSEPORT|SO_REUSEPORT_LB|SO_REUSEADDR)) == 0) {
|
|
|
|
INP_RUNLOCK(inp);
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
m_freem(m);
|
|
|
|
|
|
|
|
if (appends == 0) {
|
|
|
|
/*
|
|
|
|
* No matching pcb found; discard datagram. (No need
|
|
|
|
* to send an ICMP Port Unreachable for a broadcast
|
|
|
|
* or multicast datgram.)
|
|
|
|
*/
|
|
|
|
UDPSTAT_INC(udps_noport);
|
|
|
|
UDPSTAT_INC(udps_noportmcast);
|
|
|
|
}
|
|
|
|
|
|
|
|
return (IPPROTO_DONE);
|
|
|
|
}
|
|
|
|
|
1999-12-07 17:39:16 +00:00
|
|
|
int
|
2007-07-05 16:23:49 +00:00
|
|
|
udp6_input(struct mbuf **mp, int *offp, int proto)
|
1999-12-07 17:39:16 +00:00
|
|
|
{
|
2006-05-01 21:39:48 +00:00
|
|
|
struct mbuf *m = *mp;
|
2007-07-09 17:47:04 +00:00
|
|
|
struct ip6_hdr *ip6;
|
|
|
|
struct udphdr *uh;
|
2007-09-08 08:18:24 +00:00
|
|
|
struct inpcb *inp;
|
2014-04-07 01:53:03 +00:00
|
|
|
struct inpcbinfo *pcbinfo;
|
2009-05-23 16:51:13 +00:00
|
|
|
struct udpcb *up;
|
1999-12-07 17:39:16 +00:00
|
|
|
int off = *offp;
|
2014-04-07 01:53:03 +00:00
|
|
|
int cscov_partial;
|
1999-12-07 17:39:16 +00:00
|
|
|
int plen, ulen;
|
2017-03-06 04:01:58 +00:00
|
|
|
struct sockaddr_in6 fromsa[2];
|
2011-08-20 17:05:11 +00:00
|
|
|
struct m_tag *fwd_tag;
|
2012-05-25 02:19:17 +00:00
|
|
|
uint16_t uh_sum;
|
2014-04-07 01:53:03 +00:00
|
|
|
uint8_t nxt;
|
1999-12-07 17:39:16 +00:00
|
|
|
|
2019-11-07 20:38:53 +00:00
|
|
|
NET_EPOCH_ASSERT();
|
|
|
|
|
2019-12-01 00:22:04 +00:00
|
|
|
if (m->m_len < off + sizeof(struct udphdr)) {
|
|
|
|
m = m_pullup(m, off + sizeof(struct udphdr));
|
|
|
|
if (m == NULL) {
|
|
|
|
IP6STAT_INC(ip6s_exthdrtoolong);
|
|
|
|
*mp = NULL;
|
|
|
|
return (IPPROTO_DONE);
|
|
|
|
}
|
2019-11-15 21:40:40 +00:00
|
|
|
}
|
2018-03-24 12:43:34 +00:00
|
|
|
ip6 = mtod(m, struct ip6_hdr *);
|
2019-11-15 21:40:40 +00:00
|
|
|
uh = (struct udphdr *)((caddr_t)ip6 + off);
|
2004-02-13 15:11:47 +00:00
|
|
|
|
2009-04-12 11:53:12 +00:00
|
|
|
UDPSTAT_INC(udps_ipackets);
|
1999-12-07 17:39:16 +00:00
|
|
|
|
2007-07-19 22:34:25 +00:00
|
|
|
/*
|
|
|
|
* Destination port of 0 is illegal, based on RFC768.
|
|
|
|
*/
|
|
|
|
if (uh->uh_dport == 0)
|
|
|
|
goto badunlocked;
|
|
|
|
|
1999-12-07 17:39:16 +00:00
|
|
|
plen = ntohs(ip6->ip6_plen) - off + sizeof(*ip6);
|
|
|
|
ulen = ntohs((u_short)uh->uh_ulen);
|
|
|
|
|
2015-09-21 12:32:36 +00:00
|
|
|
nxt = proto;
|
2014-04-07 01:53:03 +00:00
|
|
|
cscov_partial = (nxt == IPPROTO_UDPLITE) ? 1 : 0;
|
2014-10-02 10:32:24 +00:00
|
|
|
if (nxt == IPPROTO_UDPLITE) {
|
2014-04-07 01:53:03 +00:00
|
|
|
/* Zero means checksum over the complete packet. */
|
2014-09-30 18:17:28 +00:00
|
|
|
if (ulen == 0)
|
|
|
|
ulen = plen;
|
2014-10-02 10:32:24 +00:00
|
|
|
if (ulen == plen)
|
|
|
|
cscov_partial = 0;
|
|
|
|
if ((ulen < sizeof(struct udphdr)) || (ulen > plen)) {
|
|
|
|
/* XXX: What is the right UDPLite MIB counter? */
|
|
|
|
goto badunlocked;
|
|
|
|
}
|
2014-10-03 08:46:49 +00:00
|
|
|
if (uh->uh_sum == 0) {
|
|
|
|
/* XXX: What is the right UDPLite MIB counter? */
|
|
|
|
goto badunlocked;
|
|
|
|
}
|
2014-10-02 10:49:01 +00:00
|
|
|
} else {
|
|
|
|
if ((ulen < sizeof(struct udphdr)) || (plen != ulen)) {
|
|
|
|
UDPSTAT_INC(udps_badlen);
|
2014-04-07 01:53:03 +00:00
|
|
|
goto badunlocked;
|
|
|
|
}
|
2014-10-03 08:46:49 +00:00
|
|
|
if (uh->uh_sum == 0) {
|
|
|
|
UDPSTAT_INC(udps_nosum);
|
2020-09-18 02:21:15 +00:00
|
|
|
/*
|
|
|
|
* dport 0 was rejected earlier so this is OK even if
|
|
|
|
* zero_checksum_port is 0 (which is its default value).
|
|
|
|
*/
|
|
|
|
if (ntohs(uh->uh_dport) == V_zero_checksum_port)
|
|
|
|
goto skip_checksum;
|
|
|
|
else
|
|
|
|
goto badunlocked;
|
2014-10-03 08:46:49 +00:00
|
|
|
}
|
2004-04-01 13:48:23 +00:00
|
|
|
}
|
2012-05-25 02:19:17 +00:00
|
|
|
|
2014-04-07 01:53:03 +00:00
|
|
|
if ((m->m_pkthdr.csum_flags & CSUM_DATA_VALID_IPV6) &&
|
|
|
|
!cscov_partial) {
|
2012-05-25 02:19:17 +00:00
|
|
|
if (m->m_pkthdr.csum_flags & CSUM_PSEUDO_HDR)
|
|
|
|
uh_sum = m->m_pkthdr.csum_data;
|
|
|
|
else
|
2014-04-07 01:53:03 +00:00
|
|
|
uh_sum = in6_cksum_pseudo(ip6, ulen, nxt,
|
|
|
|
m->m_pkthdr.csum_data);
|
2012-05-25 02:19:17 +00:00
|
|
|
uh_sum ^= 0xffff;
|
|
|
|
} else
|
2014-10-02 10:32:24 +00:00
|
|
|
uh_sum = in6_cksum_partial(m, nxt, off, plen, ulen);
|
2012-05-25 02:19:17 +00:00
|
|
|
|
|
|
|
if (uh_sum != 0) {
|
2009-04-12 11:53:12 +00:00
|
|
|
UDPSTAT_INC(udps_badsum);
|
2014-10-07 16:01:17 +00:00
|
|
|
goto badunlocked;
|
1999-12-07 17:39:16 +00:00
|
|
|
}
|
|
|
|
|
2020-09-18 02:21:15 +00:00
|
|
|
skip_checksum:
|
2006-05-01 21:39:48 +00:00
|
|
|
/*
|
|
|
|
* Construct sockaddr format source address.
|
|
|
|
*/
|
2017-03-06 04:01:58 +00:00
|
|
|
init_sin6(&fromsa[0], m, 0);
|
|
|
|
fromsa[0].sin6_port = uh->uh_sport;
|
|
|
|
init_sin6(&fromsa[1], m, 1);
|
|
|
|
fromsa[1].sin6_port = uh->uh_dport;
|
2006-05-01 21:39:48 +00:00
|
|
|
|
2015-08-27 15:27:41 +00:00
|
|
|
pcbinfo = udp_get_inpcbinfo(nxt);
|
2021-12-02 22:45:04 +00:00
|
|
|
if (IN6_IS_ADDR_MULTICAST(&ip6->ip6_dst)) {
|
2019-11-12 15:46:28 +00:00
|
|
|
*mp = NULL;
|
2021-12-02 22:45:04 +00:00
|
|
|
return (udp6_multi_input(m, off, proto, fromsa));
|
1999-12-07 17:39:16 +00:00
|
|
|
}
|
2021-12-02 22:45:04 +00:00
|
|
|
|
1999-12-07 17:39:16 +00:00
|
|
|
/*
|
|
|
|
* Locate pcb for datagram.
|
|
|
|
*/
|
2012-10-25 09:39:14 +00:00
|
|
|
|
2011-08-20 17:05:11 +00:00
|
|
|
/*
|
|
|
|
* Grab info from PACKET_TAG_IPFORWARD tag prepended to the chain.
|
|
|
|
*/
|
2012-11-02 01:20:55 +00:00
|
|
|
if ((m->m_flags & M_IP6_NEXTHOP) &&
|
2012-10-25 09:39:14 +00:00
|
|
|
(fwd_tag = m_tag_find(m, PACKET_TAG_IPFORWARD, NULL)) != NULL) {
|
2011-08-20 17:05:11 +00:00
|
|
|
struct sockaddr_in6 *next_hop6;
|
|
|
|
|
|
|
|
next_hop6 = (struct sockaddr_in6 *)(fwd_tag + 1);
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Transparently forwarded. Pretend to be the destination.
|
|
|
|
* Already got one like this?
|
|
|
|
*/
|
2014-04-07 01:53:03 +00:00
|
|
|
inp = in6_pcblookup_mbuf(pcbinfo, &ip6->ip6_src,
|
|
|
|
uh->uh_sport, &ip6->ip6_dst, uh->uh_dport,
|
2011-08-20 17:05:11 +00:00
|
|
|
INPLOOKUP_RLOCKPCB, m->m_pkthdr.rcvif, m);
|
|
|
|
if (!inp) {
|
|
|
|
/*
|
|
|
|
* It's new. Try to find the ambushing socket.
|
|
|
|
* Because we've rewritten the destination address,
|
|
|
|
* any hardware-generated hash is ignored.
|
|
|
|
*/
|
2014-04-07 01:53:03 +00:00
|
|
|
inp = in6_pcblookup(pcbinfo, &ip6->ip6_src,
|
2011-08-20 17:05:11 +00:00
|
|
|
uh->uh_sport, &next_hop6->sin6_addr,
|
|
|
|
next_hop6->sin6_port ? htons(next_hop6->sin6_port) :
|
|
|
|
uh->uh_dport, INPLOOKUP_WILDCARD |
|
|
|
|
INPLOOKUP_RLOCKPCB, m->m_pkthdr.rcvif);
|
|
|
|
}
|
|
|
|
/* Remove the tag from the packet. We don't need it anymore. */
|
|
|
|
m_tag_delete(m, fwd_tag);
|
2012-11-02 01:20:55 +00:00
|
|
|
m->m_flags &= ~M_IP6_NEXTHOP;
|
2011-08-20 17:05:11 +00:00
|
|
|
} else
|
2014-04-07 01:53:03 +00:00
|
|
|
inp = in6_pcblookup_mbuf(pcbinfo, &ip6->ip6_src,
|
2011-08-20 17:05:11 +00:00
|
|
|
uh->uh_sport, &ip6->ip6_dst, uh->uh_dport,
|
|
|
|
INPLOOKUP_WILDCARD | INPLOOKUP_RLOCKPCB,
|
|
|
|
m->m_pkthdr.rcvif, m);
|
2007-09-08 08:18:24 +00:00
|
|
|
if (inp == NULL) {
|
2020-01-08 23:30:26 +00:00
|
|
|
if (V_udp_log_in_vain) {
|
2006-12-12 12:17:58 +00:00
|
|
|
char ip6bufs[INET6_ADDRSTRLEN];
|
|
|
|
char ip6bufd[INET6_ADDRSTRLEN];
|
1999-12-07 17:39:16 +00:00
|
|
|
|
|
|
|
log(LOG_INFO,
|
2002-08-19 19:47:13 +00:00
|
|
|
"Connection attempt to UDP [%s]:%d from [%s]:%d\n",
|
2006-12-12 12:17:58 +00:00
|
|
|
ip6_sprintf(ip6bufd, &ip6->ip6_dst),
|
|
|
|
ntohs(uh->uh_dport),
|
|
|
|
ip6_sprintf(ip6bufs, &ip6->ip6_src),
|
|
|
|
ntohs(uh->uh_sport));
|
1999-12-07 17:39:16 +00:00
|
|
|
}
|
2018-07-31 22:56:03 +00:00
|
|
|
if (nxt == IPPROTO_UDPLITE)
|
|
|
|
UDPLITE_PROBE(receive, NULL, NULL, ip6, NULL, uh);
|
|
|
|
else
|
|
|
|
UDP_PROBE(receive, NULL, NULL, ip6, NULL, uh);
|
2009-04-12 11:53:12 +00:00
|
|
|
UDPSTAT_INC(udps_noport);
|
1999-12-07 17:39:16 +00:00
|
|
|
if (m->m_flags & M_MCAST) {
|
|
|
|
printf("UDP6: M_MCAST is set in a unicast packet.\n");
|
2009-04-12 11:53:12 +00:00
|
|
|
UDPSTAT_INC(udps_noportmcast);
|
Decompose the current single inpcbinfo lock into two locks:
- The existing ipi_lock continues to protect the global inpcb list and
inpcb counter. This lock is now relegated to a small number of
allocation and free operations, and occasional operations that walk
all connections (including, awkwardly, certain UDP multicast receive
operations -- something to revisit).
- A new ipi_hash_lock protects the two inpcbinfo hash tables for
looking up connections and bound sockets, manipulated using new
INP_HASH_*() macros. This lock, combined with inpcb locks, protects
the 4-tuple address space.
Unlike the current ipi_lock, ipi_hash_lock follows the individual inpcb
connection locks, so may be acquired while manipulating a connection on
which a lock is already held, avoiding the need to acquire the inpcbinfo
lock preemptively when a binding change might later be required. As a
result, however, lookup operations necessarily go through a reference
acquire while holding the lookup lock, later acquiring an inpcb lock --
if required.
A new function in_pcblookup() looks up connections, and accepts flags
indicating how to return the inpcb. Due to lock order changes, callers
no longer need acquire locks before performing a lookup: the lookup
routine will acquire the ipi_hash_lock as needed. In the future, it will
also be able to use alternative lookup and locking strategies
transparently to callers, such as pcbgroup lookup. New lookup flags are,
supplementing the existing INPLOOKUP_WILDCARD flag:
INPLOOKUP_RLOCKPCB - Acquire a read lock on the returned inpcb
INPLOOKUP_WLOCKPCB - Acquire a write lock on the returned inpcb
Callers must pass exactly one of these flags (for the time being).
Some notes:
- All protocols are updated to work within the new regime; especially,
TCP, UDPv4, and UDPv6. pcbinfo ipi_lock acquisitions are largely
eliminated, and global hash lock hold times are dramatically reduced
compared to previous locking.
- The TCP syncache still relies on the pcbinfo lock, something that we
may want to revisit.
- Support for reverting to the FreeBSD 7.x locking strategy in TCP input
is no longer available -- hash lookup locks are now held only very
briefly during inpcb lookup, rather than for potentially extended
periods. However, the pcbinfo ipi_lock will still be acquired if a
connection state might change such that a connection is added or
removed.
- Raw IP sockets continue to use the pcbinfo ipi_lock for protection,
due to maintaining their own hash tables.
- The interface in6_pcblookup_hash_locked() is maintained, which allows
callers to acquire hash locks and perform one or more lookups atomically
with 4-tuple allocation: this is required only for TCPv6, as there is no
in6_pcbconnect_setup(), which there should be.
- UDPv6 locking remains significantly more conservative than UDPv4
locking, which relates to source address selection. This needs
attention, as it likely significantly reduces parallelism in this code
for multithreaded socket use (such as in BIND).
- In the UDPv4 and UDPv6 multicast cases, we need to revisit locking
somewhat, as they relied on ipi_lock to stablise 4-tuple matches, which
is no longer sufficient. A second check once the inpcb lock is held
should do the trick, keeping the general case from requiring the inpcb
lock for every inpcb visited.
- This work reminds us that we need to revisit locking of the v4/v6 flags,
which may be accessed lock-free both before and after this change.
- Right now, a single lock name is used for the pcbhash lock -- this is
undesirable, and probably another argument is required to take care of
this (or a char array name field in the pcbinfo?).
This is not an MFC candidate for 8.x due to its impact on lookup and
locking semantics. It's possible some of these issues could be worked
around with compatibility wrappers, if necessary.
Reviewed by: bz
Sponsored by: Juniper Networks, Inc.
2011-05-30 09:43:55 +00:00
|
|
|
goto badunlocked;
|
1999-12-07 17:39:16 +00:00
|
|
|
}
|
2021-10-28 15:11:45 +00:00
|
|
|
if (V_udp_blackhole && (V_udp_blackhole_local ||
|
|
|
|
!in6_localaddr(&ip6->ip6_src)))
|
2007-07-19 22:34:25 +00:00
|
|
|
goto badunlocked;
|
1999-12-07 17:39:16 +00:00
|
|
|
icmp6_error(m, ICMP6_DST_UNREACH, ICMP6_DST_UNREACH_NOPORT, 0);
|
2019-11-12 15:46:28 +00:00
|
|
|
*mp = NULL;
|
2007-07-09 17:47:04 +00:00
|
|
|
return (IPPROTO_DONE);
|
1999-12-07 17:39:16 +00:00
|
|
|
}
|
Decompose the current single inpcbinfo lock into two locks:
- The existing ipi_lock continues to protect the global inpcb list and
inpcb counter. This lock is now relegated to a small number of
allocation and free operations, and occasional operations that walk
all connections (including, awkwardly, certain UDP multicast receive
operations -- something to revisit).
- A new ipi_hash_lock protects the two inpcbinfo hash tables for
looking up connections and bound sockets, manipulated using new
INP_HASH_*() macros. This lock, combined with inpcb locks, protects
the 4-tuple address space.
Unlike the current ipi_lock, ipi_hash_lock follows the individual inpcb
connection locks, so may be acquired while manipulating a connection on
which a lock is already held, avoiding the need to acquire the inpcbinfo
lock preemptively when a binding change might later be required. As a
result, however, lookup operations necessarily go through a reference
acquire while holding the lookup lock, later acquiring an inpcb lock --
if required.
A new function in_pcblookup() looks up connections, and accepts flags
indicating how to return the inpcb. Due to lock order changes, callers
no longer need acquire locks before performing a lookup: the lookup
routine will acquire the ipi_hash_lock as needed. In the future, it will
also be able to use alternative lookup and locking strategies
transparently to callers, such as pcbgroup lookup. New lookup flags are,
supplementing the existing INPLOOKUP_WILDCARD flag:
INPLOOKUP_RLOCKPCB - Acquire a read lock on the returned inpcb
INPLOOKUP_WLOCKPCB - Acquire a write lock on the returned inpcb
Callers must pass exactly one of these flags (for the time being).
Some notes:
- All protocols are updated to work within the new regime; especially,
TCP, UDPv4, and UDPv6. pcbinfo ipi_lock acquisitions are largely
eliminated, and global hash lock hold times are dramatically reduced
compared to previous locking.
- The TCP syncache still relies on the pcbinfo lock, something that we
may want to revisit.
- Support for reverting to the FreeBSD 7.x locking strategy in TCP input
is no longer available -- hash lookup locks are now held only very
briefly during inpcb lookup, rather than for potentially extended
periods. However, the pcbinfo ipi_lock will still be acquired if a
connection state might change such that a connection is added or
removed.
- Raw IP sockets continue to use the pcbinfo ipi_lock for protection,
due to maintaining their own hash tables.
- The interface in6_pcblookup_hash_locked() is maintained, which allows
callers to acquire hash locks and perform one or more lookups atomically
with 4-tuple allocation: this is required only for TCPv6, as there is no
in6_pcbconnect_setup(), which there should be.
- UDPv6 locking remains significantly more conservative than UDPv4
locking, which relates to source address selection. This needs
attention, as it likely significantly reduces parallelism in this code
for multithreaded socket use (such as in BIND).
- In the UDPv4 and UDPv6 multicast cases, we need to revisit locking
somewhat, as they relied on ipi_lock to stablise 4-tuple matches, which
is no longer sufficient. A second check once the inpcb lock is held
should do the trick, keeping the general case from requiring the inpcb
lock for every inpcb visited.
- This work reminds us that we need to revisit locking of the v4/v6 flags,
which may be accessed lock-free both before and after this change.
- Right now, a single lock name is used for the pcbhash lock -- this is
undesirable, and probably another argument is required to take care of
this (or a char array name field in the pcbinfo?).
This is not an MFC candidate for 8.x due to its impact on lookup and
locking semantics. It's possible some of these issues could be worked
around with compatibility wrappers, if necessary.
Reviewed by: bz
Sponsored by: Juniper Networks, Inc.
2011-05-30 09:43:55 +00:00
|
|
|
INP_RLOCK_ASSERT(inp);
|
2009-05-23 16:51:13 +00:00
|
|
|
up = intoudpcb(inp);
|
2014-04-07 01:53:03 +00:00
|
|
|
if (cscov_partial) {
|
2014-10-01 05:43:29 +00:00
|
|
|
if (up->u_rxcslen == 0 || up->u_rxcslen > ulen) {
|
2014-04-07 01:53:03 +00:00
|
|
|
INP_RUNLOCK(inp);
|
|
|
|
m_freem(m);
|
2019-11-12 15:46:28 +00:00
|
|
|
*mp = NULL;
|
2014-04-07 01:53:03 +00:00
|
|
|
return (IPPROTO_DONE);
|
|
|
|
}
|
|
|
|
}
|
2018-07-31 22:56:03 +00:00
|
|
|
if (nxt == IPPROTO_UDPLITE)
|
|
|
|
UDPLITE_PROBE(receive, NULL, inp, ip6, inp, uh);
|
|
|
|
else
|
|
|
|
UDP_PROBE(receive, NULL, inp, ip6, inp, uh);
|
2017-03-06 04:01:58 +00:00
|
|
|
if (udp6_append(inp, m, off, fromsa) == 0)
|
2015-07-21 09:54:31 +00:00
|
|
|
INP_RUNLOCK(inp);
|
2019-11-12 15:46:28 +00:00
|
|
|
*mp = NULL;
|
2007-07-09 17:47:04 +00:00
|
|
|
return (IPPROTO_DONE);
|
2007-07-19 22:34:25 +00:00
|
|
|
|
2007-07-09 17:47:04 +00:00
|
|
|
badunlocked:
|
2019-12-05 16:50:54 +00:00
|
|
|
m_freem(m);
|
2019-11-12 15:46:28 +00:00
|
|
|
*mp = NULL;
|
2007-07-09 17:47:04 +00:00
|
|
|
return (IPPROTO_DONE);
|
1999-12-07 17:39:16 +00:00
|
|
|
}
|
|
|
|
|
2014-04-07 01:53:03 +00:00
|
|
|
static void
|
2022-10-04 03:53:04 +00:00
|
|
|
udp6_common_ctlinput(struct ip6ctlparam *ip6cp, struct inpcbinfo *pcbinfo)
|
1999-12-07 17:39:16 +00:00
|
|
|
{
|
|
|
|
struct udphdr uh;
|
|
|
|
struct ip6_hdr *ip6;
|
|
|
|
struct mbuf *m;
|
2022-10-04 03:53:04 +00:00
|
|
|
struct inpcb *inp;
|
2022-10-04 03:53:04 +00:00
|
|
|
int errno, off = 0;
|
2001-06-11 12:39:29 +00:00
|
|
|
struct udp_portonly {
|
|
|
|
u_int16_t uh_sport;
|
|
|
|
u_int16_t uh_dport;
|
|
|
|
} *uhp;
|
1999-12-07 17:39:16 +00:00
|
|
|
|
2022-10-04 03:53:04 +00:00
|
|
|
if ((errno = icmp6_errmap(ip6cp->ip6c_icmp6)) == 0)
|
2000-07-04 16:35:15 +00:00
|
|
|
return;
|
2022-10-04 03:53:04 +00:00
|
|
|
|
|
|
|
m = ip6cp->ip6c_m;
|
|
|
|
ip6 = ip6cp->ip6c_ip6;
|
|
|
|
off = ip6cp->ip6c_off;
|
|
|
|
|
|
|
|
/* Check if we can safely examine src and dst ports. */
|
|
|
|
if (m->m_pkthdr.len < off + sizeof(*uhp))
|
1999-12-07 17:39:16 +00:00
|
|
|
return;
|
|
|
|
|
2022-10-04 03:53:04 +00:00
|
|
|
bzero(&uh, sizeof(uh));
|
|
|
|
m_copydata(m, off, sizeof(*uhp), (caddr_t)&uh);
|
1999-12-07 17:39:16 +00:00
|
|
|
|
2022-10-04 03:53:04 +00:00
|
|
|
/* Check to see if its tunneled */
|
|
|
|
inp = in6_pcblookup_mbuf(pcbinfo, &ip6->ip6_dst, uh.uh_dport,
|
|
|
|
&ip6->ip6_src, uh.uh_sport, INPLOOKUP_WILDCARD | INPLOOKUP_RLOCKPCB,
|
|
|
|
m->m_pkthdr.rcvif, m);
|
|
|
|
if (inp != NULL) {
|
|
|
|
struct udpcb *up;
|
|
|
|
udp_tun_icmp_t *func;
|
1999-12-07 17:39:16 +00:00
|
|
|
|
2022-10-04 03:53:04 +00:00
|
|
|
up = intoudpcb(inp);
|
|
|
|
func = up->u_icmp_func;
|
|
|
|
INP_RUNLOCK(inp);
|
|
|
|
if (func != NULL)
|
2022-10-04 03:53:04 +00:00
|
|
|
func(ip6cp);
|
2022-10-04 03:53:04 +00:00
|
|
|
}
|
2022-10-04 03:53:04 +00:00
|
|
|
in6_pcbnotify(pcbinfo, ip6cp->ip6c_finaldst, uh.uh_dport,
|
|
|
|
ip6cp->ip6c_src, uh.uh_sport, errno, ip6cp->ip6c_cmdarg,
|
|
|
|
udp_notify);
|
1999-12-07 17:39:16 +00:00
|
|
|
}
|
|
|
|
|
2022-10-04 03:53:04 +00:00
|
|
|
static void
|
2022-10-04 03:53:04 +00:00
|
|
|
udp6_ctlinput(struct ip6ctlparam *ctl)
|
2014-04-07 01:53:03 +00:00
|
|
|
{
|
|
|
|
|
2022-10-04 03:53:04 +00:00
|
|
|
return (udp6_common_ctlinput(ctl, &V_udbinfo));
|
2014-04-07 01:53:03 +00:00
|
|
|
}
|
|
|
|
|
2022-10-04 03:53:04 +00:00
|
|
|
static void
|
2022-10-04 03:53:04 +00:00
|
|
|
udplite6_ctlinput(struct ip6ctlparam *ctl)
|
2014-04-07 01:53:03 +00:00
|
|
|
{
|
|
|
|
|
2022-10-04 03:53:04 +00:00
|
|
|
return (udp6_common_ctlinput(ctl, &V_ulitecbinfo));
|
2014-04-07 01:53:03 +00:00
|
|
|
}
|
|
|
|
|
1999-12-07 17:39:16 +00:00
|
|
|
static int
|
2000-07-04 11:25:35 +00:00
|
|
|
udp6_getcred(SYSCTL_HANDLER_ARGS)
|
1999-12-07 17:39:16 +00:00
|
|
|
{
|
2001-02-18 13:30:20 +00:00
|
|
|
struct xucred xuc;
|
1999-12-07 17:39:16 +00:00
|
|
|
struct sockaddr_in6 addrs[2];
|
2019-11-07 20:49:56 +00:00
|
|
|
struct epoch_tracker et;
|
1999-12-07 17:39:16 +00:00
|
|
|
struct inpcb *inp;
|
2006-04-12 03:03:47 +00:00
|
|
|
int error;
|
1999-12-07 17:39:16 +00:00
|
|
|
|
2007-06-12 00:12:01 +00:00
|
|
|
error = priv_check(req->td, PRIV_NETINET_GETCRED);
|
1999-12-07 17:39:16 +00:00
|
|
|
if (error)
|
|
|
|
return (error);
|
2000-07-04 16:35:15 +00:00
|
|
|
|
|
|
|
if (req->newlen != sizeof(addrs))
|
|
|
|
return (EINVAL);
|
2001-02-18 13:30:20 +00:00
|
|
|
if (req->oldlen != sizeof(struct xucred))
|
2000-07-04 16:35:15 +00:00
|
|
|
return (EINVAL);
|
1999-12-07 17:39:16 +00:00
|
|
|
error = SYSCTL_IN(req, addrs, sizeof(addrs));
|
|
|
|
if (error)
|
|
|
|
return (error);
|
Commit step 1 of the vimage project, (network stack)
virtualization work done by Marko Zec (zec@).
This is the first in a series of commits over the course
of the next few weeks.
Mark all uses of global variables to be virtualized
with a V_ prefix.
Use macros to map them back to their global names for
now, so this is a NOP change only.
We hope to have caught at least 85-90% of what is needed
so we do not invalidate a lot of outstanding patches again.
Obtained from: //depot/projects/vimage-commit2/...
Reviewed by: brooks, des, ed, mav, julian,
jamie, kris, rwatson, zec, ...
(various people I forgot, different versions)
md5 (with a bit of help)
Sponsored by: NLnet Foundation, The FreeBSD Foundation
X-MFC after: never
V_Commit_Message_Reviewed_By: more people than the patch
2008-08-17 23:27:27 +00:00
|
|
|
if ((error = sa6_embedscope(&addrs[0], V_ip6_use_defzone)) != 0 ||
|
|
|
|
(error = sa6_embedscope(&addrs[1], V_ip6_use_defzone)) != 0) {
|
2005-07-25 12:31:43 +00:00
|
|
|
return (error);
|
|
|
|
}
|
2019-11-07 20:49:56 +00:00
|
|
|
NET_EPOCH_ENTER(et);
|
Decompose the current single inpcbinfo lock into two locks:
- The existing ipi_lock continues to protect the global inpcb list and
inpcb counter. This lock is now relegated to a small number of
allocation and free operations, and occasional operations that walk
all connections (including, awkwardly, certain UDP multicast receive
operations -- something to revisit).
- A new ipi_hash_lock protects the two inpcbinfo hash tables for
looking up connections and bound sockets, manipulated using new
INP_HASH_*() macros. This lock, combined with inpcb locks, protects
the 4-tuple address space.
Unlike the current ipi_lock, ipi_hash_lock follows the individual inpcb
connection locks, so may be acquired while manipulating a connection on
which a lock is already held, avoiding the need to acquire the inpcbinfo
lock preemptively when a binding change might later be required. As a
result, however, lookup operations necessarily go through a reference
acquire while holding the lookup lock, later acquiring an inpcb lock --
if required.
A new function in_pcblookup() looks up connections, and accepts flags
indicating how to return the inpcb. Due to lock order changes, callers
no longer need acquire locks before performing a lookup: the lookup
routine will acquire the ipi_hash_lock as needed. In the future, it will
also be able to use alternative lookup and locking strategies
transparently to callers, such as pcbgroup lookup. New lookup flags are,
supplementing the existing INPLOOKUP_WILDCARD flag:
INPLOOKUP_RLOCKPCB - Acquire a read lock on the returned inpcb
INPLOOKUP_WLOCKPCB - Acquire a write lock on the returned inpcb
Callers must pass exactly one of these flags (for the time being).
Some notes:
- All protocols are updated to work within the new regime; especially,
TCP, UDPv4, and UDPv6. pcbinfo ipi_lock acquisitions are largely
eliminated, and global hash lock hold times are dramatically reduced
compared to previous locking.
- The TCP syncache still relies on the pcbinfo lock, something that we
may want to revisit.
- Support for reverting to the FreeBSD 7.x locking strategy in TCP input
is no longer available -- hash lookup locks are now held only very
briefly during inpcb lookup, rather than for potentially extended
periods. However, the pcbinfo ipi_lock will still be acquired if a
connection state might change such that a connection is added or
removed.
- Raw IP sockets continue to use the pcbinfo ipi_lock for protection,
due to maintaining their own hash tables.
- The interface in6_pcblookup_hash_locked() is maintained, which allows
callers to acquire hash locks and perform one or more lookups atomically
with 4-tuple allocation: this is required only for TCPv6, as there is no
in6_pcbconnect_setup(), which there should be.
- UDPv6 locking remains significantly more conservative than UDPv4
locking, which relates to source address selection. This needs
attention, as it likely significantly reduces parallelism in this code
for multithreaded socket use (such as in BIND).
- In the UDPv4 and UDPv6 multicast cases, we need to revisit locking
somewhat, as they relied on ipi_lock to stablise 4-tuple matches, which
is no longer sufficient. A second check once the inpcb lock is held
should do the trick, keeping the general case from requiring the inpcb
lock for every inpcb visited.
- This work reminds us that we need to revisit locking of the v4/v6 flags,
which may be accessed lock-free both before and after this change.
- Right now, a single lock name is used for the pcbhash lock -- this is
undesirable, and probably another argument is required to take care of
this (or a char array name field in the pcbinfo?).
This is not an MFC candidate for 8.x due to its impact on lookup and
locking semantics. It's possible some of these issues could be worked
around with compatibility wrappers, if necessary.
Reviewed by: bz
Sponsored by: Juniper Networks, Inc.
2011-05-30 09:43:55 +00:00
|
|
|
inp = in6_pcblookup(&V_udbinfo, &addrs[1].sin6_addr,
|
|
|
|
addrs[1].sin6_port, &addrs[0].sin6_addr, addrs[0].sin6_port,
|
|
|
|
INPLOOKUP_WILDCARD | INPLOOKUP_RLOCKPCB, NULL);
|
2019-11-07 20:49:56 +00:00
|
|
|
NET_EPOCH_EXIT(et);
|
2008-05-29 08:27:14 +00:00
|
|
|
if (inp != NULL) {
|
Decompose the current single inpcbinfo lock into two locks:
- The existing ipi_lock continues to protect the global inpcb list and
inpcb counter. This lock is now relegated to a small number of
allocation and free operations, and occasional operations that walk
all connections (including, awkwardly, certain UDP multicast receive
operations -- something to revisit).
- A new ipi_hash_lock protects the two inpcbinfo hash tables for
looking up connections and bound sockets, manipulated using new
INP_HASH_*() macros. This lock, combined with inpcb locks, protects
the 4-tuple address space.
Unlike the current ipi_lock, ipi_hash_lock follows the individual inpcb
connection locks, so may be acquired while manipulating a connection on
which a lock is already held, avoiding the need to acquire the inpcbinfo
lock preemptively when a binding change might later be required. As a
result, however, lookup operations necessarily go through a reference
acquire while holding the lookup lock, later acquiring an inpcb lock --
if required.
A new function in_pcblookup() looks up connections, and accepts flags
indicating how to return the inpcb. Due to lock order changes, callers
no longer need acquire locks before performing a lookup: the lookup
routine will acquire the ipi_hash_lock as needed. In the future, it will
also be able to use alternative lookup and locking strategies
transparently to callers, such as pcbgroup lookup. New lookup flags are,
supplementing the existing INPLOOKUP_WILDCARD flag:
INPLOOKUP_RLOCKPCB - Acquire a read lock on the returned inpcb
INPLOOKUP_WLOCKPCB - Acquire a write lock on the returned inpcb
Callers must pass exactly one of these flags (for the time being).
Some notes:
- All protocols are updated to work within the new regime; especially,
TCP, UDPv4, and UDPv6. pcbinfo ipi_lock acquisitions are largely
eliminated, and global hash lock hold times are dramatically reduced
compared to previous locking.
- The TCP syncache still relies on the pcbinfo lock, something that we
may want to revisit.
- Support for reverting to the FreeBSD 7.x locking strategy in TCP input
is no longer available -- hash lookup locks are now held only very
briefly during inpcb lookup, rather than for potentially extended
periods. However, the pcbinfo ipi_lock will still be acquired if a
connection state might change such that a connection is added or
removed.
- Raw IP sockets continue to use the pcbinfo ipi_lock for protection,
due to maintaining their own hash tables.
- The interface in6_pcblookup_hash_locked() is maintained, which allows
callers to acquire hash locks and perform one or more lookups atomically
with 4-tuple allocation: this is required only for TCPv6, as there is no
in6_pcbconnect_setup(), which there should be.
- UDPv6 locking remains significantly more conservative than UDPv4
locking, which relates to source address selection. This needs
attention, as it likely significantly reduces parallelism in this code
for multithreaded socket use (such as in BIND).
- In the UDPv4 and UDPv6 multicast cases, we need to revisit locking
somewhat, as they relied on ipi_lock to stablise 4-tuple matches, which
is no longer sufficient. A second check once the inpcb lock is held
should do the trick, keeping the general case from requiring the inpcb
lock for every inpcb visited.
- This work reminds us that we need to revisit locking of the v4/v6 flags,
which may be accessed lock-free both before and after this change.
- Right now, a single lock name is used for the pcbhash lock -- this is
undesirable, and probably another argument is required to take care of
this (or a char array name field in the pcbinfo?).
This is not an MFC candidate for 8.x due to its impact on lookup and
locking semantics. It's possible some of these issues could be worked
around with compatibility wrappers, if necessary.
Reviewed by: bz
Sponsored by: Juniper Networks, Inc.
2011-05-30 09:43:55 +00:00
|
|
|
INP_RLOCK_ASSERT(inp);
|
2008-05-29 08:27:14 +00:00
|
|
|
if (inp->inp_socket == NULL)
|
|
|
|
error = ENOENT;
|
|
|
|
if (error == 0)
|
|
|
|
error = cr_canseesocket(req->td->td_ucred,
|
|
|
|
inp->inp_socket);
|
|
|
|
if (error == 0)
|
2008-10-04 15:06:34 +00:00
|
|
|
cru2x(inp->inp_cred, &xuc);
|
2008-05-29 08:27:14 +00:00
|
|
|
INP_RUNLOCK(inp);
|
Decompose the current single inpcbinfo lock into two locks:
- The existing ipi_lock continues to protect the global inpcb list and
inpcb counter. This lock is now relegated to a small number of
allocation and free operations, and occasional operations that walk
all connections (including, awkwardly, certain UDP multicast receive
operations -- something to revisit).
- A new ipi_hash_lock protects the two inpcbinfo hash tables for
looking up connections and bound sockets, manipulated using new
INP_HASH_*() macros. This lock, combined with inpcb locks, protects
the 4-tuple address space.
Unlike the current ipi_lock, ipi_hash_lock follows the individual inpcb
connection locks, so may be acquired while manipulating a connection on
which a lock is already held, avoiding the need to acquire the inpcbinfo
lock preemptively when a binding change might later be required. As a
result, however, lookup operations necessarily go through a reference
acquire while holding the lookup lock, later acquiring an inpcb lock --
if required.
A new function in_pcblookup() looks up connections, and accepts flags
indicating how to return the inpcb. Due to lock order changes, callers
no longer need acquire locks before performing a lookup: the lookup
routine will acquire the ipi_hash_lock as needed. In the future, it will
also be able to use alternative lookup and locking strategies
transparently to callers, such as pcbgroup lookup. New lookup flags are,
supplementing the existing INPLOOKUP_WILDCARD flag:
INPLOOKUP_RLOCKPCB - Acquire a read lock on the returned inpcb
INPLOOKUP_WLOCKPCB - Acquire a write lock on the returned inpcb
Callers must pass exactly one of these flags (for the time being).
Some notes:
- All protocols are updated to work within the new regime; especially,
TCP, UDPv4, and UDPv6. pcbinfo ipi_lock acquisitions are largely
eliminated, and global hash lock hold times are dramatically reduced
compared to previous locking.
- The TCP syncache still relies on the pcbinfo lock, something that we
may want to revisit.
- Support for reverting to the FreeBSD 7.x locking strategy in TCP input
is no longer available -- hash lookup locks are now held only very
briefly during inpcb lookup, rather than for potentially extended
periods. However, the pcbinfo ipi_lock will still be acquired if a
connection state might change such that a connection is added or
removed.
- Raw IP sockets continue to use the pcbinfo ipi_lock for protection,
due to maintaining their own hash tables.
- The interface in6_pcblookup_hash_locked() is maintained, which allows
callers to acquire hash locks and perform one or more lookups atomically
with 4-tuple allocation: this is required only for TCPv6, as there is no
in6_pcbconnect_setup(), which there should be.
- UDPv6 locking remains significantly more conservative than UDPv4
locking, which relates to source address selection. This needs
attention, as it likely significantly reduces parallelism in this code
for multithreaded socket use (such as in BIND).
- In the UDPv4 and UDPv6 multicast cases, we need to revisit locking
somewhat, as they relied on ipi_lock to stablise 4-tuple matches, which
is no longer sufficient. A second check once the inpcb lock is held
should do the trick, keeping the general case from requiring the inpcb
lock for every inpcb visited.
- This work reminds us that we need to revisit locking of the v4/v6 flags,
which may be accessed lock-free both before and after this change.
- Right now, a single lock name is used for the pcbhash lock -- this is
undesirable, and probably another argument is required to take care of
this (or a char array name field in the pcbinfo?).
This is not an MFC candidate for 8.x due to its impact on lookup and
locking semantics. It's possible some of these issues could be worked
around with compatibility wrappers, if necessary.
Reviewed by: bz
Sponsored by: Juniper Networks, Inc.
2011-05-30 09:43:55 +00:00
|
|
|
} else
|
2007-07-27 08:25:02 +00:00
|
|
|
error = ENOENT;
|
|
|
|
if (error == 0)
|
|
|
|
error = SYSCTL_OUT(req, &xuc, sizeof(struct xucred));
|
1999-12-07 17:39:16 +00:00
|
|
|
return (error);
|
|
|
|
}
|
|
|
|
|
2020-02-26 14:26:36 +00:00
|
|
|
SYSCTL_PROC(_net_inet6_udp6, OID_AUTO, getcred,
|
|
|
|
CTLTYPE_OPAQUE | CTLFLAG_RW | CTLFLAG_MPSAFE,
|
|
|
|
0, 0, udp6_getcred, "S,xucred",
|
|
|
|
"Get the xucred of a UDP6 connection");
|
1999-12-07 17:39:16 +00:00
|
|
|
|
2007-07-23 07:58:58 +00:00
|
|
|
static int
|
2022-12-07 19:51:48 +00:00
|
|
|
udp6_send(struct socket *so, int flags_arg, struct mbuf *m,
|
2018-08-23 16:54:22 +00:00
|
|
|
struct sockaddr *addr6, struct mbuf *control, struct thread *td)
|
2007-07-23 07:58:58 +00:00
|
|
|
{
|
2018-08-23 16:54:22 +00:00
|
|
|
struct inpcb *inp;
|
2007-07-23 07:58:58 +00:00
|
|
|
struct ip6_hdr *ip6;
|
|
|
|
struct udphdr *udp6;
|
2009-06-23 22:08:55 +00:00
|
|
|
struct in6_addr *laddr, *faddr, in6a;
|
2007-07-23 07:58:58 +00:00
|
|
|
struct ip6_pktopts *optp, opt;
|
2018-08-23 16:54:22 +00:00
|
|
|
struct sockaddr_in6 *sin6, tmp;
|
|
|
|
struct epoch_tracker et;
|
|
|
|
int cscov_partial, error, flags, hlen, scope_ambiguous;
|
|
|
|
u_int32_t ulen, plen;
|
|
|
|
uint16_t cscov;
|
|
|
|
u_short fport;
|
Now with epoch synchronized PCB lookup tables we can greatly simplify
locking in udp_output() and udp6_output().
First, we select if we need read or write lock in PCB itself, we take
the lock and enter network epoch. Then, we proceed for the rest of
the function. In case if we need to modify PCB hash, we would take
write lock on it for a short piece of code.
We could exit the epoch before allocating an mbuf, but with this
patch we are keeping it all the way into ip_output()/ip6_output().
Today this creates an epoch recursion, since ip_output() enters epoch
itself. However, once all protocols are reviewed, ip_output() and
ip6_output() would require epoch instead of entering it.
Note: I'm not 100% sure that in udp6_output() the epoch is required.
We don't do PCB hash lookup for a bound socket. And all branches of
in6_select_src() don't require epoch, at least they lack assertions.
Today inet6 address list is protected by rmlock, although it is CKLIST.
AFAIU, the future plan is to protect it by network epoch. That would
require epoch in in6_select_src(). Anyway, in future ip6_output()
would require epoch, udp6_output() would need to enter it.
2019-11-07 21:01:36 +00:00
|
|
|
uint8_t nxt;
|
2007-07-23 07:58:58 +00:00
|
|
|
|
2022-12-07 19:51:48 +00:00
|
|
|
if (addr6) {
|
|
|
|
error = 0;
|
|
|
|
if (addr6->sa_family != AF_INET6)
|
|
|
|
error = EAFNOSUPPORT;
|
|
|
|
else if (addr6->sa_len != sizeof(struct sockaddr_in6))
|
|
|
|
error = EINVAL;
|
|
|
|
if (__predict_false(error != 0)) {
|
|
|
|
m_freem(control);
|
|
|
|
m_freem(m);
|
|
|
|
return (error);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2018-08-23 16:54:22 +00:00
|
|
|
sin6 = (struct sockaddr_in6 *)addr6;
|
2007-07-23 07:58:58 +00:00
|
|
|
|
2018-08-23 16:54:22 +00:00
|
|
|
/*
|
2022-09-10 11:01:44 +00:00
|
|
|
* In contrast to IPv4 we do not validate the max. packet length
|
2018-08-23 16:54:22 +00:00
|
|
|
* here due to IPv6 Jumbograms (RFC2675).
|
|
|
|
*/
|
2007-07-23 07:58:58 +00:00
|
|
|
|
2018-08-23 16:54:22 +00:00
|
|
|
scope_ambiguous = 0;
|
|
|
|
if (sin6) {
|
|
|
|
/* Protect *addr6 from overwrites. */
|
2007-07-23 07:58:58 +00:00
|
|
|
tmp = *sin6;
|
|
|
|
sin6 = &tmp;
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Application should provide a proper zone ID or the use of
|
|
|
|
* default zone IDs should be enabled. Unfortunately, some
|
|
|
|
* applications do not behave as it should, so we need a
|
|
|
|
* workaround. Even if an appropriate ID is not determined,
|
|
|
|
* we'll see if we can determine the outgoing interface. If we
|
|
|
|
* can, determine the zone ID based on the interface below.
|
|
|
|
*/
|
Commit step 1 of the vimage project, (network stack)
virtualization work done by Marko Zec (zec@).
This is the first in a series of commits over the course
of the next few weeks.
Mark all uses of global variables to be virtualized
with a V_ prefix.
Use macros to map them back to their global names for
now, so this is a NOP change only.
We hope to have caught at least 85-90% of what is needed
so we do not invalidate a lot of outstanding patches again.
Obtained from: //depot/projects/vimage-commit2/...
Reviewed by: brooks, des, ed, mav, julian,
jamie, kris, rwatson, zec, ...
(various people I forgot, different versions)
md5 (with a bit of help)
Sponsored by: NLnet Foundation, The FreeBSD Foundation
X-MFC after: never
V_Commit_Message_Reviewed_By: more people than the patch
2008-08-17 23:27:27 +00:00
|
|
|
if (sin6->sin6_scope_id == 0 && !V_ip6_use_defzone)
|
2007-07-23 07:58:58 +00:00
|
|
|
scope_ambiguous = 1;
|
2018-08-23 16:54:22 +00:00
|
|
|
if ((error = sa6_embedscope(sin6, V_ip6_use_defzone)) != 0) {
|
|
|
|
if (control)
|
|
|
|
m_freem(control);
|
|
|
|
m_freem(m);
|
2007-07-23 07:58:58 +00:00
|
|
|
return (error);
|
2018-08-23 16:54:22 +00:00
|
|
|
}
|
2007-07-23 07:58:58 +00:00
|
|
|
}
|
|
|
|
|
2018-08-23 16:54:22 +00:00
|
|
|
inp = sotoinpcb(so);
|
|
|
|
KASSERT(inp != NULL, ("%s: inp == NULL", __func__));
|
2018-09-19 18:49:37 +00:00
|
|
|
/*
|
|
|
|
* In the following cases we want a write lock on the inp for either
|
|
|
|
* local operations or for possible route cache updates in the IPv6
|
|
|
|
* output path:
|
|
|
|
* - on connected sockets (sin6 is NULL) for route cache updates,
|
|
|
|
* - when we are not bound to an address and source port (it is
|
|
|
|
* in6_pcbsetport() which will require the write lock).
|
Now with epoch synchronized PCB lookup tables we can greatly simplify
locking in udp_output() and udp6_output().
First, we select if we need read or write lock in PCB itself, we take
the lock and enter network epoch. Then, we proceed for the rest of
the function. In case if we need to modify PCB hash, we would take
write lock on it for a short piece of code.
We could exit the epoch before allocating an mbuf, but with this
patch we are keeping it all the way into ip_output()/ip6_output().
Today this creates an epoch recursion, since ip_output() enters epoch
itself. However, once all protocols are reviewed, ip_output() and
ip6_output() would require epoch instead of entering it.
Note: I'm not 100% sure that in udp6_output() the epoch is required.
We don't do PCB hash lookup for a bound socket. And all branches of
in6_select_src() don't require epoch, at least they lack assertions.
Today inet6 address list is protected by rmlock, although it is CKLIST.
AFAIU, the future plan is to protect it by network epoch. That would
require epoch in in6_select_src(). Anyway, in future ip6_output()
would require epoch, udp6_output() would need to enter it.
2019-11-07 21:01:36 +00:00
|
|
|
*
|
|
|
|
* We check the inp fields before actually locking the inp, so
|
|
|
|
* here exists a race, and we may WLOCK the inp and end with already
|
|
|
|
* bound one by other thread. This is fine.
|
2018-09-19 18:49:37 +00:00
|
|
|
*/
|
|
|
|
if (sin6 == NULL || (IN6_IS_ADDR_UNSPECIFIED(&inp->in6p_laddr) &&
|
Now with epoch synchronized PCB lookup tables we can greatly simplify
locking in udp_output() and udp6_output().
First, we select if we need read or write lock in PCB itself, we take
the lock and enter network epoch. Then, we proceed for the rest of
the function. In case if we need to modify PCB hash, we would take
write lock on it for a short piece of code.
We could exit the epoch before allocating an mbuf, but with this
patch we are keeping it all the way into ip_output()/ip6_output().
Today this creates an epoch recursion, since ip_output() enters epoch
itself. However, once all protocols are reviewed, ip_output() and
ip6_output() would require epoch instead of entering it.
Note: I'm not 100% sure that in udp6_output() the epoch is required.
We don't do PCB hash lookup for a bound socket. And all branches of
in6_select_src() don't require epoch, at least they lack assertions.
Today inet6 address list is protected by rmlock, although it is CKLIST.
AFAIU, the future plan is to protect it by network epoch. That would
require epoch in in6_select_src(). Anyway, in future ip6_output()
would require epoch, udp6_output() would need to enter it.
2019-11-07 21:01:36 +00:00
|
|
|
inp->inp_lport == 0))
|
2018-09-19 18:49:37 +00:00
|
|
|
INP_WLOCK(inp);
|
Now with epoch synchronized PCB lookup tables we can greatly simplify
locking in udp_output() and udp6_output().
First, we select if we need read or write lock in PCB itself, we take
the lock and enter network epoch. Then, we proceed for the rest of
the function. In case if we need to modify PCB hash, we would take
write lock on it for a short piece of code.
We could exit the epoch before allocating an mbuf, but with this
patch we are keeping it all the way into ip_output()/ip6_output().
Today this creates an epoch recursion, since ip_output() enters epoch
itself. However, once all protocols are reviewed, ip_output() and
ip6_output() would require epoch instead of entering it.
Note: I'm not 100% sure that in udp6_output() the epoch is required.
We don't do PCB hash lookup for a bound socket. And all branches of
in6_select_src() don't require epoch, at least they lack assertions.
Today inet6 address list is protected by rmlock, although it is CKLIST.
AFAIU, the future plan is to protect it by network epoch. That would
require epoch in in6_select_src(). Anyway, in future ip6_output()
would require epoch, udp6_output() would need to enter it.
2019-11-07 21:01:36 +00:00
|
|
|
else
|
2018-09-19 18:49:37 +00:00
|
|
|
INP_RLOCK(inp);
|
Now with epoch synchronized PCB lookup tables we can greatly simplify
locking in udp_output() and udp6_output().
First, we select if we need read or write lock in PCB itself, we take
the lock and enter network epoch. Then, we proceed for the rest of
the function. In case if we need to modify PCB hash, we would take
write lock on it for a short piece of code.
We could exit the epoch before allocating an mbuf, but with this
patch we are keeping it all the way into ip_output()/ip6_output().
Today this creates an epoch recursion, since ip_output() enters epoch
itself. However, once all protocols are reviewed, ip_output() and
ip6_output() would require epoch instead of entering it.
Note: I'm not 100% sure that in udp6_output() the epoch is required.
We don't do PCB hash lookup for a bound socket. And all branches of
in6_select_src() don't require epoch, at least they lack assertions.
Today inet6 address list is protected by rmlock, although it is CKLIST.
AFAIU, the future plan is to protect it by network epoch. That would
require epoch in in6_select_src(). Anyway, in future ip6_output()
would require epoch, udp6_output() would need to enter it.
2019-11-07 21:01:36 +00:00
|
|
|
|
2015-09-21 12:32:36 +00:00
|
|
|
nxt = (inp->inp_socket->so_proto->pr_protocol == IPPROTO_UDP) ?
|
|
|
|
IPPROTO_UDP : IPPROTO_UDPLITE;
|
2018-08-23 16:54:22 +00:00
|
|
|
|
|
|
|
#ifdef INET
|
|
|
|
if ((inp->inp_flags & IN6P_IPV6_V6ONLY) == 0) {
|
|
|
|
int hasv4addr;
|
|
|
|
|
|
|
|
if (sin6 == NULL)
|
|
|
|
hasv4addr = (inp->inp_vflag & INP_IPV4);
|
|
|
|
else
|
|
|
|
hasv4addr = IN6_IS_ADDR_V4MAPPED(&sin6->sin6_addr)
|
|
|
|
? 1 : 0;
|
|
|
|
if (hasv4addr) {
|
|
|
|
/*
|
|
|
|
* XXXRW: We release UDP-layer locks before calling
|
|
|
|
* udp_send() in order to avoid recursion. However,
|
|
|
|
* this does mean there is a short window where inp's
|
|
|
|
* fields are unstable. Could this lead to a
|
|
|
|
* potential race in which the factors causing us to
|
|
|
|
* select the UDPv4 output routine are invalidated?
|
|
|
|
*/
|
Now with epoch synchronized PCB lookup tables we can greatly simplify
locking in udp_output() and udp6_output().
First, we select if we need read or write lock in PCB itself, we take
the lock and enter network epoch. Then, we proceed for the rest of
the function. In case if we need to modify PCB hash, we would take
write lock on it for a short piece of code.
We could exit the epoch before allocating an mbuf, but with this
patch we are keeping it all the way into ip_output()/ip6_output().
Today this creates an epoch recursion, since ip_output() enters epoch
itself. However, once all protocols are reviewed, ip_output() and
ip6_output() would require epoch instead of entering it.
Note: I'm not 100% sure that in udp6_output() the epoch is required.
We don't do PCB hash lookup for a bound socket. And all branches of
in6_select_src() don't require epoch, at least they lack assertions.
Today inet6 address list is protected by rmlock, although it is CKLIST.
AFAIU, the future plan is to protect it by network epoch. That would
require epoch in in6_select_src(). Anyway, in future ip6_output()
would require epoch, udp6_output() would need to enter it.
2019-11-07 21:01:36 +00:00
|
|
|
INP_UNLOCK(inp);
|
2018-08-23 16:54:22 +00:00
|
|
|
if (sin6)
|
|
|
|
in6_sin6_2_sin_in_sock((struct sockaddr *)sin6);
|
|
|
|
/* addr will just be freed in sendit(). */
|
2022-08-16 19:40:36 +00:00
|
|
|
return (udp_send(so, flags_arg | PRUS_IPV6, m,
|
2018-08-23 16:54:22 +00:00
|
|
|
(struct sockaddr *)sin6, control, td));
|
|
|
|
}
|
2018-10-02 17:29:56 +00:00
|
|
|
} else
|
2018-08-23 16:54:22 +00:00
|
|
|
#endif
|
2018-10-02 17:29:56 +00:00
|
|
|
if (sin6 && IN6_IS_ADDR_V4MAPPED(&sin6->sin6_addr)) {
|
|
|
|
/*
|
|
|
|
* Given this is either an IPv6-only socket or no INET is
|
|
|
|
* supported we will fail the send if the given destination
|
|
|
|
* address is a v4mapped address.
|
|
|
|
*/
|
Now with epoch synchronized PCB lookup tables we can greatly simplify
locking in udp_output() and udp6_output().
First, we select if we need read or write lock in PCB itself, we take
the lock and enter network epoch. Then, we proceed for the rest of
the function. In case if we need to modify PCB hash, we would take
write lock on it for a short piece of code.
We could exit the epoch before allocating an mbuf, but with this
patch we are keeping it all the way into ip_output()/ip6_output().
Today this creates an epoch recursion, since ip_output() enters epoch
itself. However, once all protocols are reviewed, ip_output() and
ip6_output() would require epoch instead of entering it.
Note: I'm not 100% sure that in udp6_output() the epoch is required.
We don't do PCB hash lookup for a bound socket. And all branches of
in6_select_src() don't require epoch, at least they lack assertions.
Today inet6 address list is protected by rmlock, although it is CKLIST.
AFAIU, the future plan is to protect it by network epoch. That would
require epoch in in6_select_src(). Anyway, in future ip6_output()
would require epoch, udp6_output() would need to enter it.
2019-11-07 21:01:36 +00:00
|
|
|
INP_UNLOCK(inp);
|
2023-03-14 14:26:29 +00:00
|
|
|
m_freem(m);
|
|
|
|
m_freem(control);
|
2018-10-02 17:29:56 +00:00
|
|
|
return (EINVAL);
|
|
|
|
}
|
2018-08-23 16:54:22 +00:00
|
|
|
|
2021-05-19 19:10:03 +00:00
|
|
|
NET_EPOCH_ENTER(et);
|
2007-07-23 07:58:58 +00:00
|
|
|
if (control) {
|
|
|
|
if ((error = ip6_setpktopts(control, &opt,
|
2018-08-23 16:54:22 +00:00
|
|
|
inp->in6p_outputopts, td->td_ucred, nxt)) != 0) {
|
2021-05-19 19:10:03 +00:00
|
|
|
goto release;
|
2018-08-23 16:54:22 +00:00
|
|
|
}
|
2007-07-23 07:58:58 +00:00
|
|
|
optp = &opt;
|
|
|
|
} else
|
2007-09-08 08:18:24 +00:00
|
|
|
optp = inp->in6p_outputopts;
|
2007-07-23 07:58:58 +00:00
|
|
|
|
|
|
|
if (sin6) {
|
|
|
|
/*
|
|
|
|
* Since we saw no essential reason for calling in_pcbconnect,
|
|
|
|
* we get rid of such kind of logic, and call in6_selectsrc
|
|
|
|
* and in6_pcbsetport in order to fill in the local address
|
|
|
|
* and the local port.
|
|
|
|
*/
|
|
|
|
if (sin6->sin6_port == 0) {
|
|
|
|
error = EADDRNOTAVAIL;
|
|
|
|
goto release;
|
|
|
|
}
|
|
|
|
|
2007-09-08 08:18:24 +00:00
|
|
|
if (!IN6_IS_ADDR_UNSPECIFIED(&inp->in6p_faddr)) {
|
2007-07-23 07:58:58 +00:00
|
|
|
/* how about ::ffff:0.0.0.0 case? */
|
|
|
|
error = EISCONN;
|
|
|
|
goto release;
|
|
|
|
}
|
|
|
|
|
2018-08-23 16:54:22 +00:00
|
|
|
/*
|
|
|
|
* Given we handle the v4mapped case in the INET block above
|
|
|
|
* assert here that it must not happen anymore.
|
|
|
|
*/
|
|
|
|
KASSERT(!IN6_IS_ADDR_V4MAPPED(&sin6->sin6_addr),
|
|
|
|
("%s: sin6(%p)->sin6_addr is v4mapped which we "
|
|
|
|
"should have handled.", __func__, sin6));
|
2007-07-23 07:58:58 +00:00
|
|
|
|
2018-08-23 16:54:22 +00:00
|
|
|
/* This only requires read-locking. */
|
|
|
|
error = in6_selectsrc_socket(sin6, optp, inp,
|
|
|
|
td->td_ucred, scope_ambiguous, &in6a, NULL);
|
|
|
|
if (error)
|
|
|
|
goto release;
|
|
|
|
laddr = &in6a;
|
2007-07-23 07:58:58 +00:00
|
|
|
|
2018-08-23 16:54:22 +00:00
|
|
|
if (inp->inp_lport == 0) {
|
Now with epoch synchronized PCB lookup tables we can greatly simplify
locking in udp_output() and udp6_output().
First, we select if we need read or write lock in PCB itself, we take
the lock and enter network epoch. Then, we proceed for the rest of
the function. In case if we need to modify PCB hash, we would take
write lock on it for a short piece of code.
We could exit the epoch before allocating an mbuf, but with this
patch we are keeping it all the way into ip_output()/ip6_output().
Today this creates an epoch recursion, since ip_output() enters epoch
itself. However, once all protocols are reviewed, ip_output() and
ip6_output() would require epoch instead of entering it.
Note: I'm not 100% sure that in udp6_output() the epoch is required.
We don't do PCB hash lookup for a bound socket. And all branches of
in6_select_src() don't require epoch, at least they lack assertions.
Today inet6 address list is protected by rmlock, although it is CKLIST.
AFAIU, the future plan is to protect it by network epoch. That would
require epoch in in6_select_src(). Anyway, in future ip6_output()
would require epoch, udp6_output() would need to enter it.
2019-11-07 21:01:36 +00:00
|
|
|
struct inpcbinfo *pcbinfo;
|
2007-07-23 07:58:58 +00:00
|
|
|
|
2018-08-23 16:54:22 +00:00
|
|
|
INP_WLOCK_ASSERT(inp);
|
Now with epoch synchronized PCB lookup tables we can greatly simplify
locking in udp_output() and udp6_output().
First, we select if we need read or write lock in PCB itself, we take
the lock and enter network epoch. Then, we proceed for the rest of
the function. In case if we need to modify PCB hash, we would take
write lock on it for a short piece of code.
We could exit the epoch before allocating an mbuf, but with this
patch we are keeping it all the way into ip_output()/ip6_output().
Today this creates an epoch recursion, since ip_output() enters epoch
itself. However, once all protocols are reviewed, ip_output() and
ip6_output() would require epoch instead of entering it.
Note: I'm not 100% sure that in udp6_output() the epoch is required.
We don't do PCB hash lookup for a bound socket. And all branches of
in6_select_src() don't require epoch, at least they lack assertions.
Today inet6 address list is protected by rmlock, although it is CKLIST.
AFAIU, the future plan is to protect it by network epoch. That would
require epoch in in6_select_src(). Anyway, in future ip6_output()
would require epoch, udp6_output() would need to enter it.
2019-11-07 21:01:36 +00:00
|
|
|
|
|
|
|
pcbinfo = udp_get_inpcbinfo(so->so_proto->pr_protocol);
|
|
|
|
INP_HASH_WLOCK(pcbinfo);
|
2018-08-23 16:54:22 +00:00
|
|
|
error = in6_pcbsetport(laddr, inp, td->td_ucred);
|
Now with epoch synchronized PCB lookup tables we can greatly simplify
locking in udp_output() and udp6_output().
First, we select if we need read or write lock in PCB itself, we take
the lock and enter network epoch. Then, we proceed for the rest of
the function. In case if we need to modify PCB hash, we would take
write lock on it for a short piece of code.
We could exit the epoch before allocating an mbuf, but with this
patch we are keeping it all the way into ip_output()/ip6_output().
Today this creates an epoch recursion, since ip_output() enters epoch
itself. However, once all protocols are reviewed, ip_output() and
ip6_output() would require epoch instead of entering it.
Note: I'm not 100% sure that in udp6_output() the epoch is required.
We don't do PCB hash lookup for a bound socket. And all branches of
in6_select_src() don't require epoch, at least they lack assertions.
Today inet6 address list is protected by rmlock, although it is CKLIST.
AFAIU, the future plan is to protect it by network epoch. That would
require epoch in in6_select_src(). Anyway, in future ip6_output()
would require epoch, udp6_output() would need to enter it.
2019-11-07 21:01:36 +00:00
|
|
|
INP_HASH_WUNLOCK(pcbinfo);
|
2018-08-23 16:54:22 +00:00
|
|
|
if (error != 0) {
|
|
|
|
/* Undo an address bind that may have occurred. */
|
|
|
|
inp->in6p_laddr = in6addr_any;
|
2009-06-23 22:08:55 +00:00
|
|
|
goto release;
|
2018-08-23 16:54:22 +00:00
|
|
|
}
|
2011-03-12 16:45:15 +00:00
|
|
|
}
|
2018-08-23 16:54:22 +00:00
|
|
|
faddr = &sin6->sin6_addr;
|
|
|
|
fport = sin6->sin6_port; /* allow 0 port */
|
|
|
|
|
2007-07-23 07:58:58 +00:00
|
|
|
} else {
|
2007-09-08 08:18:24 +00:00
|
|
|
if (IN6_IS_ADDR_UNSPECIFIED(&inp->in6p_faddr)) {
|
2007-07-23 07:58:58 +00:00
|
|
|
error = ENOTCONN;
|
|
|
|
goto release;
|
|
|
|
}
|
2007-09-08 08:18:24 +00:00
|
|
|
laddr = &inp->in6p_laddr;
|
|
|
|
faddr = &inp->in6p_faddr;
|
Another step assimilating IPv[46] PCB code - directly use
the inpcb names rather than the following IPv6 compat macros:
in6pcb,in6p_sp, in6p_ip6_nxt,in6p_flowinfo,in6p_vflag,
in6p_flags,in6p_socket,in6p_lport,in6p_fport,in6p_ppcb and
sotoin6pcb().
Apart from removing duplicate code in netipsec, this is a pure
whitespace, not a functional change.
Discussed with: rwatson
Reviewed by: rwatson (version before review requested changes)
MFC after: 4 weeks (set the timer and see then)
2008-12-15 21:50:54 +00:00
|
|
|
fport = inp->inp_fport;
|
2007-07-23 07:58:58 +00:00
|
|
|
}
|
|
|
|
|
2018-08-23 16:54:22 +00:00
|
|
|
ulen = m->m_pkthdr.len;
|
|
|
|
plen = sizeof(struct udphdr) + ulen;
|
|
|
|
hlen = sizeof(struct ip6_hdr);
|
2007-07-23 07:58:58 +00:00
|
|
|
|
|
|
|
/*
|
|
|
|
* Calculate data length and get a mbuf
|
|
|
|
* for UDP and IP6 headers.
|
|
|
|
*/
|
2012-12-05 08:04:20 +00:00
|
|
|
M_PREPEND(m, hlen + sizeof(struct udphdr), M_NOWAIT);
|
2015-09-21 12:53:26 +00:00
|
|
|
if (m == NULL) {
|
2007-07-23 07:58:58 +00:00
|
|
|
error = ENOBUFS;
|
|
|
|
goto release;
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Stuff checksum and output datagram.
|
|
|
|
*/
|
2018-08-23 16:54:22 +00:00
|
|
|
cscov = cscov_partial = 0;
|
2007-07-23 07:58:58 +00:00
|
|
|
udp6 = (struct udphdr *)(mtod(m, caddr_t) + hlen);
|
Another step assimilating IPv[46] PCB code - directly use
the inpcb names rather than the following IPv6 compat macros:
in6pcb,in6p_sp, in6p_ip6_nxt,in6p_flowinfo,in6p_vflag,
in6p_flags,in6p_socket,in6p_lport,in6p_fport,in6p_ppcb and
sotoin6pcb().
Apart from removing duplicate code in netipsec, this is a pure
whitespace, not a functional change.
Discussed with: rwatson
Reviewed by: rwatson (version before review requested changes)
MFC after: 4 weeks (set the timer and see then)
2008-12-15 21:50:54 +00:00
|
|
|
udp6->uh_sport = inp->inp_lport; /* lport is always set in the PCB */
|
2007-07-23 07:58:58 +00:00
|
|
|
udp6->uh_dport = fport;
|
2014-04-07 01:53:03 +00:00
|
|
|
if (nxt == IPPROTO_UDPLITE) {
|
|
|
|
struct udpcb *up;
|
|
|
|
|
|
|
|
up = intoudpcb(inp);
|
|
|
|
cscov = up->u_txcslen;
|
|
|
|
if (cscov >= plen)
|
|
|
|
cscov = 0;
|
|
|
|
udp6->uh_ulen = htons(cscov);
|
|
|
|
/*
|
|
|
|
* For UDP-Lite, checksum coverage length of zero means
|
|
|
|
* the entire UDPLite packet is covered by the checksum.
|
|
|
|
*/
|
|
|
|
cscov_partial = (cscov == 0) ? 0 : 1;
|
|
|
|
} else if (plen <= 0xffff)
|
2007-07-23 07:58:58 +00:00
|
|
|
udp6->uh_ulen = htons((u_short)plen);
|
|
|
|
else
|
|
|
|
udp6->uh_ulen = 0;
|
|
|
|
udp6->uh_sum = 0;
|
|
|
|
|
2018-08-23 16:54:22 +00:00
|
|
|
ip6 = mtod(m, struct ip6_hdr *);
|
|
|
|
ip6->ip6_flow = inp->inp_flow & IPV6_FLOWINFO_MASK;
|
|
|
|
ip6->ip6_vfc &= ~IPV6_VERSION_MASK;
|
|
|
|
ip6->ip6_vfc |= IPV6_VERSION;
|
|
|
|
ip6->ip6_plen = htons((u_short)plen);
|
|
|
|
ip6->ip6_nxt = nxt;
|
|
|
|
ip6->ip6_hlim = in6_selecthlim(inp, NULL);
|
|
|
|
ip6->ip6_src = *laddr;
|
|
|
|
ip6->ip6_dst = *faddr;
|
|
|
|
|
|
|
|
#ifdef MAC
|
|
|
|
mac_inpcb_create_mbuf(inp, m);
|
|
|
|
#endif
|
2007-07-23 07:58:58 +00:00
|
|
|
|
2018-08-23 16:54:22 +00:00
|
|
|
if (cscov_partial) {
|
|
|
|
if ((udp6->uh_sum = in6_cksum_partial(m, nxt,
|
|
|
|
sizeof(struct ip6_hdr), plen, cscov)) == 0)
|
|
|
|
udp6->uh_sum = 0xffff;
|
|
|
|
} else {
|
|
|
|
udp6->uh_sum = in6_cksum_pseudo(ip6, plen, nxt, 0);
|
|
|
|
m->m_pkthdr.csum_flags = CSUM_UDP_IPV6;
|
|
|
|
m->m_pkthdr.csum_data = offsetof(struct udphdr, uh_sum);
|
|
|
|
}
|
|
|
|
|
|
|
|
flags = 0;
|
2020-10-18 17:15:47 +00:00
|
|
|
#if defined(ROUTE_MPATH) || defined(RSS)
|
|
|
|
if (CALC_FLOWID_OUTBOUND_SENDTO) {
|
|
|
|
uint32_t hash_type, hash_val;
|
2018-08-23 16:54:22 +00:00
|
|
|
uint8_t pr;
|
2015-08-25 06:12:59 +00:00
|
|
|
|
2018-08-23 16:54:22 +00:00
|
|
|
pr = inp->inp_socket->so_proto->pr_protocol;
|
|
|
|
|
2020-10-18 17:15:47 +00:00
|
|
|
hash_val = fib6_calc_packet_hash(laddr, faddr,
|
|
|
|
inp->inp_lport, fport, pr, &hash_type);
|
|
|
|
m->m_pkthdr.flowid = hash_val;
|
|
|
|
M_HASHTYPE_SET(m, hash_type);
|
2018-08-24 21:49:21 +00:00
|
|
|
}
|
2020-10-18 17:15:47 +00:00
|
|
|
/* do not use inp flowid */
|
|
|
|
flags |= IP_NODEFAULTFLOWID;
|
2014-09-09 04:20:53 +00:00
|
|
|
#endif
|
|
|
|
|
2018-08-23 16:54:22 +00:00
|
|
|
UDPSTAT_INC(udps_opackets);
|
|
|
|
if (nxt == IPPROTO_UDPLITE)
|
|
|
|
UDPLITE_PROBE(send, NULL, inp, ip6, inp, udp6);
|
|
|
|
else
|
|
|
|
UDP_PROBE(send, NULL, inp, ip6, inp, udp6);
|
2018-09-19 18:49:37 +00:00
|
|
|
error = ip6_output(m, optp,
|
Now with epoch synchronized PCB lookup tables we can greatly simplify
locking in udp_output() and udp6_output().
First, we select if we need read or write lock in PCB itself, we take
the lock and enter network epoch. Then, we proceed for the rest of
the function. In case if we need to modify PCB hash, we would take
write lock on it for a short piece of code.
We could exit the epoch before allocating an mbuf, but with this
patch we are keeping it all the way into ip_output()/ip6_output().
Today this creates an epoch recursion, since ip_output() enters epoch
itself. However, once all protocols are reviewed, ip_output() and
ip6_output() would require epoch instead of entering it.
Note: I'm not 100% sure that in udp6_output() the epoch is required.
We don't do PCB hash lookup for a bound socket. And all branches of
in6_select_src() don't require epoch, at least they lack assertions.
Today inet6 address list is protected by rmlock, although it is CKLIST.
AFAIU, the future plan is to protect it by network epoch. That would
require epoch in in6_select_src(). Anyway, in future ip6_output()
would require epoch, udp6_output() would need to enter it.
2019-11-07 21:01:36 +00:00
|
|
|
INP_WLOCKED(inp) ? &inp->inp_route6 : NULL, flags,
|
2018-08-23 16:54:22 +00:00
|
|
|
inp->in6p_moptions, NULL, inp);
|
Now with epoch synchronized PCB lookup tables we can greatly simplify
locking in udp_output() and udp6_output().
First, we select if we need read or write lock in PCB itself, we take
the lock and enter network epoch. Then, we proceed for the rest of
the function. In case if we need to modify PCB hash, we would take
write lock on it for a short piece of code.
We could exit the epoch before allocating an mbuf, but with this
patch we are keeping it all the way into ip_output()/ip6_output().
Today this creates an epoch recursion, since ip_output() enters epoch
itself. However, once all protocols are reviewed, ip_output() and
ip6_output() would require epoch instead of entering it.
Note: I'm not 100% sure that in udp6_output() the epoch is required.
We don't do PCB hash lookup for a bound socket. And all branches of
in6_select_src() don't require epoch, at least they lack assertions.
Today inet6 address list is protected by rmlock, although it is CKLIST.
AFAIU, the future plan is to protect it by network epoch. That would
require epoch in in6_select_src(). Anyway, in future ip6_output()
would require epoch, udp6_output() would need to enter it.
2019-11-07 21:01:36 +00:00
|
|
|
INP_UNLOCK(inp);
|
|
|
|
NET_EPOCH_EXIT(et);
|
2018-08-23 16:54:22 +00:00
|
|
|
|
|
|
|
if (control) {
|
|
|
|
ip6_clearpktopts(&opt, -1);
|
|
|
|
m_freem(control);
|
2007-07-23 07:58:58 +00:00
|
|
|
}
|
2018-08-23 16:54:22 +00:00
|
|
|
return (error);
|
2007-07-23 07:58:58 +00:00
|
|
|
|
|
|
|
release:
|
Now with epoch synchronized PCB lookup tables we can greatly simplify
locking in udp_output() and udp6_output().
First, we select if we need read or write lock in PCB itself, we take
the lock and enter network epoch. Then, we proceed for the rest of
the function. In case if we need to modify PCB hash, we would take
write lock on it for a short piece of code.
We could exit the epoch before allocating an mbuf, but with this
patch we are keeping it all the way into ip_output()/ip6_output().
Today this creates an epoch recursion, since ip_output() enters epoch
itself. However, once all protocols are reviewed, ip_output() and
ip6_output() would require epoch instead of entering it.
Note: I'm not 100% sure that in udp6_output() the epoch is required.
We don't do PCB hash lookup for a bound socket. And all branches of
in6_select_src() don't require epoch, at least they lack assertions.
Today inet6 address list is protected by rmlock, although it is CKLIST.
AFAIU, the future plan is to protect it by network epoch. That would
require epoch in in6_select_src(). Anyway, in future ip6_output()
would require epoch, udp6_output() would need to enter it.
2019-11-07 21:01:36 +00:00
|
|
|
INP_UNLOCK(inp);
|
|
|
|
NET_EPOCH_EXIT(et);
|
2007-07-23 07:58:58 +00:00
|
|
|
if (control) {
|
|
|
|
ip6_clearpktopts(&opt, -1);
|
|
|
|
m_freem(control);
|
|
|
|
}
|
2018-08-23 16:54:22 +00:00
|
|
|
m_freem(m);
|
|
|
|
|
2007-07-23 07:58:58 +00:00
|
|
|
return (error);
|
|
|
|
}
|
|
|
|
|
2006-04-01 15:15:05 +00:00
|
|
|
static void
|
1999-12-07 17:39:16 +00:00
|
|
|
udp6_abort(struct socket *so)
|
|
|
|
{
|
|
|
|
struct inpcb *inp;
|
2014-04-07 01:53:03 +00:00
|
|
|
struct inpcbinfo *pcbinfo;
|
1999-12-07 17:39:16 +00:00
|
|
|
|
2015-08-27 15:27:41 +00:00
|
|
|
pcbinfo = udp_get_inpcbinfo(so->so_proto->pr_protocol);
|
1999-12-07 17:39:16 +00:00
|
|
|
inp = sotoinpcb(so);
|
Update in_pcb-derived basic socket types following changes to
pru_abort(), pru_detach(), and in_pcbdetach():
- Universally support and enforce the invariant that so_pcb is
never NULL, converting dozens of unnecessary NULL checks into
assertions, and eliminating dozens of unnecessary error handling
cases in protocol code.
- In some cases, eliminate unnecessary pcbinfo locking, as it is no
longer required to ensure so_pcb != NULL. For example, in protocol
shutdown methods, and in raw IP send.
- Abort and detach protocol switch methods no longer return failures,
nor attempt to free sockets, as the socket layer does this.
- Invoke in_pcbfree() after in_pcbdetach() in order to free the
detached in_pcb structure for a socket.
MFC after: 3 months
2006-04-01 16:20:54 +00:00
|
|
|
KASSERT(inp != NULL, ("udp6_abort: inp == NULL"));
|
2006-04-12 03:03:47 +00:00
|
|
|
|
2015-09-21 12:32:36 +00:00
|
|
|
INP_WLOCK(inp);
|
2006-07-21 17:11:15 +00:00
|
|
|
#ifdef INET
|
|
|
|
if (inp->inp_vflag & INP_IPV4) {
|
2015-09-21 12:32:36 +00:00
|
|
|
INP_WUNLOCK(inp);
|
2022-08-16 19:40:36 +00:00
|
|
|
udp_abort(so);
|
2006-07-21 17:11:15 +00:00
|
|
|
return;
|
|
|
|
}
|
|
|
|
#endif
|
|
|
|
|
|
|
|
if (!IN6_IS_ADDR_UNSPECIFIED(&inp->in6p_faddr)) {
|
2014-04-07 01:53:03 +00:00
|
|
|
INP_HASH_WLOCK(pcbinfo);
|
2006-07-21 17:11:15 +00:00
|
|
|
in6_pcbdisconnect(inp);
|
2014-04-07 01:53:03 +00:00
|
|
|
INP_HASH_WUNLOCK(pcbinfo);
|
2006-07-21 17:11:15 +00:00
|
|
|
soisdisconnected(so);
|
|
|
|
}
|
2008-04-17 21:38:18 +00:00
|
|
|
INP_WUNLOCK(inp);
|
1999-12-07 17:39:16 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
static int
|
2001-09-12 08:38:13 +00:00
|
|
|
udp6_attach(struct socket *so, int proto, struct thread *td)
|
1999-12-07 17:39:16 +00:00
|
|
|
{
|
2014-04-07 01:53:03 +00:00
|
|
|
struct inpcbinfo *pcbinfo;
|
2023-01-26 18:16:32 +00:00
|
|
|
struct inpcb *inp;
|
|
|
|
struct udpcb *up;
|
2006-04-12 03:03:47 +00:00
|
|
|
int error;
|
1999-12-07 17:39:16 +00:00
|
|
|
|
2015-08-27 15:27:41 +00:00
|
|
|
pcbinfo = udp_get_inpcbinfo(so->so_proto->pr_protocol);
|
1999-12-07 17:39:16 +00:00
|
|
|
inp = sotoinpcb(so);
|
2006-04-09 16:33:41 +00:00
|
|
|
KASSERT(inp == NULL, ("udp6_attach: inp != NULL"));
|
2006-04-12 03:03:47 +00:00
|
|
|
|
1999-12-07 17:39:16 +00:00
|
|
|
if (so->so_snd.sb_hiwat == 0 || so->so_rcv.sb_hiwat == 0) {
|
|
|
|
error = soreserve(so, udp_sendspace, udp_recvspace);
|
Update in_pcb-derived basic socket types following changes to
pru_abort(), pru_detach(), and in_pcbdetach():
- Universally support and enforce the invariant that so_pcb is
never NULL, converting dozens of unnecessary NULL checks into
assertions, and eliminating dozens of unnecessary error handling
cases in protocol code.
- In some cases, eliminate unnecessary pcbinfo locking, as it is no
longer required to ensure so_pcb != NULL. For example, in protocol
shutdown methods, and in raw IP send.
- Abort and detach protocol switch methods no longer return failures,
nor attempt to free sockets, as the socket layer does this.
- Invoke in_pcbfree() after in_pcbdetach() in order to free the
detached in_pcb structure for a socket.
MFC after: 3 months
2006-04-01 16:20:54 +00:00
|
|
|
if (error)
|
2007-07-09 17:47:04 +00:00
|
|
|
return (error);
|
1999-12-07 17:39:16 +00:00
|
|
|
}
|
2014-04-07 01:53:03 +00:00
|
|
|
error = in_pcballoc(so, pcbinfo);
|
2021-12-02 22:45:04 +00:00
|
|
|
if (error)
|
2007-07-09 17:47:04 +00:00
|
|
|
return (error);
|
1999-12-07 17:39:16 +00:00
|
|
|
inp = (struct inpcb *)so->so_pcb;
|
|
|
|
inp->in6p_cksum = -1; /* just to be sure */
|
2000-06-22 16:48:59 +00:00
|
|
|
/*
|
|
|
|
* XXX: ugly!!
|
|
|
|
* IPv4 TTL initialization is necessary for an IPv6 socket as well,
|
|
|
|
* because the socket may be bound to an IPv6 wildcard address,
|
|
|
|
* which may match an IPv4-mapped IPv6 address.
|
|
|
|
*/
|
Commit step 1 of the vimage project, (network stack)
virtualization work done by Marko Zec (zec@).
This is the first in a series of commits over the course
of the next few weeks.
Mark all uses of global variables to be virtualized
with a V_ prefix.
Use macros to map them back to their global names for
now, so this is a NOP change only.
We hope to have caught at least 85-90% of what is needed
so we do not invalidate a lot of outstanding patches again.
Obtained from: //depot/projects/vimage-commit2/...
Reviewed by: brooks, des, ed, mav, julian,
jamie, kris, rwatson, zec, ...
(various people I forgot, different versions)
md5 (with a bit of help)
Sponsored by: NLnet Foundation, The FreeBSD Foundation
X-MFC after: never
V_Commit_Message_Reviewed_By: more people than the patch
2008-08-17 23:27:27 +00:00
|
|
|
inp->inp_ip_ttl = V_ip_defttl;
|
2023-01-26 18:16:32 +00:00
|
|
|
up = intoudpcb(inp);
|
|
|
|
bzero(&up->u_start_zero, u_zero_size);
|
2008-04-17 21:38:18 +00:00
|
|
|
INP_WUNLOCK(inp);
|
2007-07-09 17:47:04 +00:00
|
|
|
return (0);
|
1999-12-07 17:39:16 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
static int
|
2001-09-12 08:38:13 +00:00
|
|
|
udp6_bind(struct socket *so, struct sockaddr *nam, struct thread *td)
|
1999-12-07 17:39:16 +00:00
|
|
|
{
|
2023-02-15 18:30:16 +00:00
|
|
|
struct sockaddr_in6 *sin6_p;
|
1999-12-07 17:39:16 +00:00
|
|
|
struct inpcb *inp;
|
2014-04-07 01:53:03 +00:00
|
|
|
struct inpcbinfo *pcbinfo;
|
2006-04-12 03:03:47 +00:00
|
|
|
int error;
|
2019-10-24 20:05:10 +00:00
|
|
|
u_char vflagsav;
|
1999-12-07 17:39:16 +00:00
|
|
|
|
2015-08-27 15:27:41 +00:00
|
|
|
pcbinfo = udp_get_inpcbinfo(so->so_proto->pr_protocol);
|
1999-12-07 17:39:16 +00:00
|
|
|
inp = sotoinpcb(so);
|
Update in_pcb-derived basic socket types following changes to
pru_abort(), pru_detach(), and in_pcbdetach():
- Universally support and enforce the invariant that so_pcb is
never NULL, converting dozens of unnecessary NULL checks into
assertions, and eliminating dozens of unnecessary error handling
cases in protocol code.
- In some cases, eliminate unnecessary pcbinfo locking, as it is no
longer required to ensure so_pcb != NULL. For example, in protocol
shutdown methods, and in raw IP send.
- Abort and detach protocol switch methods no longer return failures,
nor attempt to free sockets, as the socket layer does this.
- Invoke in_pcbfree() after in_pcbdetach() in order to free the
detached in_pcb structure for a socket.
MFC after: 3 months
2006-04-01 16:20:54 +00:00
|
|
|
KASSERT(inp != NULL, ("udp6_bind: inp == NULL"));
|
2006-04-12 03:03:47 +00:00
|
|
|
|
2021-05-03 16:51:04 +00:00
|
|
|
if (nam->sa_family != AF_INET6)
|
|
|
|
return (EAFNOSUPPORT);
|
|
|
|
if (nam->sa_len != sizeof(struct sockaddr_in6))
|
|
|
|
return (EINVAL);
|
|
|
|
|
2023-02-15 18:30:16 +00:00
|
|
|
sin6_p = (struct sockaddr_in6 *)nam;
|
|
|
|
|
2008-04-17 21:38:18 +00:00
|
|
|
INP_WLOCK(inp);
|
2014-04-07 01:53:03 +00:00
|
|
|
INP_HASH_WLOCK(pcbinfo);
|
2019-10-24 20:05:10 +00:00
|
|
|
vflagsav = inp->inp_vflag;
|
1999-12-07 17:39:16 +00:00
|
|
|
inp->inp_vflag &= ~INP_IPV4;
|
|
|
|
inp->inp_vflag |= INP_IPV6;
|
2001-06-11 12:39:29 +00:00
|
|
|
if ((inp->inp_flags & IN6P_IPV6_V6ONLY) == 0) {
|
1999-12-07 17:39:16 +00:00
|
|
|
if (IN6_IS_ADDR_UNSPECIFIED(&sin6_p->sin6_addr))
|
|
|
|
inp->inp_vflag |= INP_IPV4;
|
2011-04-30 11:17:00 +00:00
|
|
|
#ifdef INET
|
1999-12-07 17:39:16 +00:00
|
|
|
else if (IN6_IS_ADDR_V4MAPPED(&sin6_p->sin6_addr)) {
|
|
|
|
struct sockaddr_in sin;
|
|
|
|
|
|
|
|
in6_sin6_2_sin(&sin, sin6_p);
|
|
|
|
inp->inp_vflag |= INP_IPV4;
|
|
|
|
inp->inp_vflag &= ~INP_IPV6;
|
2023-02-15 18:30:16 +00:00
|
|
|
error = in_pcbbind(inp, &sin, td->td_ucred);
|
2004-07-27 23:44:03 +00:00
|
|
|
goto out;
|
1999-12-07 17:39:16 +00:00
|
|
|
}
|
2011-04-30 11:17:00 +00:00
|
|
|
#endif
|
1999-12-07 17:39:16 +00:00
|
|
|
}
|
|
|
|
|
2023-02-15 18:30:16 +00:00
|
|
|
error = in6_pcbbind(inp, sin6_p, td->td_ucred);
|
2011-04-30 11:17:00 +00:00
|
|
|
#ifdef INET
|
2004-07-27 23:44:03 +00:00
|
|
|
out:
|
2011-04-30 11:17:00 +00:00
|
|
|
#endif
|
2019-10-24 20:05:10 +00:00
|
|
|
if (error != 0)
|
|
|
|
inp->inp_vflag = vflagsav;
|
2014-04-07 01:53:03 +00:00
|
|
|
INP_HASH_WUNLOCK(pcbinfo);
|
2008-04-17 21:38:18 +00:00
|
|
|
INP_WUNLOCK(inp);
|
2007-07-09 17:47:04 +00:00
|
|
|
return (error);
|
1999-12-07 17:39:16 +00:00
|
|
|
}
|
|
|
|
|
2006-07-21 17:11:15 +00:00
|
|
|
static void
|
|
|
|
udp6_close(struct socket *so)
|
|
|
|
{
|
|
|
|
struct inpcb *inp;
|
2014-04-07 01:53:03 +00:00
|
|
|
struct inpcbinfo *pcbinfo;
|
2006-07-21 17:11:15 +00:00
|
|
|
|
2015-08-27 15:27:41 +00:00
|
|
|
pcbinfo = udp_get_inpcbinfo(so->so_proto->pr_protocol);
|
2006-07-21 17:11:15 +00:00
|
|
|
inp = sotoinpcb(so);
|
|
|
|
KASSERT(inp != NULL, ("udp6_close: inp == NULL"));
|
|
|
|
|
2015-09-21 12:32:36 +00:00
|
|
|
INP_WLOCK(inp);
|
2006-07-21 17:11:15 +00:00
|
|
|
#ifdef INET
|
|
|
|
if (inp->inp_vflag & INP_IPV4) {
|
2015-09-21 12:32:36 +00:00
|
|
|
INP_WUNLOCK(inp);
|
2022-08-16 19:40:36 +00:00
|
|
|
(void)udp_disconnect(so);
|
2006-07-21 17:11:15 +00:00
|
|
|
return;
|
|
|
|
}
|
|
|
|
#endif
|
|
|
|
if (!IN6_IS_ADDR_UNSPECIFIED(&inp->in6p_faddr)) {
|
2014-04-07 01:53:03 +00:00
|
|
|
INP_HASH_WLOCK(pcbinfo);
|
2006-07-21 17:11:15 +00:00
|
|
|
in6_pcbdisconnect(inp);
|
2014-04-07 01:53:03 +00:00
|
|
|
INP_HASH_WUNLOCK(pcbinfo);
|
2006-07-21 17:11:15 +00:00
|
|
|
soisdisconnected(so);
|
|
|
|
}
|
2008-04-17 21:38:18 +00:00
|
|
|
INP_WUNLOCK(inp);
|
2006-07-21 17:11:15 +00:00
|
|
|
}
|
|
|
|
|
1999-12-07 17:39:16 +00:00
|
|
|
static int
|
2001-09-12 08:38:13 +00:00
|
|
|
udp6_connect(struct socket *so, struct sockaddr *nam, struct thread *td)
|
1999-12-07 17:39:16 +00:00
|
|
|
{
|
2020-01-22 17:06:55 +00:00
|
|
|
struct epoch_tracker et;
|
1999-12-07 17:39:16 +00:00
|
|
|
struct inpcb *inp;
|
2014-04-07 01:53:03 +00:00
|
|
|
struct inpcbinfo *pcbinfo;
|
2009-02-05 15:04:23 +00:00
|
|
|
struct sockaddr_in6 *sin6;
|
2006-04-12 03:03:47 +00:00
|
|
|
int error;
|
2019-10-24 20:05:10 +00:00
|
|
|
u_char vflagsav;
|
1999-12-07 17:39:16 +00:00
|
|
|
|
2015-08-27 15:27:41 +00:00
|
|
|
pcbinfo = udp_get_inpcbinfo(so->so_proto->pr_protocol);
|
1999-12-07 17:39:16 +00:00
|
|
|
inp = sotoinpcb(so);
|
Update in_pcb-derived basic socket types following changes to
pru_abort(), pru_detach(), and in_pcbdetach():
- Universally support and enforce the invariant that so_pcb is
never NULL, converting dozens of unnecessary NULL checks into
assertions, and eliminating dozens of unnecessary error handling
cases in protocol code.
- In some cases, eliminate unnecessary pcbinfo locking, as it is no
longer required to ensure so_pcb != NULL. For example, in protocol
shutdown methods, and in raw IP send.
- Abort and detach protocol switch methods no longer return failures,
nor attempt to free sockets, as the socket layer does this.
- Invoke in_pcbfree() after in_pcbdetach() in order to free the
detached in_pcb structure for a socket.
MFC after: 3 months
2006-04-01 16:20:54 +00:00
|
|
|
KASSERT(inp != NULL, ("udp6_connect: inp == NULL"));
|
2006-04-12 03:03:47 +00:00
|
|
|
|
2021-05-03 16:51:04 +00:00
|
|
|
sin6 = (struct sockaddr_in6 *)nam;
|
|
|
|
if (sin6->sin6_family != AF_INET6)
|
|
|
|
return (EAFNOSUPPORT);
|
|
|
|
if (sin6->sin6_len != sizeof(*sin6))
|
|
|
|
return (EINVAL);
|
|
|
|
|
Decompose the current single inpcbinfo lock into two locks:
- The existing ipi_lock continues to protect the global inpcb list and
inpcb counter. This lock is now relegated to a small number of
allocation and free operations, and occasional operations that walk
all connections (including, awkwardly, certain UDP multicast receive
operations -- something to revisit).
- A new ipi_hash_lock protects the two inpcbinfo hash tables for
looking up connections and bound sockets, manipulated using new
INP_HASH_*() macros. This lock, combined with inpcb locks, protects
the 4-tuple address space.
Unlike the current ipi_lock, ipi_hash_lock follows the individual inpcb
connection locks, so may be acquired while manipulating a connection on
which a lock is already held, avoiding the need to acquire the inpcbinfo
lock preemptively when a binding change might later be required. As a
result, however, lookup operations necessarily go through a reference
acquire while holding the lookup lock, later acquiring an inpcb lock --
if required.
A new function in_pcblookup() looks up connections, and accepts flags
indicating how to return the inpcb. Due to lock order changes, callers
no longer need acquire locks before performing a lookup: the lookup
routine will acquire the ipi_hash_lock as needed. In the future, it will
also be able to use alternative lookup and locking strategies
transparently to callers, such as pcbgroup lookup. New lookup flags are,
supplementing the existing INPLOOKUP_WILDCARD flag:
INPLOOKUP_RLOCKPCB - Acquire a read lock on the returned inpcb
INPLOOKUP_WLOCKPCB - Acquire a write lock on the returned inpcb
Callers must pass exactly one of these flags (for the time being).
Some notes:
- All protocols are updated to work within the new regime; especially,
TCP, UDPv4, and UDPv6. pcbinfo ipi_lock acquisitions are largely
eliminated, and global hash lock hold times are dramatically reduced
compared to previous locking.
- The TCP syncache still relies on the pcbinfo lock, something that we
may want to revisit.
- Support for reverting to the FreeBSD 7.x locking strategy in TCP input
is no longer available -- hash lookup locks are now held only very
briefly during inpcb lookup, rather than for potentially extended
periods. However, the pcbinfo ipi_lock will still be acquired if a
connection state might change such that a connection is added or
removed.
- Raw IP sockets continue to use the pcbinfo ipi_lock for protection,
due to maintaining their own hash tables.
- The interface in6_pcblookup_hash_locked() is maintained, which allows
callers to acquire hash locks and perform one or more lookups atomically
with 4-tuple allocation: this is required only for TCPv6, as there is no
in6_pcbconnect_setup(), which there should be.
- UDPv6 locking remains significantly more conservative than UDPv4
locking, which relates to source address selection. This needs
attention, as it likely significantly reduces parallelism in this code
for multithreaded socket use (such as in BIND).
- In the UDPv4 and UDPv6 multicast cases, we need to revisit locking
somewhat, as they relied on ipi_lock to stablise 4-tuple matches, which
is no longer sufficient. A second check once the inpcb lock is held
should do the trick, keeping the general case from requiring the inpcb
lock for every inpcb visited.
- This work reminds us that we need to revisit locking of the v4/v6 flags,
which may be accessed lock-free both before and after this change.
- Right now, a single lock name is used for the pcbhash lock -- this is
undesirable, and probably another argument is required to take care of
this (or a char array name field in the pcbinfo?).
This is not an MFC candidate for 8.x due to its impact on lookup and
locking semantics. It's possible some of these issues could be worked
around with compatibility wrappers, if necessary.
Reviewed by: bz
Sponsored by: Juniper Networks, Inc.
2011-05-30 09:43:55 +00:00
|
|
|
/*
|
|
|
|
* XXXRW: Need to clarify locking of v4/v6 flags.
|
|
|
|
*/
|
2008-04-17 21:38:18 +00:00
|
|
|
INP_WLOCK(inp);
|
2011-04-30 11:17:00 +00:00
|
|
|
#ifdef INET
|
2011-04-09 01:29:46 +00:00
|
|
|
if (IN6_IS_ADDR_V4MAPPED(&sin6->sin6_addr)) {
|
2009-02-05 15:04:23 +00:00
|
|
|
struct sockaddr_in sin;
|
1999-12-07 17:39:16 +00:00
|
|
|
|
2011-04-09 01:29:46 +00:00
|
|
|
if ((inp->inp_flags & IN6P_IPV6_V6ONLY) != 0) {
|
|
|
|
error = EINVAL;
|
|
|
|
goto out;
|
|
|
|
}
|
2017-05-22 15:29:10 +00:00
|
|
|
if ((inp->inp_vflag & INP_IPV4) == 0) {
|
|
|
|
error = EAFNOSUPPORT;
|
|
|
|
goto out;
|
|
|
|
}
|
2009-02-05 15:04:23 +00:00
|
|
|
if (inp->inp_faddr.s_addr != INADDR_ANY) {
|
|
|
|
error = EISCONN;
|
|
|
|
goto out;
|
|
|
|
}
|
|
|
|
in6_sin6_2_sin(&sin, sin6);
|
|
|
|
error = prison_remote_ip4(td->td_ucred, &sin.sin_addr);
|
|
|
|
if (error != 0)
|
2004-07-27 23:44:03 +00:00
|
|
|
goto out;
|
2019-10-24 20:05:10 +00:00
|
|
|
vflagsav = inp->inp_vflag;
|
|
|
|
inp->inp_vflag |= INP_IPV4;
|
|
|
|
inp->inp_vflag &= ~INP_IPV6;
|
2020-01-22 17:06:55 +00:00
|
|
|
NET_EPOCH_ENTER(et);
|
2014-04-07 01:53:03 +00:00
|
|
|
INP_HASH_WLOCK(pcbinfo);
|
2023-02-03 19:33:36 +00:00
|
|
|
error = in_pcbconnect(inp, &sin, td->td_ucred, true);
|
2014-04-07 01:53:03 +00:00
|
|
|
INP_HASH_WUNLOCK(pcbinfo);
|
2020-01-22 17:06:55 +00:00
|
|
|
NET_EPOCH_EXIT(et);
|
2019-10-24 20:05:10 +00:00
|
|
|
/*
|
|
|
|
* If connect succeeds, mark socket as connected. If
|
|
|
|
* connect fails and socket is unbound, reset inp_vflag
|
|
|
|
* field.
|
|
|
|
*/
|
2011-04-09 01:29:46 +00:00
|
|
|
if (error == 0)
|
2009-02-05 15:04:23 +00:00
|
|
|
soisconnected(so);
|
2019-10-24 20:05:10 +00:00
|
|
|
else if (inp->inp_laddr.s_addr == INADDR_ANY &&
|
|
|
|
inp->inp_lport == 0)
|
|
|
|
inp->inp_vflag = vflagsav;
|
2009-02-05 15:04:23 +00:00
|
|
|
goto out;
|
2017-05-22 15:29:10 +00:00
|
|
|
} else {
|
|
|
|
if ((inp->inp_vflag & INP_IPV6) == 0) {
|
|
|
|
error = EAFNOSUPPORT;
|
|
|
|
goto out;
|
|
|
|
}
|
1999-12-07 17:39:16 +00:00
|
|
|
}
|
2011-04-30 11:17:00 +00:00
|
|
|
#endif
|
2004-07-27 23:44:03 +00:00
|
|
|
if (!IN6_IS_ADDR_UNSPECIFIED(&inp->in6p_faddr)) {
|
|
|
|
error = EISCONN;
|
|
|
|
goto out;
|
|
|
|
}
|
2009-02-05 15:04:23 +00:00
|
|
|
error = prison_remote_ip6(td->td_ucred, &sin6->sin6_addr);
|
|
|
|
if (error != 0)
|
|
|
|
goto out;
|
2019-10-24 20:05:10 +00:00
|
|
|
vflagsav = inp->inp_vflag;
|
|
|
|
inp->inp_vflag &= ~INP_IPV4;
|
|
|
|
inp->inp_vflag |= INP_IPV6;
|
2023-02-03 19:33:36 +00:00
|
|
|
NET_EPOCH_ENTER(et);
|
2014-04-07 01:53:03 +00:00
|
|
|
INP_HASH_WLOCK(pcbinfo);
|
2023-02-03 19:33:36 +00:00
|
|
|
error = in6_pcbconnect(inp, sin6, td->td_ucred, true);
|
2014-04-07 01:53:03 +00:00
|
|
|
INP_HASH_WUNLOCK(pcbinfo);
|
2023-02-03 19:33:36 +00:00
|
|
|
NET_EPOCH_EXIT(et);
|
2019-10-24 20:05:10 +00:00
|
|
|
/*
|
|
|
|
* If connect succeeds, mark socket as connected. If
|
|
|
|
* connect fails and socket is unbound, reset inp_vflag
|
|
|
|
* field.
|
|
|
|
*/
|
2011-04-09 01:29:46 +00:00
|
|
|
if (error == 0)
|
1999-12-07 17:39:16 +00:00
|
|
|
soisconnected(so);
|
2019-10-24 20:05:10 +00:00
|
|
|
else if (IN6_IS_ADDR_UNSPECIFIED(&inp->in6p_laddr) &&
|
|
|
|
inp->inp_lport == 0)
|
|
|
|
inp->inp_vflag = vflagsav;
|
2004-07-27 23:44:03 +00:00
|
|
|
out:
|
2008-04-17 21:38:18 +00:00
|
|
|
INP_WUNLOCK(inp);
|
2007-07-09 17:47:04 +00:00
|
|
|
return (error);
|
1999-12-07 17:39:16 +00:00
|
|
|
}
|
|
|
|
|
Chance protocol switch method pru_detach() so that it returns void
rather than an error. Detaches do not "fail", they other occur or
the protocol flags SS_PROTOREF to take ownership of the socket.
soclose() no longer looks at so_pcb to see if it's NULL, relying
entirely on the protocol to decide whether it's time to free the
socket or not using SS_PROTOREF. so_pcb is now entirely owned and
managed by the protocol code. Likewise, no longer test so_pcb in
other socket functions, such as soreceive(), which have no business
digging into protocol internals.
Protocol detach routines no longer try to free the socket on detach,
this is performed in the socket code if the protocol permits it.
In rts_detach(), no longer test for rp != NULL in detach, and
likewise in other protocols that don't permit a NULL so_pcb, reduce
the incidence of testing for it during detach.
netinet and netinet6 are not fully updated to this change, which
will be in an upcoming commit. In their current state they may leak
memory or panic.
MFC after: 3 months
2006-04-01 15:42:02 +00:00
|
|
|
static void
|
1999-12-07 17:39:16 +00:00
|
|
|
udp6_detach(struct socket *so)
|
|
|
|
{
|
|
|
|
struct inpcb *inp;
|
|
|
|
|
|
|
|
inp = sotoinpcb(so);
|
Update in_pcb-derived basic socket types following changes to
pru_abort(), pru_detach(), and in_pcbdetach():
- Universally support and enforce the invariant that so_pcb is
never NULL, converting dozens of unnecessary NULL checks into
assertions, and eliminating dozens of unnecessary error handling
cases in protocol code.
- In some cases, eliminate unnecessary pcbinfo locking, as it is no
longer required to ensure so_pcb != NULL. For example, in protocol
shutdown methods, and in raw IP send.
- Abort and detach protocol switch methods no longer return failures,
nor attempt to free sockets, as the socket layer does this.
- Invoke in_pcbfree() after in_pcbdetach() in order to free the
detached in_pcb structure for a socket.
MFC after: 3 months
2006-04-01 16:20:54 +00:00
|
|
|
KASSERT(inp != NULL, ("udp6_detach: inp == NULL"));
|
2006-04-12 03:03:47 +00:00
|
|
|
|
2008-04-17 21:38:18 +00:00
|
|
|
INP_WLOCK(inp);
|
2008-11-26 20:52:26 +00:00
|
|
|
in_pcbdetach(inp);
|
2008-11-27 12:04:35 +00:00
|
|
|
in_pcbfree(inp);
|
1999-12-07 17:39:16 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
static int
|
|
|
|
udp6_disconnect(struct socket *so)
|
|
|
|
{
|
|
|
|
struct inpcb *inp;
|
2014-04-07 01:53:03 +00:00
|
|
|
struct inpcbinfo *pcbinfo;
|
1999-12-07 17:39:16 +00:00
|
|
|
|
2015-08-27 15:27:41 +00:00
|
|
|
pcbinfo = udp_get_inpcbinfo(so->so_proto->pr_protocol);
|
1999-12-07 17:39:16 +00:00
|
|
|
inp = sotoinpcb(so);
|
Update in_pcb-derived basic socket types following changes to
pru_abort(), pru_detach(), and in_pcbdetach():
- Universally support and enforce the invariant that so_pcb is
never NULL, converting dozens of unnecessary NULL checks into
assertions, and eliminating dozens of unnecessary error handling
cases in protocol code.
- In some cases, eliminate unnecessary pcbinfo locking, as it is no
longer required to ensure so_pcb != NULL. For example, in protocol
shutdown methods, and in raw IP send.
- Abort and detach protocol switch methods no longer return failures,
nor attempt to free sockets, as the socket layer does this.
- Invoke in_pcbfree() after in_pcbdetach() in order to free the
detached in_pcb structure for a socket.
MFC after: 3 months
2006-04-01 16:20:54 +00:00
|
|
|
KASSERT(inp != NULL, ("udp6_disconnect: inp == NULL"));
|
2006-04-12 03:03:47 +00:00
|
|
|
|
2015-09-21 12:32:36 +00:00
|
|
|
INP_WLOCK(inp);
|
2004-02-13 15:11:47 +00:00
|
|
|
#ifdef INET
|
1999-12-07 17:39:16 +00:00
|
|
|
if (inp->inp_vflag & INP_IPV4) {
|
2015-09-21 12:32:36 +00:00
|
|
|
INP_WUNLOCK(inp);
|
2022-08-16 19:40:36 +00:00
|
|
|
(void)udp_disconnect(so);
|
Decompose the current single inpcbinfo lock into two locks:
- The existing ipi_lock continues to protect the global inpcb list and
inpcb counter. This lock is now relegated to a small number of
allocation and free operations, and occasional operations that walk
all connections (including, awkwardly, certain UDP multicast receive
operations -- something to revisit).
- A new ipi_hash_lock protects the two inpcbinfo hash tables for
looking up connections and bound sockets, manipulated using new
INP_HASH_*() macros. This lock, combined with inpcb locks, protects
the 4-tuple address space.
Unlike the current ipi_lock, ipi_hash_lock follows the individual inpcb
connection locks, so may be acquired while manipulating a connection on
which a lock is already held, avoiding the need to acquire the inpcbinfo
lock preemptively when a binding change might later be required. As a
result, however, lookup operations necessarily go through a reference
acquire while holding the lookup lock, later acquiring an inpcb lock --
if required.
A new function in_pcblookup() looks up connections, and accepts flags
indicating how to return the inpcb. Due to lock order changes, callers
no longer need acquire locks before performing a lookup: the lookup
routine will acquire the ipi_hash_lock as needed. In the future, it will
also be able to use alternative lookup and locking strategies
transparently to callers, such as pcbgroup lookup. New lookup flags are,
supplementing the existing INPLOOKUP_WILDCARD flag:
INPLOOKUP_RLOCKPCB - Acquire a read lock on the returned inpcb
INPLOOKUP_WLOCKPCB - Acquire a write lock on the returned inpcb
Callers must pass exactly one of these flags (for the time being).
Some notes:
- All protocols are updated to work within the new regime; especially,
TCP, UDPv4, and UDPv6. pcbinfo ipi_lock acquisitions are largely
eliminated, and global hash lock hold times are dramatically reduced
compared to previous locking.
- The TCP syncache still relies on the pcbinfo lock, something that we
may want to revisit.
- Support for reverting to the FreeBSD 7.x locking strategy in TCP input
is no longer available -- hash lookup locks are now held only very
briefly during inpcb lookup, rather than for potentially extended
periods. However, the pcbinfo ipi_lock will still be acquired if a
connection state might change such that a connection is added or
removed.
- Raw IP sockets continue to use the pcbinfo ipi_lock for protection,
due to maintaining their own hash tables.
- The interface in6_pcblookup_hash_locked() is maintained, which allows
callers to acquire hash locks and perform one or more lookups atomically
with 4-tuple allocation: this is required only for TCPv6, as there is no
in6_pcbconnect_setup(), which there should be.
- UDPv6 locking remains significantly more conservative than UDPv4
locking, which relates to source address selection. This needs
attention, as it likely significantly reduces parallelism in this code
for multithreaded socket use (such as in BIND).
- In the UDPv4 and UDPv6 multicast cases, we need to revisit locking
somewhat, as they relied on ipi_lock to stablise 4-tuple matches, which
is no longer sufficient. A second check once the inpcb lock is held
should do the trick, keeping the general case from requiring the inpcb
lock for every inpcb visited.
- This work reminds us that we need to revisit locking of the v4/v6 flags,
which may be accessed lock-free both before and after this change.
- Right now, a single lock name is used for the pcbhash lock -- this is
undesirable, and probably another argument is required to take care of
this (or a char array name field in the pcbinfo?).
This is not an MFC candidate for 8.x due to its impact on lookup and
locking semantics. It's possible some of these issues could be worked
around with compatibility wrappers, if necessary.
Reviewed by: bz
Sponsored by: Juniper Networks, Inc.
2011-05-30 09:43:55 +00:00
|
|
|
return (0);
|
1999-12-07 17:39:16 +00:00
|
|
|
}
|
2004-02-13 15:11:47 +00:00
|
|
|
#endif
|
1999-12-07 17:39:16 +00:00
|
|
|
|
2004-07-27 23:44:03 +00:00
|
|
|
if (IN6_IS_ADDR_UNSPECIFIED(&inp->in6p_faddr)) {
|
2017-12-25 04:48:39 +00:00
|
|
|
INP_WUNLOCK(inp);
|
|
|
|
return (ENOTCONN);
|
2004-07-27 23:44:03 +00:00
|
|
|
}
|
1999-12-07 17:39:16 +00:00
|
|
|
|
2014-04-07 01:53:03 +00:00
|
|
|
INP_HASH_WLOCK(pcbinfo);
|
1999-12-07 17:39:16 +00:00
|
|
|
in6_pcbdisconnect(inp);
|
2014-04-07 01:53:03 +00:00
|
|
|
INP_HASH_WUNLOCK(pcbinfo);
|
2008-10-12 20:01:32 +00:00
|
|
|
SOCK_LOCK(so);
|
1999-12-07 17:39:16 +00:00
|
|
|
so->so_state &= ~SS_ISCONNECTED; /* XXX */
|
2008-10-12 20:01:32 +00:00
|
|
|
SOCK_UNLOCK(so);
|
2008-04-17 21:38:18 +00:00
|
|
|
INP_WUNLOCK(inp);
|
2007-07-09 17:47:04 +00:00
|
|
|
return (0);
|
1999-12-07 17:39:16 +00:00
|
|
|
}
|
|
|
|
|
2022-08-17 18:50:32 +00:00
|
|
|
#define UDP6_PROTOSW \
|
|
|
|
.pr_type = SOCK_DGRAM, \
|
|
|
|
.pr_flags = PR_ATOMIC|PR_ADDR|PR_CAPATTACH, \
|
|
|
|
.pr_ctloutput = ip6_ctloutput, \
|
|
|
|
.pr_abort = udp6_abort, \
|
|
|
|
.pr_attach = udp6_attach, \
|
|
|
|
.pr_bind = udp6_bind, \
|
|
|
|
.pr_connect = udp6_connect, \
|
|
|
|
.pr_control = in6_control, \
|
|
|
|
.pr_detach = udp6_detach, \
|
|
|
|
.pr_disconnect = udp6_disconnect, \
|
|
|
|
.pr_peeraddr = in6_mapped_peeraddr, \
|
|
|
|
.pr_send = udp6_send, \
|
|
|
|
.pr_shutdown = udp_shutdown, \
|
|
|
|
.pr_sockaddr = in6_mapped_sockaddr, \
|
|
|
|
.pr_soreceive = soreceive_dgram, \
|
|
|
|
.pr_sosend = sosend_dgram, \
|
|
|
|
.pr_sosetlabel = in_pcbsosetlabel, \
|
|
|
|
.pr_close = udp6_close
|
|
|
|
|
|
|
|
struct protosw udp6_protosw = {
|
|
|
|
.pr_protocol = IPPROTO_UDP,
|
|
|
|
UDP6_PROTOSW
|
|
|
|
};
|
|
|
|
|
|
|
|
struct protosw udplite6_protosw = {
|
|
|
|
.pr_protocol = IPPROTO_UDPLITE,
|
|
|
|
UDP6_PROTOSW
|
1999-12-07 17:39:16 +00:00
|
|
|
};
|
2022-08-17 18:50:31 +00:00
|
|
|
|
|
|
|
static void
|
|
|
|
udp6_init(void *arg __unused)
|
|
|
|
{
|
|
|
|
|
|
|
|
IP6PROTO_REGISTER(IPPROTO_UDP, udp6_input, udp6_ctlinput);
|
|
|
|
IP6PROTO_REGISTER(IPPROTO_UDPLITE, udp6_input, udplite6_ctlinput);
|
|
|
|
}
|
|
|
|
SYSINIT(udp6_init, SI_SUB_PROTO_DOMAIN, SI_ORDER_THIRD, udp6_init, NULL);
|