d/freebsd-dev
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
d0c9294b81
freebsd-dev/sys/net/if.c

4644 lines
108 KiB
C
Raw Normal View History

/* -> /*- for license, minor formatting changes
2005-01-07 01:45:51 +00:00
/*-
sys: further adoption of SPDX licensing ID tags. Mainly focus on files that use BSD 3-Clause license. The Software Package Data Exchange (SPDX) group provides a specification to make it easier for automated tools to detect and summarize well known opensource licenses. We are gradually adopting the specification, noting that the tags are considered only advisory and do not, in any way, superceed or replace the license texts. Special thanks to Wind River for providing access to "The Duke of Highlander" tool: an older (2014) run over FreeBSD tree was useful as a starting point.
2017-11-20 19:43:44 +00:00
* SPDX-License-Identifier: BSD-3-Clause
*
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
* Copyright (c) 1980, 1986, 1993
* The Regents of the University of California. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
Renumber copyright clause 4 Renumber cluase 4 to 3, per what everybody else did when BSD granted them permission to remove clause 3. My insistance on keeping the same numbering for legal reasons is too pedantic, so give up on that point. Submitted by: Jan Schaumann <jschauma@stevens.edu> Pull Request: https://github.com/freebsd/freebsd/pull/96
2017-02-28 23:42:47 +00:00
* 3. Neither the name of the University nor the names of its contributors
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
* may be used to endorse or promote products derived from this software
* without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND
* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE
* FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
* OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
* LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
* OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
* SUCH DAMAGE.
*
Revision 1.13 corresponded to CSRG revision 8.4. Revision 1.59 corresponded to CSRG revision 8.5.
2001-10-17 10:41:00 +00:00
* @(#)if.c 8.5 (Berkeley) 1/9/95
$Id$ -> $FreeBSD$
1999-08-28 01:08:13 +00:00
* $FreeBSD$
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
*/
udp IPv6 support, IPv6/IPv4 tunneling support in kernel, packet divert at kernel for IPv6/IPv4 translater daemon This includes queue related patch submitted by jburkhol@home.com. Submitted by: queue related patch from jburkhol@home.com Reviewed by: freebsd-arch, cvs-committers Obtained from: KAME project
1999-12-07 17:39:16 +00:00
#include "opt_inet6.h"
Prevent kernel panic at ifconfig up after Note PC resume. Submitted by: imp, kuriyama Reviewed by: imp
1999-12-30 18:29:55 +00:00
#include "opt_inet.h"
Make COMPAT_43 and COMPAT_SUNOS new-style options.
1997-12-16 17:40:42 +00:00
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
#include <sys/param.h>
Introduce network device nodes. Network devices will now automatically appear in /dev. Interface hardware ioctls (not protocol or routing) can be performed on the descriptor. The SIOCGIFCONF ioctl may be performed on the special /dev/network node.
2001-09-29 05:55:04 +00:00
#include <sys/conf.h>
Extract eventfilter declarations to sys/_eventfilter.h This allows replacing "sys/eventfilter.h" includes with "sys/_eventfilter.h" in other header files (e.g., sys/{bus,conf,cpu}.h) and reduces header pollution substantially. EVENTHANDLER_DECLARE and EVENTHANDLER_LIST_DECLAREs were moved out of .c files into appropriate headers (e.g., sys/proc.h, powernv/opal.h). As a side effect of reduced header pollution, many .c files and headers no longer contain needed definitions. The remainder of the patch addresses adding appropriate includes to fix those files. LOCK_DEBUG and LOCK_FILE_LINE_ARG are moved to sys/_lock.h, as required by sys/mutex.h since r326106 (but silently protected by header pollution prior to this change). No functional change (intended). Of course, any out of tree modules that relied on header pollution for sys/eventhandler.h, sys/lock.h, or sys/mutex.h inclusion need to be fixed. __FreeBSD_version has been bumped.
2019-05-20 00:38:23 +00:00
#include <sys/eventhandler.h>
Added used #include - don't depend on <sys/mbuf.h> including <sys/malloc.h> (unless we only use the bogusly shared M*WAIT flags).
1997-09-02 01:19:47 +00:00
#include <sys/malloc.h>
Track device's NUMA domain in ifnet & alloc ifnet from NUMA local memory This commit adds new if_alloc_domain() and if_alloc_dev() methods to allocate ifnets. When called with a domain on a NUMA machine, ifalloc_domain() will record the NUMA domain in the ifnet, and it will allocate the ifnet struct from memory which is local to that NUMA node. Similarly, if_alloc_dev() is a wrapper for if_alloc_domain which uses a driver supplied device_t to call ifalloc_domain() with the appropriate domain. Note that the new if_numa_domain field fits in an alignment pad in struct ifnet, and so does not alter the size of the structure. Reviewed by: glebius, kib, markj Sponsored by: Netflix Differential Revision: https://reviews.freebsd.org/D19930
2019-04-22 19:24:21 +00:00
#include <sys/domainset.h>
Fix a LOR where ifconf() used copyout while holding a mutex. This LOR was seen when configuring addresses on interfaces using ifconfig. This patch has been verified to work with over eight thousand addresses assigned to an interface. LOR id: 031
2004-09-22 08:59:41 +00:00
#include <sys/sbuf.h>
Fix the ``WARNING: Driver mistake: repeat make_dev'', caused by using the wrong index variable within a loop. I have no idea how this managed to work on my test box. Spotted by: fenner
2001-10-11 18:39:05 +00:00
#include <sys/bus.h>
ifnet: Replace if_addr_lock rwlock with epoch + mutex Run on LLNW canaries and tested by pho@ gallatin: Using a 14-core, 28-HTT single socket E5-2697 v3 with a 40GbE MLX5 based ConnectX 4-LX NIC, I see an almost 12% improvement in received packet rate, and a larger improvement in bytes delivered all the way to userspace. When the host receiving 64 streams of netperf -H $DUT -t UDP_STREAM -- -m 1, I see, using nstat -I mce0 1 before the patch: InMpps OMpps InGbs OGbs err TCP Est %CPU syscalls csw irq GBfree 4.98 0.00 4.42 0.00 4235592 33 83.80 4720653 2149771 1235 247.32 4.73 0.00 4.20 0.00 4025260 33 82.99 4724900 2139833 1204 247.32 4.72 0.00 4.20 0.00 4035252 33 82.14 4719162 2132023 1264 247.32 4.71 0.00 4.21 0.00 4073206 33 83.68 4744973 2123317 1347 247.32 4.72 0.00 4.21 0.00 4061118 33 80.82 4713615 2188091 1490 247.32 4.72 0.00 4.21 0.00 4051675 33 85.29 4727399 2109011 1205 247.32 4.73 0.00 4.21 0.00 4039056 33 84.65 4724735 2102603 1053 247.32 After the patch InMpps OMpps InGbs OGbs err TCP Est %CPU syscalls csw irq GBfree 5.43 0.00 4.20 0.00 3313143 33 84.96 5434214 1900162 2656 245.51 5.43 0.00 4.20 0.00 3308527 33 85.24 5439695 1809382 2521 245.51 5.42 0.00 4.19 0.00 3316778 33 87.54 5416028 1805835 2256 245.51 5.42 0.00 4.19 0.00 3317673 33 90.44 5426044 1763056 2332 245.51 5.42 0.00 4.19 0.00 3314839 33 88.11 5435732 1792218 2499 245.52 5.44 0.00 4.19 0.00 3293228 33 91.84 5426301 1668597 2121 245.52 Similarly, netperf reports 230Mb/s before the patch, and 270Mb/s after the patch Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15366
2018-05-18 20:13:34 +00:00
#include <sys/epoch.h>
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
#include <sys/mbuf.h>
#include <sys/systm.h>
Sweep kernel replacing suser(9) calls with priv(9) calls, assigning specific privilege names to a broad range of privileges. These may require some future tweaking. Sponsored by: nCircle Network Security, Inc. Obtained from: TrustedBSD Project Discussed on: arch@ Reviewed (at least in part) by: mlaier, jmg, pjd, bde, ceri, Alex Lyashkov <umka at sevcity dot net>, Skip Ford <skip dot ford at verizon dot net>, Antoine Brodin <antoine dot brodin at laposte dot net>
2006-11-06 13:42:10 +00:00
#include <sys/priv.h>
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
#include <sys/proc.h>
#include <sys/socket.h>
#include <sys/socketvar.h>
#include <sys/protosw.h>
#include <sys/kernel.h>
Add rwlock.h (and lock.h for that) to keep no-INET kernels compiling after RADIX_NODE_HEAD_{,UN}LOCK() were added. Must have been "learned" by pollution before (most likely: route.h -> radix.h -> rwlock.h)
2008-12-09 20:05:58 +00:00
#include <sys/lock.h>
Start to address a number of races relating to use of ifnet pointers after the corresponding interface has been destroyed: (1) Add an ifnet refcount, ifp->if_refcount. Initialize it to 1 in if_alloc(), and modify if_free_type() to decrement and check the refcount. (2) Add new if_ref() and if_rele() interfaces to allow kernel code walking global interface lists to release IFNET_[RW]LOCK() yet keep the ifnet stable. Currently, if_rele() is a no-op wrapper around if_free(), but this may change in the future. (3) Add new ifnet field, if_alloctype, which caches the type passed to if_alloc(), but unlike if_type, won't be changed by drivers. This allows asynchronous free's of the interface after the driver has released it to still use the right type. Use that instead of the type passed to if_free_type(), but assert that they are the same (might have to rethink this if that doesn't work out). (4) Add a new ifnet_byindex_ref(), which looks up an interface by index and returns a reference rather than a pointer to it. (5) Fix if_alloc() to fully initialize the if_addr_mtx before hooking up the ifnet to global lists. (6) Modify sysctls in if_mib.c to use ifnet_byindex_ref() and release the ifnet when done. When this change is MFC'd, it will need to replace if_ispare fields rather than adding new fields in order to avoid breaking the binary interface. Once this change is MFC'd, if_free_type() should be removed, as its 'type' argument is now optional. This refcount is not appropriate for counting mbuf pkthdr references, and also not for counting entry into the device driver via ifnet function pointers. An rmlock may be appropriate for the latter. Rather, this is about ensuring data structure stability when reaching an ifnet via global ifnet lists and tables followed by copy in or out of userspace. MFC after: 3 weeks Reported by: mdtancsa Reviewed by: brooks
2009-04-21 22:43:32 +00:00
#include <sys/refcount.h>
Change the curvnet variable from a global const struct vnet *, previously always pointing to the default vnet context, to a dynamically changing thread-local one. The currvnet context should be set on entry to networking code via CURVNET_SET() macros, and reverted to previous state via CURVNET_RESTORE(). Recursions on curvnet are permitted, though strongly discuouraged. This change should have no functional impact on nooptions VIMAGE kernel builds, where CURVNET_* macros expand to whitespace. The curthread->td_vnet (aka curvnet) variable's purpose is to be an indicator of the vnet context in which the current network-related operation takes place, in case we cannot deduce the current vnet context from any other source, such as by looking at mbuf's m->m_pkthdr.rcvif->if_vnet, sockets's so->so_vnet etc. Moreover, so far curvnet has turned out to be an invaluable consistency checking aid: it helps to catch cases when sockets, ifnets or any other vnet-aware structures may have leaked from one vnet to another. The exact placement of the CURVNET_SET() / CURVNET_RESTORE() macros was a result of an empirical iterative process, whith an aim to reduce recursions on CURVNET_SET() to a minimum, while still reducing the scope of CURVNET_SET() to networking only operations - the alternative would be calling CURVNET_SET() on each system call entry. In general, curvnet has to be set in three typicall cases: when processing socket-related requests from userspace or from within the kernel; when processing inbound traffic flowing from device drivers to upper layers of the networking stack, and when executing timer-driven networking functions. This change also introduces a DDB subcommand to show the list of all vnet instances. Approved by: julian (mentor)
2009-05-05 10:56:12 +00:00
#include <sys/module.h>
Add rwlock.h (and lock.h for that) to keep no-INET kernels compiling after RADIX_NODE_HEAD_{,UN}LOCK() were added. Must have been "learned" by pollution before (most likely: route.h -> radix.h -> rwlock.h)
2008-12-09 20:05:58 +00:00
#include <sys/rwlock.h>
Don't include <sys/ioctl.h> in the kernel. Stage 2: include <sys/sockio.h> instead of <sys/ioctl.h> in network files.
1997-03-24 11:33:46 +00:00
#include <sys/sockio.h>
Fix BPf to generate a header mbuf for writes. Fix loopback and discard interfaces to understand BPF writes. (These two from Bill Fenner to fix PR 512.) Move ifpromisc() from bpf.c to if.c as suggested by comment in BPF. Send a notice to the log when promiscuous mode is enabled.
1995-09-22 17:57:48 +00:00
#include <sys/syslog.h>
in_proto.c: spell ``Internet'' right and put whitespace after commas. others: start to populate the link-layer branch of the net mib, by moving ARP to its proper place. (ARP is not a protocol family, it's an interface layer between a medium-access layer and a protocol family.) sysctl(8) needs to be taught about the structure of this branch, unless Poul-Henning implements dynamic MIB exploration soon.
1995-12-20 21:53:53 +00:00
#include <sys/sysctl.h>
Fix access to ifru_buffer on freebsd32. Make all kernel accesses to ifru_buffer go via access functions which take the process ABI into account and use an appropriate union to access members in the correct place in struct ifreq. Reviewed by: kib Obtained from: CheriBSD MFC after: 1 week Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D14846
2018-03-27 18:26:50 +00:00
#include <sys/sysent.h>
Add a new network interface flag, IFF_NEEDSGIANT, which will allow device drivers to declare that the ifp->if_start() method implemented by the driver requires Giant in order to operate correctly. Add a 'struct task' to 'struct ifnet' that can be used to execute a deferred ifp->if_start() in the event that if_start needs to be called in a Giant-free environment. To do this, introduce if_start(), a wrapper function for ifp->if_start(). If the interface can run MPSAFE, it directly dispatches into the interface start routine. If it can't run MPSAFE, we're running with debug.mpsafenet != 0, and Giant isn't currently held, the task is queued to execute in a swi holding Giant via if_start_deferred(). Modify if_handoff() to use if_start() instead of direct dispatch. Modify 802.11 to use if_start() instead of direct dispatch. This is intended to provide increased compatibility for non-MPSAFE network device drivers in the presence of Giant-free operation via asynchronous dispatch. However, this commit does not mark any network interfaces as IFF_NEEDSGIANT.
2004-07-27 23:20:45 +00:00
#include <sys/taskqueue.h>
- add dom_if{attach,detach} framework. - transition to use ifp->if_afdata. Obtained from: KAME
2003-10-17 15:46:31 +00:00
#include <sys/domain.h>
o Move per-process jail pointer (p->pr_prison) to inside of the subject credential structure, ucred (cr->cr_prison). o Allow jail inheritence to be a function of credential inheritence. o Abstract prison structure reference counting behind pr_hold() and pr_free(), invoked by the similarly named credential reference management functions, removing this code from per-ABI fork/exit code. o Modify various jail() functions to use struct ucred arguments instead of struct proc arguments. o Introduce jailed() function to determine if a credential is jailed, rather than directly checking pointers all over the place. o Convert PRISON_CHECK() macro to prison_check() function. o Move jail() function prototypes to jail.h. o Emulate the P_JAILED flag in fill_kinfo_proc() and no longer set the flag in the process flags field itself. o Eliminate that "const" qualifier from suser/p_can/etc to reflect mutex use. Notes: o Some further cleanup of the linux/jail code is still required. o It's now possible to consider resolving some of the process vs credential based permission checking confusion in the socket code. o Mutex protection of struct prison is still not present, and is required to protect the reference count plus some fields in the structure. Reviewed by: freebsd-arch Obtained from: TrustedBSD Project
2001-02-21 06:39:57 +00:00
#include <sys/jail.h>
Add infrastructure to allow all frames/packets received on an interface to be assigned to a non-default FIB instance. You may need to recompile world or ports due to the change of struct ifnet. Submitted by: cjsp Submitted by: Alexander V. Chernikov (melifaro ipfw.ru) (original versions) Reviewed by: julian Reviewed by: Alexander V. Chernikov (melifaro ipfw.ru) MFC after: 2 weeks X-MFC: use spare in struct ifnet
2011-07-03 12:22:02 +00:00
#include <sys/priv.h>
Add a new helper function if_printf() modeled on device_printf(). The function takes a struct ifnet pointer followed by the usual printf arguments and prints "<interfacename>: " before the results of printf. Since this is the primary form of printf calls in network device drivers and accounts for most uses of the ifnet menber if_unit, this significantly simplifies many printf()s.
2002-09-24 17:35:08 +00:00
#include <machine/stdarg.h>
This main goals of this project are: 1. separating L2 tables (ARP, NDP) from the L3 routing tables 2. removing as much locking dependencies among these layers as possible to allow for some parallelism in the search operations 3. simplify the logic in the routing code, The most notable end result is the obsolescent of the route cloning (RTF_CLONING) concept, which translated into code reduction in both IPv4 ARP and IPv6 NDP related modules, and size reduction in struct rtentry{}. The change in design obsoletes the semantics of RTF_CLONING, RTF_WASCLONE and RTF_LLINFO routing flags. The userland applications such as "arp" and "ndp" have been modified to reflect those changes. The output from "netstat -r" shows only the routing entries. Quite a few developers have contributed to this project in the past: Glebius Smirnoff, Luigi Rizzo, Alessandro Cerri, and Andre Oppermann. And most recently: - Kip Macy revised the locking code completely, thus completing the last piece of the puzzle, Kip has also been conducting active functional testing - Sam Leffler has helped me improving/refactoring the code, and provided valuable reviews - Julian Elischer setup the perforce tree for me and has helped me maintaining that branch before the svn conversion
2008-12-15 06:10:57 +00:00
#include <vm/uma.h>
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
Introduce a procedural interface to the ifnet structure. The new interface allows the ifnet structure to be defined as an opaque type in NIC drivers. This then allows the ifnet structure to be changed without a need to change or recompile NIC drivers. Put differently, NIC drivers can be written and compiled once and be used with different network stack implementations, provided of course that those network stack implementations have an API and ABI compatible interface. This commit introduces the 'if_t' type to replace 'struct ifnet *' as the type of a network interface. The 'if_t' type is defined as 'void *' to enable the compiler to perform type conversion to 'struct ifnet *' and vice versa where needed and without warnings. The functions that implement the API are the only functions that need to have an explicit cast. The MII code has been converted to use the driver API to avoid unnecessary code churn. Code churn comes from having to work with both converted and unconverted drivers in correlation with having callback functions that take an interface. By converting the MII code first, the callback functions can be defined so that the compiler will perform the typecasts automatically. As soon as all drivers have been converted, the if_t type can be redefined as needed and the API functions can be fix to not need an explicit cast. The immediate benefactors of this change are: 1. Juniper Networks - The network stack implementation in Junos is entirely different from FreeBSD's one and this change allows Juniper to build "stock" NIC drivers that can be used in combination with both the FreeBSD and Junos stacks. 2. FreeBSD - This change opens the door towards changing ifnet and implementing new features and optimizations in the network stack without it requiring a change in the many NIC drivers FreeBSD has. Submitted by: Anuranjan Shukla <anshukla@juniper.net> Reviewed by: glebius@ Obtained from: Juniper Networks, Inc.
2014-06-02 17:54:39 +00:00
#include <net/bpf.h>
#include <net/ethernet.h>
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
#include <net/if.h>
Include if_arp.h for IFP2AC so that the netgraph parts in if.c are happy even if compiled without INET or INET6. MFC after: 2 months
2008-11-06 15:26:09 +00:00
#include <net/if_arp.h>
Major overhaul of pseudo-interface cloning. Highlights include: - Split the code out into if_clone.[ch]. - Locked struct if_clone. [1] - Add a per-cloner match function rather then simply matching names of the form <name><unit> and <name>. - Use the match function to allow creation of <interface>.<tag> vlan interfaces. The old way is preserved unchanged! - Also the match function to allow creation of stf(4) interfaces named stf0, stf, or 6to4. This is the only major user visible change in that "ifconfig stf" creates the interface stf rather then stf0 and does not print "stf0" to stdout. - Allow destroy functions to fail so they can refuse to delete interfaces. Currently, we forbid the deletion of interfaces which were created in the init function, particularly lo0, pflog0, and pfsync0. In the case of lo0 this was a panic implementation so it does not count as a user visiable change. :-) - Since most interfaces do not need the new functionality, an family of wrapper functions, ifc_simple_*(), were created to wrap old style cloner functions. - The IF_CLONE_INITIALIZER macro is replaced with a new incompatible IFC_CLONE_INITIALIZER and ifc_simple consumers use IFC_SIMPLE_DECLARE instead. Submitted by: Maurycy Pawlowski-Wieronski <maurycy at fouk.org> [1] Reviewed by: andre, mlaier Discussed on: net
2004-06-22 20:13:25 +00:00
#include <net/if_clone.h>
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
#include <net/if_dl.h>
Export the functionality of SIOCSIFLLADDR with if_setlladdr() and add some more rigorous sanity checking in the process. Reviewed by: freebsd-net
2000-08-15 00:48:38 +00:00
#include <net/if_types.h>
Add kernel infrastructure for network device cloning. Reviewed by: ru, ume Obtained from: NetBSD MFC after: 1 week
2001-07-02 20:49:25 +00:00
#include <net/if_var.h>
Introduce a procedural interface to the ifnet structure. The new interface allows the ifnet structure to be defined as an opaque type in NIC drivers. This then allows the ifnet structure to be changed without a need to change or recompile NIC drivers. Put differently, NIC drivers can be written and compiled once and be used with different network stack implementations, provided of course that those network stack implementations have an API and ABI compatible interface. This commit introduces the 'if_t' type to replace 'struct ifnet *' as the type of a network interface. The 'if_t' type is defined as 'void *' to enable the compiler to perform type conversion to 'struct ifnet *' and vice versa where needed and without warnings. The functions that implement the API are the only functions that need to have an explicit cast. The MII code has been converted to use the driver API to avoid unnecessary code churn. Code churn comes from having to work with both converted and unconverted drivers in correlation with having callback functions that take an interface. By converting the MII code first, the callback functions can be defined so that the compiler will perform the typecasts automatically. As soon as all drivers have been converted, the if_t type can be redefined as needed and the API functions can be fix to not need an explicit cast. The immediate benefactors of this change are: 1. Juniper Networks - The network stack implementation in Junos is entirely different from FreeBSD's one and this change allows Juniper to build "stock" NIC drivers that can be used in combination with both the FreeBSD and Junos stacks. 2. FreeBSD - This change opens the door towards changing ifnet and implementing new features and optimizations in the network stack without it requiring a change in the many NIC drivers FreeBSD has. Submitted by: Anuranjan Shukla <anshukla@juniper.net> Reviewed by: glebius@ Obtained from: Juniper Networks, Inc.
2014-06-02 17:54:39 +00:00
#include <net/if_media.h>
#include <net/if_vlan_var.h>
Mostly Cosmetics. Some of the procedures in if_sl.c was void, but should be int. I made them int, and let them return 0. Will have to find out what the return-val is used for.
1994-10-08 01:40:23 +00:00
#include <net/radix.h>
Two more fixes to if_detach. These are generic to all interfaces and do not pollute the interface further. o Run if_detach at splnet(). o Creatively swipe the relevant parts of the netatm atm_nif_detach which will delete the relevant references to the interface going away.
1999-12-17 06:46:07 +00:00
#include <net/route.h>
Rather than using hidden includes (with cicular dependencies), directly include only the header files needed. This reduces the unneeded spamming of various headers into lots of files. For now, this leaves us with very few modules including vnet.h and thus needing to depend on opt_route.h. Reviewed by: brooks, gnn, des, zec, imp Sponsored by: The FreeBSD Foundation
2008-12-02 21:37:28 +00:00
#include <net/vnet.h>
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
Prevent kernel panic at ifconfig up after Note PC resume. Submitted by: imp, kuriyama Reviewed by: imp
1999-12-30 18:29:55 +00:00
#if defined(INET) || defined(INET6)
For NFS mounts using rsize,wsize=65536 over TSO enabled network interfaces limited to 32 transmit segments, there are two known issues. The more serious one is that for an I/O of slightly less than 64K, the net device driver prepends an ethernet header, resulting in a TSO segment slightly larger than 64K. Since m_defrag() copies this into 33 mbuf clusters, the transmit fails with EFBIG. A tester indicated observing a similar failure using iSCSI. The second less critical problem is that the network device driver must copy the mbuf chain via m_defrag() (m_collapse() is not sufficient), resulting in measurable overhead. This patch reduces the default size of if_hw_tsomax slightly, so that the first issue is avoided. Fixing the second issue will require a way for the network device driver to inform tcp_output() that it is limited to 32 transmit segments. Reported and tested by: csforgeron@gmail.com, markus.gebert@hostpoint.ch MFC after: 2 weeks
2014-04-17 23:31:50 +00:00
#include <net/ethernet.h>
KAME netinet6 basic part(no IPsec,no V6 Multicast Forwarding, no UDP/TCP for IPv6 yet) With this patch, you can assigne IPv6 addr automatically, and can reply to IPv6 ping. Reviewed by: freebsd-arch, cvs-committers Obtained from: KAME project
1999-11-22 02:45:11 +00:00
#include <netinet/in.h>
Prevent kernel panic at ifconfig up after Note PC resume. Submitted by: imp, kuriyama Reviewed by: imp
1999-12-30 18:29:55 +00:00
#include <netinet/in_var.h>
Allow drivers to specify a maximum TSO length in bytes if they are limited in the amount of data they can handle at once. Drivers can set ifp->if_hw_tsomax before calling ether_ifattach() to change the limit. The lowest allowable size is IP_MAXPACKET / 8 (8192 bytes) as anything less wouldn't be very useful anymore. The upper limit is still at IP_MAXPACKET (65536 bytes). Raising it requires further auditing of the IPv4/v6 code path's as the length field in the IP header would overflow leading to confusion in firewalls and others packet handler on the real size of the packet. The placement into "struct ifnet" is a bit hackish but the best place that was found. When the stack/driver boundary is updated it should be handled in a better way. Submitted by: cperciva (earlier version) Reviewed by: cperciva Tested by: cperciva MFC after: 1 week (using spare struct members to preserve ABI)
2013-06-03 12:55:13 +00:00
#include <netinet/ip.h>
Unbreak LINT by moving all carp hooks to net/if.c / netinet/ip_carp.h, with the appropriate ifdefs. Reviewed by: bz Approved by: ken (mentor)
2010-08-11 20:18:19 +00:00
#include <netinet/ip_carp.h>
Allow drivers to specify a maximum TSO length in bytes if they are limited in the amount of data they can handle at once. Drivers can set ifp->if_hw_tsomax before calling ether_ifattach() to change the limit. The lowest allowable size is IP_MAXPACKET / 8 (8192 bytes) as anything less wouldn't be very useful anymore. The upper limit is still at IP_MAXPACKET (65536 bytes). Raising it requires further auditing of the IPv4/v6 code path's as the length field in the IP header would overflow leading to confusion in firewalls and others packet handler on the real size of the packet. The placement into "struct ifnet" is a bit hackish but the best place that was found. When the stack/driver boundary is updated it should be handled in a better way. Submitted by: cperciva (earlier version) Reviewed by: cperciva Tested by: cperciva MFC after: 1 week (using spare struct members to preserve ABI)
2013-06-03 12:55:13 +00:00
#ifdef INET
#include <netinet/if_ether.h>
Import the netdump client code. This is a component of a system which lets the kernel dump core to a remote host after a panic, rather than to a local storage device. The server component is available in the ports tree. netdump is particularly useful on diskless systems. The netdump(4) man page contains some details describing the protocol. Support for configuring netdump will be added to dumpon(8) in a future commit. To use netdump, the kernel must have been compiled with the NETDUMP option. The initial revision of netdump was written by Darrell Anderson and was integrated into Sandvine's OS, from which this version was derived. Reviewed by: bdrewery, cem (earlier versions), julian, sbruno MFC after: 1 month X-MFC note: use a spare field in struct ifnet Sponsored by: Dell EMC Isilon Differential Revision: https://reviews.freebsd.org/D15253
2018-05-06 00:38:29 +00:00
#include <netinet/netdump/netdump.h>
Allow drivers to specify a maximum TSO length in bytes if they are limited in the amount of data they can handle at once. Drivers can set ifp->if_hw_tsomax before calling ether_ifattach() to change the limit. The lowest allowable size is IP_MAXPACKET / 8 (8192 bytes) as anything less wouldn't be very useful anymore. The upper limit is still at IP_MAXPACKET (65536 bytes). Raising it requires further auditing of the IPv4/v6 code path's as the length field in the IP header would overflow leading to confusion in firewalls and others packet handler on the real size of the packet. The placement into "struct ifnet" is a bit hackish but the best place that was found. When the stack/driver boundary is updated it should be handled in a better way. Submitted by: cperciva (earlier version) Reviewed by: cperciva Tested by: cperciva MFC after: 1 week (using spare struct members to preserve ABI)
2013-06-03 12:55:13 +00:00
#endif /* INET */
Add workaround for fxp issue at interface initialization with IPv6. Some LAN card chip for fxp is known to cause problem at interface initialization with IPv6 enabled. It happens at some delicate timing. And also, just adding some DELAY before IPv6 address autoconfiguration is known to avoid the problem. Complete fix is changing the driver not to use interrupt at multicast filter initialization, but trying such change in this stage will be dangerous. So I add some DELAY() only inside #ifdef INET6 part, as temporal workaround only for 4.0. Approbed by: jkh Noticed by: Mattias Pantzare <pantzer@ludd.luth.se> Obtained from: openbsd-tech mailing list
2000-02-01 15:49:37 +00:00
#ifdef INET6
improve route/nd cache cleanup on interface removal. CAVEAT: haven't really tested it yet, please report
2000-07-16 01:46:42 +00:00
#include <netinet6/in6_var.h>
#include <netinet6/in6_ifattach.h>
Allow drivers to specify a maximum TSO length in bytes if they are limited in the amount of data they can handle at once. Drivers can set ifp->if_hw_tsomax before calling ether_ifattach() to change the limit. The lowest allowable size is IP_MAXPACKET / 8 (8192 bytes) as anything less wouldn't be very useful anymore. The upper limit is still at IP_MAXPACKET (65536 bytes). Raising it requires further auditing of the IPv4/v6 code path's as the length field in the IP header would overflow leading to confusion in firewalls and others packet handler on the real size of the packet. The placement into "struct ifnet" is a bit hackish but the best place that was found. When the stack/driver boundary is updated it should be handled in a better way. Submitted by: cperciva (earlier version) Reviewed by: cperciva Tested by: cperciva MFC after: 1 week (using spare struct members to preserve ABI)
2013-06-03 12:55:13 +00:00
#endif /* INET6 */
#endif /* INET || INET6 */
KAME netinet6 basic part(no IPsec,no V6 Multicast Forwarding, no UDP/TCP for IPv6 yet) With this patch, you can assigne IPv6 addr automatically, and can reply to IPv6 ping. Reviewed by: freebsd-arch, cvs-committers Obtained from: KAME project
1999-11-22 02:45:11 +00:00
Complete break-out of sys/sys/mac.h into sys/security/mac/mac_framework.h begun with a repo-copy of mac.h to mac_framework.h. sys/mac.h now contains the userspace and user<->kernel API and definitions, with all in-kernel interfaces moved to mac_framework.h, which is now included across most of the kernel instead. This change is the first step in a larger cleanup and sweep of MAC Framework interfaces in the kernel, and will not be MFC'd. Obtained from: TrustedBSD Project Sponsored by: SPARTA
2006-10-22 11:52:19 +00:00
#include <security/mac/mac_framework.h>
Document and enforce assumptions about struct (in6_)ifreq. - The two types must be type-punnable for shared members of ifr_ifru. This allows compatibility accessors to be shared. - There must be no padding gap between ifr_name and ifr_ifru. This is assumed in tcpdump's use of SIOCGIFFLAGS output which attempts to be broadly portable. This is true for all current architectures, but very large (256-bit) fat-pointers could violate this invariant. Reviewed by: kib Obtained from: CheriBSD Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D14910
2018-03-30 21:38:53 +00:00
/*
* Consumers of struct ifreq such as tcpdump assume no pad between ifr_name
* and ifr_ifru when it is used in SIOCGIFCONF.
*/
_Static_assert(sizeof(((struct ifreq *)0)->ifr_name) ==
offsetof(struct ifreq, ifr_ifru), "gap between ifr_name and ifr_ifru");
epoch: move epoch variables to read mostly section
2018-05-18 17:58:15 +00:00
__read_mostly epoch_t net_epoch_preempt;
__read_mostly epoch_t net_epoch;
Reshuffle SIOCGIFCONF32 handler from r155224. - move all the chunks into one file, which allows to hide SIOCGIFCONF32 global definition as well. - replace __amd64__ with proper COMPAT_FREEBSD32 around. - handle 32bit capacity before going into the handler itself instead of doing internal 32bit specific changes within it (e.g. as it's done for SIOCGDEFIFACE32_IN6). - use explicitely sized types for ABI compat. Approved by: kib (mentor) MFC after: 2 weeks
2010-10-21 16:20:48 +00:00
#ifdef COMPAT_FREEBSD32
#include <sys/mount.h>
#include <compat/freebsd32/freebsd32.h>
Fix access to ifru_buffer on freebsd32. Make all kernel accesses to ifru_buffer go via access functions which take the process ABI into account and use an appropriate union to access members in the correct place in struct ifreq. Reviewed by: kib Obtained from: CheriBSD MFC after: 1 week Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D14846
2018-03-27 18:26:50 +00:00
struct ifreq_buffer32 {
uint32_t length; /* (size_t) */
uint32_t buffer; /* (void *) */
};
/*
* Interface request structure used for socket
* ioctl's. All interface ioctl's must have parameter
* definitions which begin with ifr_name. The
* remainder may be interface specific.
*/
struct ifreq32 {
char ifr_name[IFNAMSIZ]; /* if name, e.g. "en0" */
union {
struct sockaddr ifru_addr;
struct sockaddr ifru_dstaddr;
struct sockaddr ifru_broadaddr;
struct ifreq_buffer32 ifru_buffer;
short ifru_flags[2];
short ifru_index;
int ifru_jid;
int ifru_metric;
int ifru_mtu;
int ifru_phys;
int ifru_media;
uint32_t ifru_data;
int ifru_cap[2];
u_int ifru_fib;
u_char ifru_vlan_pcp;
} ifr_ifru;
};
CTASSERT(sizeof(struct ifreq) == sizeof(struct ifreq32));
CTASSERT(__offsetof(struct ifreq, ifr_ifru) ==
__offsetof(struct ifreq32, ifr_ifru));
Add 32-bit compat for ioctls that take struct ifgroupreq. Use an accessor to access ifgr_group and ifgr_groups. Use an macro CASE_IOC_IFGROUPREQ(cmd) in place of case statements such as "case SIOCAIFGROUP:". This avoids poluting the switch statements with large numbers of #ifdefs. Reviewed by: kib Obtained from: CheriBSD MFC after: 1 week Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D14960
2018-04-05 22:14:55 +00:00
struct ifgroupreq32 {
char ifgr_name[IFNAMSIZ];
u_int ifgr_len;
union {
char ifgru_group[IFNAMSIZ];
uint32_t ifgru_groups;
} ifgr_ifgru;
};
Translate 32-bit ifmedia requests into native ones. We use transformation rather than accessors as virtually ever driver implements SIOCGIFMEDIA and all would have to be touched. Keep the code readable by always performing copies and (possiably no-op) transforms. Reviewed by: jhb, kib Obtained from: CheriBSD MFC after: 1 week Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D14996
2018-04-25 15:30:42 +00:00
struct ifmediareq32 {
char ifm_name[IFNAMSIZ];
int ifm_current;
int ifm_mask;
int ifm_status;
int ifm_active;
int ifm_count;
uint32_t ifm_ulist; /* (int *) */
};
#define SIOCGIFMEDIA32 _IOC_NEWTYPE(SIOCGIFMEDIA, struct ifmediareq32)
#define SIOCGIFXMEDIA32 _IOC_NEWTYPE(SIOCGIFXMEDIA, struct ifmediareq32)
Add 32-bit compat for ioctls that take struct ifgroupreq. Use an accessor to access ifgr_group and ifgr_groups. Use an macro CASE_IOC_IFGROUPREQ(cmd) in place of case statements such as "case SIOCAIFGROUP:". This avoids poluting the switch statements with large numbers of #ifdefs. Reviewed by: kib Obtained from: CheriBSD MFC after: 1 week Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D14960
2018-04-05 22:14:55 +00:00
#define _CASE_IOC_IFGROUPREQ_32(cmd) \
Rework CASE_IOC_IFGROUPREQ() to require a case before the macro. This is more compatible with formatting tools and looks more normal. Reported by: jhb (on a different review) Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D18442
2019-01-22 17:39:26 +00:00
_IOC_NEWTYPE((cmd), struct ifgroupreq32): case
Translate 32-bit ifmedia requests into native ones. We use transformation rather than accessors as virtually ever driver implements SIOCGIFMEDIA and all would have to be touched. Keep the code readable by always performing copies and (possiably no-op) transforms. Reviewed by: jhb, kib Obtained from: CheriBSD MFC after: 1 week Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D14996
2018-04-25 15:30:42 +00:00
#else /* !COMPAT_FREEBSD32 */
Add 32-bit compat for ioctls that take struct ifgroupreq. Use an accessor to access ifgr_group and ifgr_groups. Use an macro CASE_IOC_IFGROUPREQ(cmd) in place of case statements such as "case SIOCAIFGROUP:". This avoids poluting the switch statements with large numbers of #ifdefs. Reviewed by: kib Obtained from: CheriBSD MFC after: 1 week Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D14960
2018-04-05 22:14:55 +00:00
#define _CASE_IOC_IFGROUPREQ_32(cmd)
Translate 32-bit ifmedia requests into native ones. We use transformation rather than accessors as virtually ever driver implements SIOCGIFMEDIA and all would have to be touched. Keep the code readable by always performing copies and (possiably no-op) transforms. Reviewed by: jhb, kib Obtained from: CheriBSD MFC after: 1 week Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D14996
2018-04-25 15:30:42 +00:00
#endif /* !COMPAT_FREEBSD32 */
Add 32-bit compat for ioctls that take struct ifgroupreq. Use an accessor to access ifgr_group and ifgr_groups. Use an macro CASE_IOC_IFGROUPREQ(cmd) in place of case statements such as "case SIOCAIFGROUP:". This avoids poluting the switch statements with large numbers of #ifdefs. Reviewed by: kib Obtained from: CheriBSD MFC after: 1 week Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D14960
2018-04-05 22:14:55 +00:00
#define CASE_IOC_IFGROUPREQ(cmd) \
_CASE_IOC_IFGROUPREQ_32(cmd) \
Rework CASE_IOC_IFGROUPREQ() to require a case before the macro. This is more compatible with formatting tools and looks more normal. Reported by: jhb (on a different review) Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D18442
2019-01-22 17:39:26 +00:00
(cmd)
Reshuffle SIOCGIFCONF32 handler from r155224. - move all the chunks into one file, which allows to hide SIOCGIFCONF32 global definition as well. - replace __amd64__ with proper COMPAT_FREEBSD32 around. - handle 32bit capacity before going into the handler itself instead of doing internal 32bit specific changes within it (e.g. as it's done for SIOCGDEFIFACE32_IN6). - use explicitely sized types for ABI compat. Approved by: kib (mentor) MFC after: 2 weeks
2010-10-21 16:20:48 +00:00
Fix access to ifru_buffer on freebsd32. Make all kernel accesses to ifru_buffer go via access functions which take the process ABI into account and use an appropriate union to access members in the correct place in struct ifreq. Reviewed by: kib Obtained from: CheriBSD MFC after: 1 week Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D14846
2018-03-27 18:26:50 +00:00
union ifreq_union {
Fix a whitespace bug missed in refactoring prior to r331641. MFC with: r331641
2018-03-27 18:55:39 +00:00
struct ifreq ifr;
Fix access to ifru_buffer on freebsd32. Make all kernel accesses to ifru_buffer go via access functions which take the process ABI into account and use an appropriate union to access members in the correct place in struct ifreq. Reviewed by: kib Obtained from: CheriBSD MFC after: 1 week Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D14846
2018-03-27 18:26:50 +00:00
#ifdef COMPAT_FREEBSD32
Fix a whitespace bug missed in refactoring prior to r331641. MFC with: r331641
2018-03-27 18:55:39 +00:00
struct ifreq32 ifr32;
Fix access to ifru_buffer on freebsd32. Make all kernel accesses to ifru_buffer go via access functions which take the process ABI into account and use an appropriate union to access members in the correct place in struct ifreq. Reviewed by: kib Obtained from: CheriBSD MFC after: 1 week Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D14846
2018-03-27 18:26:50 +00:00
#endif
};
Add 32-bit compat for ioctls that take struct ifgroupreq. Use an accessor to access ifgr_group and ifgr_groups. Use an macro CASE_IOC_IFGROUPREQ(cmd) in place of case statements such as "case SIOCAIFGROUP:". This avoids poluting the switch statements with large numbers of #ifdefs. Reviewed by: kib Obtained from: CheriBSD MFC after: 1 week Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D14960
2018-04-05 22:14:55 +00:00
union ifgroupreq_union {
struct ifgroupreq ifgr;
#ifdef COMPAT_FREEBSD32
struct ifgroupreq32 ifgr32;
#endif
};
Add a sysctl net.link.log_link_state_change, which allows to suppress logging of interface link state changes. Requested by: sam, kan
2005-03-12 12:58:03 +00:00
SYSCTL_NODE(_net, PF_LINK, link, CTLFLAG_RW, 0, "Link layers");
SYSCTL_NODE(_net_link, 0, generic, CTLFLAG_RW, 0, "Generic link-management");
sysctl(9) cleanup checkpoint: amd64 GENERIC builds cleanly. Commit the net* piece.
2011-01-12 19:53:50 +00:00
SYSCTL_INT(_net_link, OID_AUTO, ifqmaxlen, CTLFLAG_RDTUN,
Add new tunable 'net.link.ifqmaxlen' to set default send interface queue length. The default value for this parameter is 50, which is quite low for many of today's uses and the only way to modify this parameter right now is to edit if_var.h file. Also add read-only sysctl with the same name, so that it's possible to retrieve the current value. MFC after: 1 month
2010-05-03 07:32:50 +00:00
&ifqmaxlen, 0, "max send queue size");
Add a sysctl net.link.log_link_state_change, which allows to suppress logging of interface link state changes. Requested by: sam, kan
2005-03-12 12:58:03 +00:00
/* Log link state change events */
static int log_link_state_change = 1;
SYSCTL_INT(_net_link, OID_AUTO, log_link_state_change, CTLFLAG_RW,
&log_link_state_change, 0,
"log interface link state change events");
Allow silencing of 'promiscuous mode enabled/disabled' messages. PR: 166255 Submitted by: eugen.grosbein.net Obtained from: eugen.grosbein.net MFC after: 1 week
2016-05-12 19:42:13 +00:00
/* Log promiscuous mode change events */
static int log_promisc_mode_change = 1;
Change net.link.log_promisc_mode_change to a read-only tunable PR: 166255 Submitted by: eugen.grosbein.net Obtained from: hselasky MFC after: 3 days
2016-05-25 09:00:05 +00:00
SYSCTL_INT(_net_link, OID_AUTO, log_promisc_mode_change, CTLFLAG_RDTUN,
Allow silencing of 'promiscuous mode enabled/disabled' messages. PR: 166255 Submitted by: eugen.grosbein.net Obtained from: eugen.grosbein.net MFC after: 1 week
2016-05-12 19:42:13 +00:00
&log_promisc_mode_change, 1,
"log promiscuous mode change events");
Revised revision 199201 (add interface description capability as inspired by OpenBSD), based on comments from many, including rwatson, jhb, brooks and others. Sponsored by: iXsystems, Inc. MFC after: 1 month
2010-01-27 00:30:07 +00:00
/* Interface description */
static unsigned int ifdescr_maxlen = 1024;
SYSCTL_UINT(_net, OID_AUTO, ifdescr_maxlen, CTLFLAG_RW,
&ifdescr_maxlen, 0,
"administrative maximum length for interface description");
Mark MALLOC_DEFINEs static that have no corresponding MALLOC_DECLAREs. This means that their use is restricted to a single C file.
2011-11-07 06:44:47 +00:00
static MALLOC_DEFINE(M_IFDESCR, "ifdescr", "ifnet descriptions");
Revised revision 199201 (add interface description capability as inspired by OpenBSD), based on comments from many, including rwatson, jhb, brooks and others. Sponsored by: iXsystems, Inc. MFC after: 1 month
2010-01-27 00:30:07 +00:00
/* global sx for non-critical path ifdescr */
static struct sx ifdescr_sx;
SX_SYSINIT(ifdescr_sx, &ifdescr_sx, "ifnet descr");
This change adds reliability for Ethernet trunks built with ng_one2many: - Introduce another ng_ether(4) callback ng_ether_link_state_p, which is called from if_link_state_change(), every time link is changed. - In ng_ether_link_state() send netgraph control message notifying of link state change to a node connected to "lower" hook. Reviewed by: sam MFC after: 2 weeks
2005-01-08 12:42:03 +00:00
void (*ng_ether_link_state_p)(struct ifnet *ifp, int state);
Revert r292275 & r292379 glebius has concerns about these changes so reverting those can be discussed and addressed. Sponsored by: Multiplay
2015-12-17 14:41:30 +00:00
void (*lagg_linkstate_p)(struct ifnet *ifp, int state);
Unbreak LINT by moving all carp hooks to net/if.c / netinet/ip_carp.h, with the appropriate ifdefs. Reviewed by: bz Approved by: ken (mentor)
2010-08-11 20:18:19 +00:00
/* These are external hooks for CARP. */
Allow carp(4) to be loaded as a kernel module. Follow precedent set by bridge(4), lagg(4) etc. and make use of function pointers and pf_proto_register() to hook carp into the network stack. Currently, because of the uncertainty about whether the unload path is free of race condition panics, unloads are disallowed by default. Compiling with CARPMOD_CAN_UNLOAD in CFLAGS removes this anti foot shooting measure. This commit requires IP6PROTOSPACER, introduced in r211115. Reviewed by: bz, simon Approved by: ken (mentor) MFC after: 2 weeks
2010-08-11 00:51:50 +00:00
void (*carp_linkstate_p)(struct ifnet *ifp);
Restore a feature that was present in 5.x and 6.x, and was cleared in 7.x, 8.x and 9.x with pf(4) imports: pfsync(4) should suppress CARP preemption, while it is running its bulk update. However, reimplement the feature in more elegant manner, that is partially inspired by newer OpenBSD: - Rename term "suppression" to "demotion", to match with OpenBSD. - Keep a global demotion factor, that can be raised by several conditions, for now these are: - interface goes down - carp(4) has problems with ip_output() or ip6_output() - pfsync performs bulk update - Unlike in OpenBSD the demotion factor isn't a counter, but is actual value added to advskew. The adjustment values for particular error conditions are also configurable, and their defaults are maximum advskew value, so a single failure bumps demotion to maximum. This is for POLA compatibility, and should satisfy most users. - Demotion factor is a writable sysctl, so user can do foot shooting, if he desires to.
2011-12-20 13:53:31 +00:00
void (*carp_demote_adj_p)(int, char *);
Resolve source address selection in presense of CARP. Add a couple of helper functions: - carp_master() - boolean function which is true if an address is in the MASTER state. - ifa_preferred() - boolean function that compares two addresses, and is aware of CARP. Utilize ifa_preferred() in ifa_ifwithnet(). The previous version of patch also changed source address selection logic in jails using carp_master(), but we failed to negotiate this part with Bjoern. May be we will approach this problem again later. Reported & tested by: Anton Yuzhaninov <citrin citrin.ru> Sponsored by: Nginx, Inc
2013-02-11 10:58:22 +00:00
int (*carp_master_p)(struct ifaddr *);
Unbreak LINT by moving all carp hooks to net/if.c / netinet/ip_carp.h, with the appropriate ifdefs. Reviewed by: bz Approved by: ken (mentor)
2010-08-11 20:18:19 +00:00
#if defined(INET) || defined(INET6)
A major overhaul of the CARP implementation. The ip_carp.c was started from scratch, copying needed functionality from the old implemenation on demand, with a thorough review of all code. The main change is that interface layer has been removed from the CARP. Now redundant addresses are configured exactly on the interfaces, they run on. The CARP configuration itself is, as before, configured and read via SIOCSVH/SIOCGVH ioctls. A new prefix created with SIOCAIFADDR or SIOCAIFADDR_IN6 may now be configured to a particular virtual host id, which makes the prefix redundant. ifconfig(8) semantics has been changed too: now one doesn't need to clone carpXX interface, he/she should directly configure a vhid on a Ethernet interface. To supply vhid data from the kernel to an application the getifaddrs(8) function had been changed to pass ifam_data with each address. [1] The new implementation definitely closes all PRs related to carp(4) being an interface, and may close several others. It also allows to run a single redundant IP per interface. Big thanks to Bjoern Zeeb for his help with inet6 part of patch, for idea on using ifam_data and for several rounds of reviewing! PR: kern/117000, kern/126945, kern/126714, kern/120130, kern/117448 Reviewed by: bz Submitted by: bz [1]
2011-12-16 12:16:56 +00:00
int (*carp_forus_p)(struct ifnet *ifp, u_char *dhost);
Unbreak LINT by moving all carp hooks to net/if.c / netinet/ip_carp.h, with the appropriate ifdefs. Reviewed by: bz Approved by: ken (mentor)
2010-08-11 20:18:19 +00:00
int (*carp_output_p)(struct ifnet *ifp, struct mbuf *m,
Add const qualifier to the dst parameter of the ifnet if_output method.
2013-04-26 12:50:32 +00:00
const struct sockaddr *sa);
A major overhaul of the CARP implementation. The ip_carp.c was started from scratch, copying needed functionality from the old implemenation on demand, with a thorough review of all code. The main change is that interface layer has been removed from the CARP. Now redundant addresses are configured exactly on the interfaces, they run on. The CARP configuration itself is, as before, configured and read via SIOCSVH/SIOCGVH ioctls. A new prefix created with SIOCAIFADDR or SIOCAIFADDR_IN6 may now be configured to a particular virtual host id, which makes the prefix redundant. ifconfig(8) semantics has been changed too: now one doesn't need to clone carpXX interface, he/she should directly configure a vhid on a Ethernet interface. To supply vhid data from the kernel to an application the getifaddrs(8) function had been changed to pass ifam_data with each address. [1] The new implementation definitely closes all PRs related to carp(4) being an interface, and may close several others. It also allows to run a single redundant IP per interface. Big thanks to Bjoern Zeeb for his help with inet6 part of patch, for idea on using ifam_data and for several rounds of reviewing! PR: kern/117000, kern/126945, kern/126714, kern/120130, kern/117448 Reviewed by: bz Submitted by: bz [1]
2011-12-16 12:16:56 +00:00
int (*carp_ioctl_p)(struct ifreq *, u_long, struct thread *);
int (*carp_attach_p)(struct ifaddr *, int);
After the in_control() changes in r257692, an existing address is (intentionally) deleted first and then completely added again (so all the events, announces and hooks are given a chance to run). This cause an issue with CARP where the existing CARP data structure is removed together with the last address for a given VHID, which will cause a subsequent fail when the address is later re-added. This change fixes this issue by adding a new flag to keep the CARP data structure when an address is not being removed. There was an additional issue with IPv6 CARP addresses, where the CARP data structure would never be removed after a change and lead to VHIDs which cannot be destroyed. Reviewed by: glebius Obtained from: pfSense MFC after: 2 weeks Sponsored by: Rubicon Communications, LLC (Netgate)
2017-01-25 19:04:08 +00:00
void (*carp_detach_p)(struct ifaddr *, bool);
Unbreak LINT by moving all carp hooks to net/if.c / netinet/ip_carp.h, with the appropriate ifdefs. Reviewed by: bz Approved by: ken (mentor)
2010-08-11 20:18:19 +00:00
#endif
#ifdef INET
A major overhaul of the CARP implementation. The ip_carp.c was started from scratch, copying needed functionality from the old implemenation on demand, with a thorough review of all code. The main change is that interface layer has been removed from the CARP. Now redundant addresses are configured exactly on the interfaces, they run on. The CARP configuration itself is, as before, configured and read via SIOCSVH/SIOCGVH ioctls. A new prefix created with SIOCAIFADDR or SIOCAIFADDR_IN6 may now be configured to a particular virtual host id, which makes the prefix redundant. ifconfig(8) semantics has been changed too: now one doesn't need to clone carpXX interface, he/she should directly configure a vhid on a Ethernet interface. To supply vhid data from the kernel to an application the getifaddrs(8) function had been changed to pass ifam_data with each address. [1] The new implementation definitely closes all PRs related to carp(4) being an interface, and may close several others. It also allows to run a single redundant IP per interface. Big thanks to Bjoern Zeeb for his help with inet6 part of patch, for idea on using ifam_data and for several rounds of reviewing! PR: kern/117000, kern/126945, kern/126714, kern/120130, kern/117448 Reviewed by: bz Submitted by: bz [1]
2011-12-16 12:16:56 +00:00
int (*carp_iamatch_p)(struct ifaddr *, uint8_t **);
Unbreak LINT by moving all carp hooks to net/if.c / netinet/ip_carp.h, with the appropriate ifdefs. Reviewed by: bz Approved by: ken (mentor)
2010-08-11 20:18:19 +00:00
#endif
#ifdef INET6
struct ifaddr *(*carp_iamatch6_p)(struct ifnet *ifp, struct in6_addr *taddr6);
A major overhaul of the CARP implementation. The ip_carp.c was started from scratch, copying needed functionality from the old implemenation on demand, with a thorough review of all code. The main change is that interface layer has been removed from the CARP. Now redundant addresses are configured exactly on the interfaces, they run on. The CARP configuration itself is, as before, configured and read via SIOCSVH/SIOCGVH ioctls. A new prefix created with SIOCAIFADDR or SIOCAIFADDR_IN6 may now be configured to a particular virtual host id, which makes the prefix redundant. ifconfig(8) semantics has been changed too: now one doesn't need to clone carpXX interface, he/she should directly configure a vhid on a Ethernet interface. To supply vhid data from the kernel to an application the getifaddrs(8) function had been changed to pass ifam_data with each address. [1] The new implementation definitely closes all PRs related to carp(4) being an interface, and may close several others. It also allows to run a single redundant IP per interface. Big thanks to Bjoern Zeeb for his help with inet6 part of patch, for idea on using ifam_data and for several rounds of reviewing! PR: kern/117000, kern/126945, kern/126714, kern/120130, kern/117448 Reviewed by: bz Submitted by: bz [1]
2011-12-16 12:16:56 +00:00
caddr_t (*carp_macmatch6_p)(struct ifnet *ifp, struct mbuf *m,
Unbreak LINT by moving all carp hooks to net/if.c / netinet/ip_carp.h, with the appropriate ifdefs. Reviewed by: bz Approved by: ken (mentor)
2010-08-11 20:18:19 +00:00
const struct in6_addr *taddr);
#endif
This change adds reliability for Ethernet trunks built with ng_one2many: - Introduce another ng_ether(4) callback ng_ether_link_state_p, which is called from if_link_state_change(), every time link is changed. - In ng_ether_link_state() send netgraph control message notifying of link state change to a node connected to "lower" hook. Reviewed by: sam MFC after: 2 weeks
2005-01-08 12:42:03 +00:00
Transform tbr_dequeue into a function pointer in order to build drivers with ALTQ enabled versions of IFQ_* macros by default, as requested by serveral others. This is a follow-up to the quick fix I committed yesterday which turned off the ALTQ checks for non-ALTQ kernels.
2004-06-15 01:45:19 +00:00
struct mbuf *(*tbr_dequeue_ptr)(struct ifaltq *, int) = NULL;
Implement reference counting for ifmultiaddr, in_multi, and in6_multi structures. Detect when ifnet instances are detached from the network stack and perform appropriate cleanup to prevent memory leaks. This has been implemented in such a way as to be backwards ABI compatible. Kernel consumers are changed to use if_delmulti_ifma(); in_delmulti() is unable to detect interface removal by design, as it performs searches on structures which are removed with the interface. With this architectural change, the panics FreeBSD users have experienced with carp and pfsync should be resolved. Obtained from: p4 branch bms_netdev Reviewed by: andre Sponsored by: Garance A Drosehn Idea from: NetBSD MFC after: 1 month
2007-03-20 00:36:10 +00:00
/*
* XXX: Style; these should be sorted alphabetically, and unprototyped
* static functions should be prototyped. Currently they are sorted by
* declaration order.
*/
- add dom_if{attach,detach} framework. - transition to use ifp->if_afdata. Obtained from: KAME
2003-10-17 15:46:31 +00:00
static void if_attachdomain(void *);
static void if_attachdomain1(struct ifnet *);
Cosmetic cleanups and rearrangement for code to come. There should be no functional change in this commit.
2001-09-06 00:44:45 +00:00
static int ifconf(u_long, caddr_t);
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
static void *if_grow(void);
Add if_input_default() method, that will be used for if_input initialization, when no input method specified before if_attach(). This prevents panics when if_input() method called directly e.g. from bpf(4) code. PR: 192426 Reviewed by: glebius MFC after: 1 week
2015-03-12 14:55:33 +00:00
static void if_input_default(struct ifnet *, struct mbuf *);
Implement interface link header precomputation API. Add if_requestencap() interface method which is capable of calculating various link headers for given interface. Right now there is support for INET/INET6/ARP llheader calculation (IFENCAP_LL type request). Other types are planned to support more complex calculation (L2 multipath lagg nexthops, tunnel encap nexthops, etc..). Reshape 'struct route' to be able to pass additional data (with is length) to prepend to mbuf. These two changes permits routing code to pass pre-calculated nexthop data (like L2 header for route w/gateway) down to the stack eliminating the need for other lookups. It also brings us closer to more complex scenarios like transparently handling MPLS nexthops and tunnel interfaces. Last, but not least, it removes layering violation introduced by flowtable code (ro_lle) and simplifies handling of existing if_output consumers. ARP/ND changes: Make arp/ndp stack pre-calculate link header upon installing/updating lle record. Interface link address change are handled by re-calculating headers for all lles based on if_lladdr event. After these changes, arpresolve()/nd6_resolve() returns full pre-calculated header for supported interfaces thus simplifying if_output(). Move these lookups to separate ether_resolve_addr() function which ether returs error or fully-prepared link header. Add <arp|nd6_>resolve_addr() compat versions to return link addresses instead of pre-calculated data. BPF changes: Raw bpf writes occupied _two_ cases: AF_UNSPEC and pseudo_AF_HDRCMPLT. Despite the naming, both of there have ther header "complete". The only difference is that interface source mac has to be filled by OS for AF_UNSPEC (controlled via BIOCGHDRCMPLT). This logic has to stay inside BPF and not pollute if_output() routines. Convert BPF to pass prepend data via new 'struct route' mechanism. Note that it does not change non-optimized if_output(): ro_prepend handling is purely optional. Side note: hackish pseudo_AF_HDRCMPLT is supported for ethernet and FDDI. It is not needed for ethernet anymore. The only remaining FDDI user is dev/pdq mostly untouched since 2007. FDDI support was eliminated from OpenBSD in 2013 (sys/net/if_fddisubr.c rev 1.65). Flowtable changes: Flowtable violates layering by saving (and not correctly managing) rtes/lles. Instead of passing lle pointer, pass pointer to pre-calculated header data from that lle. Differential Revision: https://reviews.freebsd.org/D4102
2015-12-31 05:03:27 +00:00
static int if_requestencap_default(struct ifnet *, struct if_encap_req *);
Make if_(un)route static in if.c as they are called from if_up/if_down only. This is also cleanup to make locking easier. Reviewed by: luigi Approved by: bms(mentor)
2004-04-18 18:59:44 +00:00
static void if_route(struct ifnet *, int flag, int fam);
MFp4: - Introduce a helper function if_setflag() containing the code common to ifpromisc() and if_allmulti() instead of duplicating the code poorly, with different bugs. - Call ifp->if_ioctl() in a consistent way: always use more compatible C syntax and check whether ifp->if_ioctl is not NULL prior to the call. MFC after: 1 month
2005-07-14 13:56:51 +00:00
static int if_setflag(struct ifnet *, int, int, int *, int);
- bump __FreeBSD version to reflect added buf_ring, memory barriers, and ifnet functions - add memory barriers to <machine/atomic.h> - update drivers to only conditionally define their own - add lockless producer / consumer ring buffer - remove ring buffer implementation from cxgb and update its callers - add if_transmit(struct ifnet *ifp, struct mbuf *m) to ifnet to allow drivers to efficiently manage multiple hardware queues (i.e. not serialize all packets through one ifq) - expose if_qflush to allow drivers to flush any driver managed queues This work was supported by Bitgravity Inc. and Chelsio Inc.
2008-11-22 05:55:56 +00:00
static int if_transmit(struct ifnet *ifp, struct mbuf *m);
Make if_(un)route static in if.c as they are called from if_up/if_down only. This is also cleanup to make locking easier. Reviewed by: luigi Approved by: bms(mentor)
2004-04-18 18:59:44 +00:00
static void if_unroute(struct ifnet *, int flag, int fam);
Implement reference counting for ifmultiaddr, in_multi, and in6_multi structures. Detect when ifnet instances are detached from the network stack and perform appropriate cleanup to prevent memory leaks. This has been implemented in such a way as to be backwards ABI compatible. Kernel consumers are changed to use if_delmulti_ifma(); in_delmulti() is unable to detect interface removal by design, as it performs searches on structures which are removed with the interface. With this architectural change, the panics FreeBSD users have experienced with carp and pfsync should be resolved. Obtained from: p4 branch bms_netdev Reviewed by: andre Sponsored by: Garance A Drosehn Idea from: NetBSD MFC after: 1 month
2007-03-20 00:36:10 +00:00
static int if_delmulti_locked(struct ifnet *, struct ifmultiaddr *, int);
Do not call all link state callbacks directly, but schedule a taskqueue(9) task. This fixes LORs and adds possibility to serve such events pseudorecursively, when link state change of interface causes subsequent change on other interfaces. Sponsored by: Rambler Reviewed by: sam, brooks, mux
2005-04-20 09:30:54 +00:00
static void do_link_state_change(void *, int);
Import interface groups from OpenBSD. This allows to group interfaces in order to - for example - apply firewall rules to a whole group of interfaces. This is required for importing pf from OpenBSD 3.9 Obtained from: OpenBSD (with changes) Discussed on: -net (back in April)
2006-06-19 22:20:45 +00:00
static int if_getgroup(struct ifgroupreq *, struct ifnet *);
static int if_getgroupmembers(struct ifgroupreq *);
Follow up for r190895 It's not only the "all" group that is affected, but all groups on the given interface. PR: kern/130977, kern/131310 MFC after: 3 days (%vnet)
2009-04-10 19:16:14 +00:00
static void if_delgroups(struct ifnet *);
Fix group membership of cloned interfaces when one is moved by if_vmove(). In if_vmove(), if_detach_internal() and if_attach_internal() were called in series to detach and reattach the interface. When detaching, if_delgroup() was called and the interface leaves all of the group membership. And then upon attachment, if_addgroup(ifp, IFG_ALL) was called and it joined only "all" group again. This had a problem. Normally, a cloned interface automatically joins a group whose name is ifc_name of the cloner in addition to "all" upon creation. However, if_vmove() removed the membership and did not restore upon attachment. Differential Revision: https://reviews.freebsd.org/D1859
2015-03-02 20:00:03 +00:00
static void if_attach_internal(struct ifnet *, int, struct if_clone *);
If vnets are torn down while ifconfig runs an ioctl to say, destroy an epair(4), we may hit if_detach_internal() without holding a lock and by the time we aquire it the interface might be gone. We should not panic() in this case as it is our fault for not holding the lock all the way. It is not ideal to return silently without error to user space, but other callers will all ignore the return values so do not change the entire KPI for little benefit for now. The ifp will be dealt with one way or another still. Sponsored by: The FreeBSD Foundation MFC after: 2 weeks Reviewed by: gnn Differential Revision: https://reviews.freebsd.org/D4529
2015-12-22 15:03:45 +00:00
static int if_detach_internal(struct ifnet *, int, struct if_clone **);
Rather than having the if_vmove() code intermixed in the vnet_destroy() function in vnet.c move it to if.c where it logically belongs and put it under a VNET_SYSUNINIT() call. To not change the current behaviour make sure it runs first thing during teardown. In the future this will allow us more flexibility on changing the order on when we want to get rid of interfaces. Stop exporting if_vmove() and make it file static. Reviewed by: gnn Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D6438
2016-05-18 20:06:45 +00:00
#ifdef VIMAGE
static void if_vmove(struct ifnet *, struct vnet *);
#endif
- bump __FreeBSD version to reflect added buf_ring, memory barriers, and ifnet functions - add memory barriers to <machine/atomic.h> - update drivers to only conditionally define their own - add lockless producer / consumer ring buffer - remove ring buffer implementation from cxgb and update its callers - add if_transmit(struct ifnet *ifp, struct mbuf *m) to ifnet to allow drivers to efficiently manage multiple hardware queues (i.e. not serialize all packets through one ifq) - expose if_qflush to allow drivers to flush any driver managed queues This work was supported by Bitgravity Inc. and Chelsio Inc.
2008-11-22 05:55:56 +00:00
KAME netinet6 basic part(no IPsec,no V6 Multicast Forwarding, no UDP/TCP for IPv6 yet) With this patch, you can assigne IPv6 addr automatically, and can reply to IPv6 ping. Reviewed by: freebsd-arch, cvs-committers Obtained from: KAME project
1999-11-22 02:45:11 +00:00
#ifdef INET6
/*
* XXX: declare here to avoid to include many inet6 related files..
* should be more generalized?
*/
Remove __P.
2002-03-19 21:54:18 +00:00
extern void nd6_setmtu(struct ifnet *);
KAME netinet6 basic part(no IPsec,no V6 Multicast Forwarding, no UDP/TCP for IPv6 yet) With this patch, you can assigne IPv6 addr automatically, and can reply to IPv6 ping. Reviewed by: freebsd-arch, cvs-committers Obtained from: KAME project
1999-11-22 02:45:11 +00:00
#endif
Overhaul if_enc(4) and make it loadable in run-time. Use hhook(9) framework to achieve ability of loading and unloading if_enc(4) kernel module. INET and INET6 code on initialization registers two helper hooks points in the kernel. if_enc(4) module uses these helper hook points and registers its hooks. IPSEC code uses these hhook points to call helper hooks implemented in if_enc(4).
2015-11-25 07:31:59 +00:00
/* ipsec helper hooks */
VNET_DEFINE(struct hhook_head *, ipsec_hhh_in[HHOOK_IPSEC_COUNT]);
VNET_DEFINE(struct hhook_head *, ipsec_hhh_out[HHOOK_IPSEC_COUNT]);
MFP4: @176978-176982, 176984, 176990-176994, 177441 "Whitspace" churn after the VIMAGE/VNET whirls. Remove the need for some "init" functions within the network stack, like pim6_init(), icmp_init() or significantly shorten others like ip6_init() and nd6_init(), using static initialization again where possible and formerly missed. Move (most) variables back to the place they used to be before the container structs and VIMAGE_GLOABLS (before r185088) and try to reduce the diff to stable/7 and earlier as good as possible, to help out-of-tree consumers to update from 6.x or 7.x to 8 or 9. This also removes some header file pollution for putatively static global variables. Revert VIMAGE specific changes in ipfilter::ip_auth.c, that are no longer needed. Reviewed by: jhb Discussed with: rwatson Sponsored by: The FreeBSD Foundation Sponsored by: CK Software GmbH MFC after: 6 days
2010-04-29 11:52:42 +00:00
VNET_DEFINE(int, if_index);
int ifqmaxlen = IFQ_MAXLEN;
Build on Jeff Roberson's linker-set based dynamic per-CPU allocator (DPCPU), as suggested by Peter Wemm, and implement a new per-virtual network stack memory allocator. Modify vnet to use the allocator instead of monolithic global container structures (vinet, ...). This change solves many binary compatibility problems associated with VIMAGE, and restores ELF symbols for virtualized global variables. Each virtualized global variable exists as a "reference copy", and also once per virtual network stack. Virtualized global variables are tagged at compile-time, placing the in a special linker set, which is loaded into a contiguous region of kernel memory. Virtualized global variables in the base kernel are linked as normal, but those in modules are copied and relocated to a reserved portion of the kernel's vnet region with the help of a the kernel linker. Virtualized global variables exist in per-vnet memory set up when the network stack instance is created, and are initialized statically from the reference copy. Run-time access occurs via an accessor macro, which converts from the current vnet and requested symbol to a per-vnet address. When "options VIMAGE" is not compiled into the kernel, normal global ELF symbols will be used instead and indirection is avoided. This change restores static initialization for network stack global variables, restores support for non-global symbols and types, eliminates the need for many subsystem constructors, eliminates large per-subsystem structures that caused many binary compatibility issues both for monitoring applications (netstat) and kernel modules, removes the per-function INIT_VNET_*() macros throughout the stack, eliminates the need for vnet_symmap ksym(2) munging, and eliminates duplicate definitions of virtualized globals under VIMAGE_GLOBALS. Bump __FreeBSD_version and update UPDATING. Portions submitted by: bz Reviewed by: bz, zec Discussed with: gnn, jamie, jeff, jhb, julian, sam Suggested by: peter Approved by: re (kensmith)
2009-07-14 22:48:30 +00:00
VNET_DEFINE(struct ifnethead, ifnet); /* depend on static init XXX */
VNET_DEFINE(struct ifgrouphead, ifg_head);
MFP4: @176978-176982, 176984, 176990-176994, 177441 "Whitspace" churn after the VIMAGE/VNET whirls. Remove the need for some "init" functions within the network stack, like pim6_init(), icmp_init() or significantly shorten others like ip6_init() and nd6_init(), using static initialization again where possible and formerly missed. Move (most) variables back to the place they used to be before the container structs and VIMAGE_GLOABLS (before r185088) and try to reduce the diff to stable/7 and earlier as good as possible, to help out-of-tree consumers to update from 6.x or 7.x to 8 or 9. This also removes some header file pollution for putatively static global variables. Revert VIMAGE specific changes in ipfilter::ip_auth.c, that are no longer needed. Reviewed by: jhb Discussed with: rwatson Sponsored by: The FreeBSD Foundation Sponsored by: CK Software GmbH MFC after: 6 days
2010-04-29 11:52:42 +00:00
Use the new VNET_DEFINE_STATIC macro when we are defining static VNET variables. Reviewed by: bz Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D16147
2018-07-24 16:35:52 +00:00
VNET_DEFINE_STATIC(int, if_indexlim) = 8;
Build on Jeff Roberson's linker-set based dynamic per-CPU allocator (DPCPU), as suggested by Peter Wemm, and implement a new per-virtual network stack memory allocator. Modify vnet to use the allocator instead of monolithic global container structures (vinet, ...). This change solves many binary compatibility problems associated with VIMAGE, and restores ELF symbols for virtualized global variables. Each virtualized global variable exists as a "reference copy", and also once per virtual network stack. Virtualized global variables are tagged at compile-time, placing the in a special linker set, which is loaded into a contiguous region of kernel memory. Virtualized global variables in the base kernel are linked as normal, but those in modules are copied and relocated to a reserved portion of the kernel's vnet region with the help of a the kernel linker. Virtualized global variables exist in per-vnet memory set up when the network stack instance is created, and are initialized statically from the reference copy. Run-time access occurs via an accessor macro, which converts from the current vnet and requested symbol to a per-vnet address. When "options VIMAGE" is not compiled into the kernel, normal global ELF symbols will be used instead and indirection is avoided. This change restores static initialization for network stack global variables, restores support for non-global symbols and types, eliminates the need for many subsystem constructors, eliminates large per-subsystem structures that caused many binary compatibility issues both for monitoring applications (netstat) and kernel modules, removes the per-function INIT_VNET_*() macros throughout the stack, eliminates the need for vnet_symmap ksym(2) munging, and eliminates duplicate definitions of virtualized globals under VIMAGE_GLOBALS. Bump __FreeBSD_version and update UPDATING. Portions submitted by: bz Reviewed by: bz, zec Discussed with: gnn, jamie, jeff, jhb, julian, sam Suggested by: peter Approved by: re (kensmith)
2009-07-14 22:48:30 +00:00
Rework global locks for interface list and index management, correcting several critical bugs, including race conditions and lock order issues: Replace the single rwlock, ifnet_lock, with two locks, an rwlock and an sxlock. Either can be held to stablize the lists and indexes, but both are required to write. This allows the list to be held stable in both network interrupt contexts and sleepable user threads across sleeping memory allocations or device driver interactions. As before, writes to the interface list must occur from sleepable contexts. Reviewed by: bz, julian MFC after: 3 days
2009-08-23 20:40:19 +00:00
/* Table of ifnet by index. */
Remove useless structure ifindex_entry. Sponsored by: Nginx, Inc. Sponsored by: Netflix
2014-11-07 09:15:39 +00:00
VNET_DEFINE(struct ifnet **, ifindex_table);
Build on Jeff Roberson's linker-set based dynamic per-CPU allocator (DPCPU), as suggested by Peter Wemm, and implement a new per-virtual network stack memory allocator. Modify vnet to use the allocator instead of monolithic global container structures (vinet, ...). This change solves many binary compatibility problems associated with VIMAGE, and restores ELF symbols for virtualized global variables. Each virtualized global variable exists as a "reference copy", and also once per virtual network stack. Virtualized global variables are tagged at compile-time, placing the in a special linker set, which is loaded into a contiguous region of kernel memory. Virtualized global variables in the base kernel are linked as normal, but those in modules are copied and relocated to a reserved portion of the kernel's vnet region with the help of a the kernel linker. Virtualized global variables exist in per-vnet memory set up when the network stack instance is created, and are initialized statically from the reference copy. Run-time access occurs via an accessor macro, which converts from the current vnet and requested symbol to a per-vnet address. When "options VIMAGE" is not compiled into the kernel, normal global ELF symbols will be used instead and indirection is avoided. This change restores static initialization for network stack global variables, restores support for non-global symbols and types, eliminates the need for many subsystem constructors, eliminates large per-subsystem structures that caused many binary compatibility issues both for monitoring applications (netstat) and kernel modules, removes the per-function INIT_VNET_*() macros throughout the stack, eliminates the need for vnet_symmap ksym(2) munging, and eliminates duplicate definitions of virtualized globals under VIMAGE_GLOBALS. Bump __FreeBSD_version and update UPDATING. Portions submitted by: bz Reviewed by: bz, zec Discussed with: gnn, jamie, jeff, jhb, julian, sam Suggested by: peter Approved by: re (kensmith)
2009-07-14 22:48:30 +00:00
Remove unused VNET_SET() and related macros; only VNET_GET() is ever actually used. Rename VNET_GET() to VNET() to shorten variable references. Discussed with: bz, julian Reviewed by: bz Approved by: re (kensmith, kib)
2009-07-16 21:13:04 +00:00
#define V_if_indexlim VNET(if_indexlim)
#define V_ifindex_table VNET(ifindex_table)
Change the initialization methodology for global variables scheduled for virtualization. Instead of initializing the affected global variables at instatiation, assign initial values to them in initializer functions. As a rule, initialization at instatiation for such variables should never be introduced again from now on. Furthermore, enclose all instantiations of such global variables in #ifdef VIMAGE_GLOBALS blocks. Essentialy, this change should have zero functional impact. In the next phase of merging network stack virtualization infrastructure from p4/vimage branch, the new initialization methology will allow us to switch between using global variables and their counterparts residing in virtualization containers with minimum code churn, and in the long run allow us to intialize multiple instances of such container structures. Discussed at: devsummit Strassburg Reviewed by: bz, julian Approved by: julian (mentor) Obtained from: //depot/projects/vimage-commit2/... X-MFC after: never Sponsored by: NLnet Foundation, The FreeBSD Foundation
2008-11-19 09:39:34 +00:00
Rework global locks for interface list and index management, correcting several critical bugs, including race conditions and lock order issues: Replace the single rwlock, ifnet_lock, with two locks, an rwlock and an sxlock. Either can be held to stablize the lists and indexes, but both are required to write. This allows the list to be held stable in both network interrupt contexts and sleepable user threads across sleeping memory allocations or device driver interactions. As before, writes to the interface list must occur from sleepable contexts. Reviewed by: bz, julian MFC after: 3 days
2009-08-23 20:40:19 +00:00
/*
* The global network interface list (V_ifnet) and related state (such as
* if_index, if_indexlim, and ifindex_table) are protected by an sxlock and
* an rwlock. Either may be acquired shared to stablize the list, but both
* must be acquired writable to modify the list. This model allows us to
* both stablize the interface list during interrupt thread processing, but
* also to stablize it over long-running ioctls, without introducing priority
* inversions and deadlocks.
*/
Back out r249318, r249320 and r249327 due to a heisenbug most likely related to a race condition in the ipi_hash_lock with the exact cause currently unknown but under investigation.
2013-05-06 16:42:18 +00:00
struct rwlock ifnet_rwlock;
Use standard mtx(9), rwlock(9), sx(9) system initialization macros instead of doing initialization manually. Sponsored by: Nginx, Inc. Sponsored by: Netflix
2014-11-09 11:11:08 +00:00
RW_SYSINIT_FLAGS(ifnet_rw, &ifnet_rwlock, "ifnet_rw", RW_RECURSE);
Rework global locks for interface list and index management, correcting several critical bugs, including race conditions and lock order issues: Replace the single rwlock, ifnet_lock, with two locks, an rwlock and an sxlock. Either can be held to stablize the lists and indexes, but both are required to write. This allows the list to be held stable in both network interrupt contexts and sleepable user threads across sleeping memory allocations or device driver interactions. As before, writes to the interface list must occur from sleepable contexts. Reviewed by: bz, julian MFC after: 3 days
2009-08-23 20:40:19 +00:00
struct sx ifnet_sxlock;
Use standard mtx(9), rwlock(9), sx(9) system initialization macros instead of doing initialization manually. Sponsored by: Nginx, Inc. Sponsored by: Netflix
2014-11-09 11:11:08 +00:00
SX_SYSINIT_FLAGS(ifnet_sx, &ifnet_sxlock, "ifnet_sx", SX_RECURSE);
Rework global locks for interface list and index management, correcting several critical bugs, including race conditions and lock order issues: Replace the single rwlock, ifnet_lock, with two locks, an rwlock and an sxlock. Either can be held to stablize the lists and indexes, but both are required to write. This allows the list to be held stable in both network interrupt contexts and sleepable user threads across sleeping memory allocations or device driver interactions. As before, writes to the interface list must occur from sleepable contexts. Reviewed by: bz, julian MFC after: 3 days
2009-08-23 20:40:19 +00:00
Add IFNET_HOLD reserved pointer value for the ifindex ifnet array, which allows an index to be reserved for an ifnet without making the ifnet available for management operations. Use this in if_alloc() while the ifnet lock is released between initial index allocation and completion of ifnet initialization. Add ifindex_free() to centralize the implementation of releasing an ifindex value. Use in if_free() and if_vmove(), as well as when releasing a held index in if_alloc(). Reviewed by: bz MFC after: 3 days
2009-08-26 11:13:10 +00:00
/*
* The allocation of network interfaces is a rather non-atomic affair; we
* need to select an index before we are ready to expose the interface for
* use, so will use this pointer value to indicate reservation.
*/
#define IFNET_HOLD (void *)(uintptr_t)(-1)
Stop embedding struct ifnet at the top of driver softcs. Instead the struct ifnet or the layer 2 common structure it was embedded in have been replaced with a struct ifnet pointer to be filled by a call to the new function, if_alloc(). The layer 2 common structure is also allocated via if_alloc() based on the interface type. It is hung off the new struct ifnet member, if_l2com. This change removes the size of these structures from the kernel ABI and will allow us to better manage them as interfaces come and go. Other changes of note: - Struct arpcom is no longer referenced in normal interface code. Instead the Ethernet address is accessed via the IFP2ENADDR() macro. To enforce this ac_enaddr has been renamed to _ac_enaddr. - The second argument to ether_ifattach is now always the mac address from driver private storage rather than sometimes being ac_enaddr. Reviewed by: sobomax, sam
2005-06-10 16:49:24 +00:00
static if_com_alloc_t *if_com_alloc[256];
static if_com_free_t *if_com_free[256];
Cosmetic cleanups and rearrangement for code to come. There should be no functional change in this commit.
2001-09-06 00:44:45 +00:00
Mark MALLOC_DEFINEs static that have no corresponding MALLOC_DECLAREs. This means that their use is restricted to a single C file.
2011-11-07 06:44:47 +00:00
static MALLOC_DEFINE(M_IFNET, "ifnet", "interface internals");
Cosmetic cleanups and rearrangement for code to come. There should be no functional change in this commit.
2001-09-06 00:44:45 +00:00
MALLOC_DEFINE(M_IFADDR, "ifaddr", "interface address");
MALLOC_DEFINE(M_IFMADDR, "ether_multi", "link-level multicast address");
Add kernel infrastructure for network device cloning. Reviewed by: ru, ume Obtained from: NetBSD MFC after: 1 week
2001-07-02 20:49:25 +00:00
Start to address a number of races relating to use of ifnet pointers after the corresponding interface has been destroyed: (1) Add an ifnet refcount, ifp->if_refcount. Initialize it to 1 in if_alloc(), and modify if_free_type() to decrement and check the refcount. (2) Add new if_ref() and if_rele() interfaces to allow kernel code walking global interface lists to release IFNET_[RW]LOCK() yet keep the ifnet stable. Currently, if_rele() is a no-op wrapper around if_free(), but this may change in the future. (3) Add new ifnet field, if_alloctype, which caches the type passed to if_alloc(), but unlike if_type, won't be changed by drivers. This allows asynchronous free's of the interface after the driver has released it to still use the right type. Use that instead of the type passed to if_free_type(), but assert that they are the same (might have to rethink this if that doesn't work out). (4) Add a new ifnet_byindex_ref(), which looks up an interface by index and returns a reference rather than a pointer to it. (5) Fix if_alloc() to fully initialize the if_addr_mtx before hooking up the ifnet to global lists. (6) Modify sysctls in if_mib.c to use ifnet_byindex_ref() and release the ifnet when done. When this change is MFC'd, it will need to replace if_ispare fields rather than adding new fields in order to avoid breaking the binary interface. Once this change is MFC'd, if_free_type() should be removed, as its 'type' argument is now optional. This refcount is not appropriate for counting mbuf pkthdr references, and also not for counting entry into the device driver via ifnet function pointers. An rmlock may be appropriate for the latter. Rather, this is about ensuring data structure stability when reaching an ifnet via global ifnet lists and tables followed by copy in or out of userspace. MFC after: 3 weeks Reported by: mdtancsa Reviewed by: brooks
2009-04-21 22:43:32 +00:00
struct ifnet *
ifnet_byindex(u_short idx)
{
Introduce locking around use of ifindex_table, whose use was previously unsynchronized. While races were extremely rare, we've now had a couple of reports of panics in environments involving large numbers of IPSEC tunnels being added very quickly on an active system. - Add accessor functions ifnet_byindex(), ifaddr_byindex(), ifdev_byindex() to replace existing accessor macros. These functions now acquire the ifnet lock before derefencing the table. - Add IFNET_WLOCK_ASSERT(). - Add static accessor functions ifnet_setbyindex(), ifdev_setbyindex(), which set values in the table either asserting of acquiring the ifnet lock. - Use accessor functions throughout if.c to modify and read ifindex_table. - Rework ifnet attach/detach to lock around ifindex_table modification. Note that these changes simply close races around use of ifindex_table, and make no attempt to solve the probem of disappearing ifnets. Further refinement of this work, including with respect to ifindex_table resizing, is still required. In a future change, the ifnet lock should be converted from a mutex to an rwlock in order to reduce contention. Reviewed and tested by: brooks
2008-06-26 23:05:28 +00:00
struct ifnet *ifp;
The two functions ifnet_byindex() and ifnet_byindex_locked() are exactly the same after the network stack was epochified. Merge the two into one function and cleanup all uses of ifnet_byindex_locked(). While at it: - Add branch prediction macros. - Make sure the ifnet pointer is only deferred once, also when code optimisation is disabled. Sponsored by: Mellanox Technologies
2019-10-15 12:08:09 +00:00
if (__predict_false(idx > V_if_index))
return (NULL);
ifp = *(struct ifnet * const volatile *)(V_ifindex_table + idx);
return (__predict_false(ifp == IFNET_HOLD) ? NULL : ifp);
add ifnet_byindex_locked to allow for use of IFNET_RLOCK
2008-12-18 04:50:44 +00:00
}
struct ifnet *
Start to address a number of races relating to use of ifnet pointers after the corresponding interface has been destroyed: (1) Add an ifnet refcount, ifp->if_refcount. Initialize it to 1 in if_alloc(), and modify if_free_type() to decrement and check the refcount. (2) Add new if_ref() and if_rele() interfaces to allow kernel code walking global interface lists to release IFNET_[RW]LOCK() yet keep the ifnet stable. Currently, if_rele() is a no-op wrapper around if_free(), but this may change in the future. (3) Add new ifnet field, if_alloctype, which caches the type passed to if_alloc(), but unlike if_type, won't be changed by drivers. This allows asynchronous free's of the interface after the driver has released it to still use the right type. Use that instead of the type passed to if_free_type(), but assert that they are the same (might have to rethink this if that doesn't work out). (4) Add a new ifnet_byindex_ref(), which looks up an interface by index and returns a reference rather than a pointer to it. (5) Fix if_alloc() to fully initialize the if_addr_mtx before hooking up the ifnet to global lists. (6) Modify sysctls in if_mib.c to use ifnet_byindex_ref() and release the ifnet when done. When this change is MFC'd, it will need to replace if_ispare fields rather than adding new fields in order to avoid breaking the binary interface. Once this change is MFC'd, if_free_type() should be removed, as its 'type' argument is now optional. This refcount is not appropriate for counting mbuf pkthdr references, and also not for counting entry into the device driver via ifnet function pointers. An rmlock may be appropriate for the latter. Rather, this is about ensuring data structure stability when reaching an ifnet via global ifnet lists and tables followed by copy in or out of userspace. MFC after: 3 weeks Reported by: mdtancsa Reviewed by: brooks
2009-04-21 22:43:32 +00:00
ifnet_byindex_ref(u_short idx)
add ifnet_byindex_locked to allow for use of IFNET_RLOCK
2008-12-18 04:50:44 +00:00
{
struct ifnet *ifp;
Widen NET_EPOCH coverage. When epoch(9) was introduced to network stack, it was basically dropped in place of existing locking, which was mutexes and rwlocks. For the sake of performance mutex covered areas were as small as possible, so became epoch covered areas. However, epoch doesn't introduce any contention, it just delays memory reclaim. So, there is no point to minimise epoch covered areas in sense of performance. Meanwhile entering/exiting epoch also has non-zero CPU usage, so doing this less often is a win. Not the least is also code maintainability. In the new paradigm we can assume that at any stage of processing a packet, we are inside network epoch. This makes coding both input and output path way easier. On output path we already enter epoch quite early - in the ip_output(), in the ip6_output(). This patch does the same for the input path. All ISR processing, network related callouts, other ways of packet injection to the network stack shall be performed in net_epoch. Any leaf function that walks network configuration now asserts epoch. Tricky part is configuration code paths - ioctls, sysctls. They also call into leaf functions, so some need to be changed. This patch would introduce more epoch recursions (see EPOCH_TRACE) than we had before. They will be cleaned up separately, as several of them aren't trivial. Note, that unlike a lock recursion the epoch recursion is safe and just wastes a bit of resources. Reviewed by: gallatin, hselasky, cy, adrian, kristof Differential Revision: https://reviews.freebsd.org/D19111
2019-10-07 22:40:05 +00:00
NET_EPOCH_ASSERT();
The two functions ifnet_byindex() and ifnet_byindex_locked() are exactly the same after the network stack was epochified. Merge the two into one function and cleanup all uses of ifnet_byindex_locked(). While at it: - Add branch prediction macros. - Make sure the ifnet pointer is only deferred once, also when code optimisation is disabled. Sponsored by: Mellanox Technologies
2019-10-15 12:08:09 +00:00
ifp = ifnet_byindex(idx);
Widen NET_EPOCH coverage. When epoch(9) was introduced to network stack, it was basically dropped in place of existing locking, which was mutexes and rwlocks. For the sake of performance mutex covered areas were as small as possible, so became epoch covered areas. However, epoch doesn't introduce any contention, it just delays memory reclaim. So, there is no point to minimise epoch covered areas in sense of performance. Meanwhile entering/exiting epoch also has non-zero CPU usage, so doing this less often is a win. Not the least is also code maintainability. In the new paradigm we can assume that at any stage of processing a packet, we are inside network epoch. This makes coding both input and output path way easier. On output path we already enter epoch quite early - in the ip_output(), in the ip6_output(). This patch does the same for the input path. All ISR processing, network related callouts, other ways of packet injection to the network stack shall be performed in net_epoch. Any leaf function that walks network configuration now asserts epoch. Tricky part is configuration code paths - ioctls, sysctls. They also call into leaf functions, so some need to be changed. This patch would introduce more epoch recursions (see EPOCH_TRACE) than we had before. They will be cleaned up separately, as several of them aren't trivial. Note, that unlike a lock recursion the epoch recursion is safe and just wastes a bit of resources. Reviewed by: gallatin, hselasky, cy, adrian, kristof Differential Revision: https://reviews.freebsd.org/D19111
2019-10-07 22:40:05 +00:00
if (ifp == NULL || (ifp->if_flags & IFF_DYING))
Start to address a number of races relating to use of ifnet pointers after the corresponding interface has been destroyed: (1) Add an ifnet refcount, ifp->if_refcount. Initialize it to 1 in if_alloc(), and modify if_free_type() to decrement and check the refcount. (2) Add new if_ref() and if_rele() interfaces to allow kernel code walking global interface lists to release IFNET_[RW]LOCK() yet keep the ifnet stable. Currently, if_rele() is a no-op wrapper around if_free(), but this may change in the future. (3) Add new ifnet field, if_alloctype, which caches the type passed to if_alloc(), but unlike if_type, won't be changed by drivers. This allows asynchronous free's of the interface after the driver has released it to still use the right type. Use that instead of the type passed to if_free_type(), but assert that they are the same (might have to rethink this if that doesn't work out). (4) Add a new ifnet_byindex_ref(), which looks up an interface by index and returns a reference rather than a pointer to it. (5) Fix if_alloc() to fully initialize the if_addr_mtx before hooking up the ifnet to global lists. (6) Modify sysctls in if_mib.c to use ifnet_byindex_ref() and release the ifnet when done. When this change is MFC'd, it will need to replace if_ispare fields rather than adding new fields in order to avoid breaking the binary interface. Once this change is MFC'd, if_free_type() should be removed, as its 'type' argument is now optional. This refcount is not appropriate for counting mbuf pkthdr references, and also not for counting entry into the device driver via ifnet function pointers. An rmlock may be appropriate for the latter. Rather, this is about ensuring data structure stability when reaching an ifnet via global ifnet lists and tables followed by copy in or out of userspace. MFC after: 3 weeks Reported by: mdtancsa Reviewed by: brooks
2009-04-21 22:43:32 +00:00
return (NULL);
if_ref(ifp);
Introduce locking around use of ifindex_table, whose use was previously unsynchronized. While races were extremely rare, we've now had a couple of reports of panics in environments involving large numbers of IPSEC tunnels being added very quickly on an active system. - Add accessor functions ifnet_byindex(), ifaddr_byindex(), ifdev_byindex() to replace existing accessor macros. These functions now acquire the ifnet lock before derefencing the table. - Add IFNET_WLOCK_ASSERT(). - Add static accessor functions ifnet_setbyindex(), ifdev_setbyindex(), which set values in the table either asserting of acquiring the ifnet lock. - Use accessor functions throughout if.c to modify and read ifindex_table. - Rework ifnet attach/detach to lock around ifindex_table modification. Note that these changes simply close races around use of ifindex_table, and make no attempt to solve the probem of disappearing ifnets. Further refinement of this work, including with respect to ifindex_table resizing, is still required. In a future change, the ifnet lock should be converted from a mutex to an rwlock in order to reduce contention. Reviewed and tested by: brooks
2008-06-26 23:05:28 +00:00
return (ifp);
}
Break out allocation of new ifindex values from if_alloc() and if_vmove(), and centralize in a single function ifindex_alloc(). Assert the IFNET_WLOCK, and add missing IFNET_WLOCK in if_alloc(). This does not close all known races in this code. Reviewed by: bz MFC after: 3 days
2009-08-25 20:21:16 +00:00
/*
* Allocate an ifindex array entry; return 0 on success or an error on
* failure.
*/
ifindex_alloc_locked() never fails and doesn't have no-lock version, so change the prototype. Sponsored by: Netflix Sponsored by: Nginx, Inc.
2014-11-08 07:23:01 +00:00
static u_short
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
ifindex_alloc(void **old)
Break out allocation of new ifindex values from if_alloc() and if_vmove(), and centralize in a single function ifindex_alloc(). Assert the IFNET_WLOCK, and add missing IFNET_WLOCK in if_alloc(). This does not close all known races in this code. Reviewed by: bz MFC after: 3 days
2009-08-25 20:21:16 +00:00
{
u_short idx;
IFNET_WLOCK_ASSERT();
/*
Add IFNET_HOLD reserved pointer value for the ifindex ifnet array, which allows an index to be reserved for an ifnet without making the ifnet available for management operations. Use this in if_alloc() while the ifnet lock is released between initial index allocation and completion of ifnet initialization. Add ifindex_free() to centralize the implementation of releasing an ifindex value. Use in if_free() and if_vmove(), as well as when releasing a held index in if_alloc(). Reviewed by: bz MFC after: 3 days
2009-08-26 11:13:10 +00:00
* Try to find an empty slot below V_if_index. If we fail, take the
Break out allocation of new ifindex values from if_alloc() and if_vmove(), and centralize in a single function ifindex_alloc(). Assert the IFNET_WLOCK, and add missing IFNET_WLOCK in if_alloc(). This does not close all known races in this code. Reviewed by: bz MFC after: 3 days
2009-08-25 20:21:16 +00:00
* next slot.
*/
for (idx = 1; idx <= V_if_index; idx++) {
Remove useless structure ifindex_entry. Sponsored by: Nginx, Inc. Sponsored by: Netflix
2014-11-07 09:15:39 +00:00
if (V_ifindex_table[idx] == NULL)
Break out allocation of new ifindex values from if_alloc() and if_vmove(), and centralize in a single function ifindex_alloc(). Assert the IFNET_WLOCK, and add missing IFNET_WLOCK in if_alloc(). This does not close all known races in this code. Reviewed by: bz MFC after: 3 days
2009-08-25 20:21:16 +00:00
break;
}
/* Catch if_index overflow. */
Fix a LOR by dropping the global ifnet locks while allocating a new ifnet table in if_grow(). The order of the SYSINIT's for ifnet state were swapped so that the various locks were initialized before being used. Reviewed by: pluknet, bz MFC after: 2 weeks
2011-01-24 22:21:58 +00:00
if (idx >= V_if_indexlim) {
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
*old = if_grow();
return (USHRT_MAX);
Fix a LOR by dropping the global ifnet locks while allocating a new ifnet table in if_grow(). The order of the SYSINIT's for ifnet state were swapped so that the various locks were initialized before being used. Reviewed by: pluknet, bz MFC after: 2 weeks
2011-01-24 22:21:58 +00:00
}
Break out allocation of new ifindex values from if_alloc() and if_vmove(), and centralize in a single function ifindex_alloc(). Assert the IFNET_WLOCK, and add missing IFNET_WLOCK in if_alloc(). This does not close all known races in this code. Reviewed by: bz MFC after: 3 days
2009-08-25 20:21:16 +00:00
if (idx > V_if_index)
V_if_index = idx;
ifindex_alloc_locked() never fails and doesn't have no-lock version, so change the prototype. Sponsored by: Netflix Sponsored by: Nginx, Inc.
2014-11-08 07:23:01 +00:00
return (idx);
Break out allocation of new ifindex values from if_alloc() and if_vmove(), and centralize in a single function ifindex_alloc(). Assert the IFNET_WLOCK, and add missing IFNET_WLOCK in if_alloc(). This does not close all known races in this code. Reviewed by: bz MFC after: 3 days
2009-08-25 20:21:16 +00:00
}
Add IFNET_HOLD reserved pointer value for the ifindex ifnet array, which allows an index to be reserved for an ifnet without making the ifnet available for management operations. Use this in if_alloc() while the ifnet lock is released between initial index allocation and completion of ifnet initialization. Add ifindex_free() to centralize the implementation of releasing an ifindex value. Use in if_free() and if_vmove(), as well as when releasing a held index in if_alloc(). Reviewed by: bz MFC after: 3 days
2009-08-26 11:13:10 +00:00
static void
ifindex_free_locked(u_short idx)
{
IFNET_WLOCK_ASSERT();
Remove useless structure ifindex_entry. Sponsored by: Nginx, Inc. Sponsored by: Netflix
2014-11-07 09:15:39 +00:00
V_ifindex_table[idx] = NULL;
Add IFNET_HOLD reserved pointer value for the ifindex ifnet array, which allows an index to be reserved for an ifnet without making the ifnet available for management operations. Use this in if_alloc() while the ifnet lock is released between initial index allocation and completion of ifnet initialization. Add ifindex_free() to centralize the implementation of releasing an ifindex value. Use in if_free() and if_vmove(), as well as when releasing a held index in if_alloc(). Reviewed by: bz MFC after: 3 days
2009-08-26 11:13:10 +00:00
while (V_if_index > 0 &&
Remove useless structure ifindex_entry. Sponsored by: Nginx, Inc. Sponsored by: Netflix
2014-11-07 09:15:39 +00:00
V_ifindex_table[V_if_index] == NULL)
Add IFNET_HOLD reserved pointer value for the ifindex ifnet array, which allows an index to be reserved for an ifnet without making the ifnet available for management operations. Use this in if_alloc() while the ifnet lock is released between initial index allocation and completion of ifnet initialization. Add ifindex_free() to centralize the implementation of releasing an ifindex value. Use in if_free() and if_vmove(), as well as when releasing a held index in if_alloc(). Reviewed by: bz MFC after: 3 days
2009-08-26 11:13:10 +00:00
V_if_index--;
}
static void
ifindex_free(u_short idx)
{
IFNET_WLOCK();
ifindex_free_locked(idx);
IFNET_WUNLOCK();
}
Rework global locks for interface list and index management, correcting several critical bugs, including race conditions and lock order issues: Replace the single rwlock, ifnet_lock, with two locks, an rwlock and an sxlock. Either can be held to stablize the lists and indexes, but both are required to write. This allows the list to be held stable in both network interrupt contexts and sleepable user threads across sleeping memory allocations or device driver interactions. As before, writes to the interface list must occur from sleepable contexts. Reviewed by: bz, julian MFC after: 3 days
2009-08-23 20:40:19 +00:00
static void
ifnet_setbyindex(u_short idx, struct ifnet *ifp)
{
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
V_ifindex_table[idx] = ifp;
Rework global locks for interface list and index management, correcting several critical bugs, including race conditions and lock order issues: Replace the single rwlock, ifnet_lock, with two locks, an rwlock and an sxlock. Either can be held to stablize the lists and indexes, but both are required to write. This allows the list to be held stable in both network interrupt contexts and sleepable user threads across sleeping memory allocations or device driver interactions. As before, writes to the interface list must occur from sleepable contexts. Reviewed by: bz, julian MFC after: 3 days
2009-08-23 20:40:19 +00:00
}
Introduce locking around use of ifindex_table, whose use was previously unsynchronized. While races were extremely rare, we've now had a couple of reports of panics in environments involving large numbers of IPSEC tunnels being added very quickly on an active system. - Add accessor functions ifnet_byindex(), ifaddr_byindex(), ifdev_byindex() to replace existing accessor macros. These functions now acquire the ifnet lock before derefencing the table. - Add IFNET_WLOCK_ASSERT(). - Add static accessor functions ifnet_setbyindex(), ifdev_setbyindex(), which set values in the table either asserting of acquiring the ifnet lock. - Use accessor functions throughout if.c to modify and read ifindex_table. - Rework ifnet attach/detach to lock around ifindex_table modification. Note that these changes simply close races around use of ifindex_table, and make no attempt to solve the probem of disappearing ifnets. Further refinement of this work, including with respect to ifindex_table resizing, is still required. In a future change, the ifnet lock should be converted from a mutex to an rwlock in order to reduce contention. Reviewed and tested by: brooks
2008-06-26 23:05:28 +00:00
struct ifaddr *
ifaddr_byindex(u_short idx)
{
Prevent null-pointer dereferencing. MFC after: 3 days
2015-07-20 08:21:51 +00:00
struct ifnet *ifp;
struct ifaddr *ifa = NULL;
Introduce locking around use of ifindex_table, whose use was previously unsynchronized. While races were extremely rare, we've now had a couple of reports of panics in environments involving large numbers of IPSEC tunnels being added very quickly on an active system. - Add accessor functions ifnet_byindex(), ifaddr_byindex(), ifdev_byindex() to replace existing accessor macros. These functions now acquire the ifnet lock before derefencing the table. - Add IFNET_WLOCK_ASSERT(). - Add static accessor functions ifnet_setbyindex(), ifdev_setbyindex(), which set values in the table either asserting of acquiring the ifnet lock. - Use accessor functions throughout if.c to modify and read ifindex_table. - Rework ifnet attach/detach to lock around ifindex_table modification. Note that these changes simply close races around use of ifindex_table, and make no attempt to solve the probem of disappearing ifnets. Further refinement of this work, including with respect to ifindex_table resizing, is still required. In a future change, the ifnet lock should be converted from a mutex to an rwlock in order to reduce contention. Reviewed and tested by: brooks
2008-06-26 23:05:28 +00:00
Widen NET_EPOCH coverage. When epoch(9) was introduced to network stack, it was basically dropped in place of existing locking, which was mutexes and rwlocks. For the sake of performance mutex covered areas were as small as possible, so became epoch covered areas. However, epoch doesn't introduce any contention, it just delays memory reclaim. So, there is no point to minimise epoch covered areas in sense of performance. Meanwhile entering/exiting epoch also has non-zero CPU usage, so doing this less often is a win. Not the least is also code maintainability. In the new paradigm we can assume that at any stage of processing a packet, we are inside network epoch. This makes coding both input and output path way easier. On output path we already enter epoch quite early - in the ip_output(), in the ip6_output(). This patch does the same for the input path. All ISR processing, network related callouts, other ways of packet injection to the network stack shall be performed in net_epoch. Any leaf function that walks network configuration now asserts epoch. Tricky part is configuration code paths - ioctls, sysctls. They also call into leaf functions, so some need to be changed. This patch would introduce more epoch recursions (see EPOCH_TRACE) than we had before. They will be cleaned up separately, as several of them aren't trivial. Note, that unlike a lock recursion the epoch recursion is safe and just wastes a bit of resources. Reviewed by: gallatin, hselasky, cy, adrian, kristof Differential Revision: https://reviews.freebsd.org/D19111
2019-10-07 22:40:05 +00:00
NET_EPOCH_ASSERT();
The two functions ifnet_byindex() and ifnet_byindex_locked() are exactly the same after the network stack was epochified. Merge the two into one function and cleanup all uses of ifnet_byindex_locked(). While at it: - Add branch prediction macros. - Make sure the ifnet pointer is only deferred once, also when code optimisation is disabled. Sponsored by: Mellanox Technologies
2019-10-15 12:08:09 +00:00
ifp = ifnet_byindex(idx);
Prevent null-pointer dereferencing. MFC after: 3 days
2015-07-20 08:21:51 +00:00
if (ifp != NULL && (ifa = ifp->if_addr) != NULL)
Modify most routines returning 'struct ifaddr *' to return references rather than pointers, requiring callers to properly dispose of those references. The following routines now return references: ifaddr_byindex ifa_ifwithaddr ifa_ifwithbroadaddr ifa_ifwithdstaddr ifa_ifwithnet ifaof_ifpforaddr ifa_ifwithroute ifa_ifwithroute_fib rt_getifa rt_getifa_fib IFP_TO_IA ip_rtaddr in6_ifawithifp in6ifa_ifpforlinklocal in6ifa_ifpwithaddr in6_ifadd carp_iamatch6 ip6_getdstifaddr Remove unused macro which didn't have required referencing: IFP_TO_IA6 This closes many small races in which changes to interface or address lists while an ifaddr was in use could lead to use of freed memory (etc). In a few cases, add missing if_addr_list locking required to safely acquire references. Because of a lack of deep copying support, we accept a race in which an in6_ifaddr pointed to by mbuf tags and extracted with ip6_getdstifaddr() doesn't hold a reference while in transmit. Once we have mbuf tag deep copy support, this can be fixed. Reviewed by: bz Obtained from: Apple, Inc. (portions) MFC after: 6 weeks (portions)
2009-06-23 20:19:09 +00:00
ifa_ref(ifa);
Introduce locking around use of ifindex_table, whose use was previously unsynchronized. While races were extremely rare, we've now had a couple of reports of panics in environments involving large numbers of IPSEC tunnels being added very quickly on an active system. - Add accessor functions ifnet_byindex(), ifaddr_byindex(), ifdev_byindex() to replace existing accessor macros. These functions now acquire the ifnet lock before derefencing the table. - Add IFNET_WLOCK_ASSERT(). - Add static accessor functions ifnet_setbyindex(), ifdev_setbyindex(), which set values in the table either asserting of acquiring the ifnet lock. - Use accessor functions throughout if.c to modify and read ifindex_table. - Rework ifnet attach/detach to lock around ifindex_table modification. Note that these changes simply close races around use of ifindex_table, and make no attempt to solve the probem of disappearing ifnets. Further refinement of this work, including with respect to ifindex_table resizing, is still required. In a future change, the ifnet lock should be converted from a mutex to an rwlock in order to reduce contention. Reviewed and tested by: brooks
2008-06-26 23:05:28 +00:00
return (ifa);
}
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
/*
* Network interface utility routines.
*
* Routines with ifa_ifwith* names take sockaddr *'s as
* parameters.
*/
Update the comments on if_alloc(), if_free(), if_free_type(), and if_attach. Remove a comment about pre-3.0 network drivers from if_attach(). Be a bit more consistant about whitespace near comments.
2007-05-16 19:59:01 +00:00
Introduce and use a sysinit-based initialization scheme for virtual network stacks, VNET_SYSINIT: - Add VNET_SYSINIT and VNET_SYSUNINIT macros to declare events that will occur each time a network stack is instantiated and destroyed. In the !VIMAGE case, these are simply mapped into regular SYSINIT/SYSUNINIT. For the VIMAGE case, we instead use SYSINIT's to track their order and properties on registration, using them for each vnet when created/ destroyed, or immediately on module load for already-started vnets. - Remove vnet_modinfo mechanism that existed to serve this purpose previously, as well as its dependency scheme: we now just use the SYSINIT ordering scheme. - Implement VNET_DOMAIN_SET() to allow protocol domains to declare that they want init functions to be called for each virtual network stack rather than just once at boot, compiling down to DOMAIN_SET() in the non-VIMAGE case. - Walk all virtualized kernel subsystems and make use of these instead of modinfo or DOMAIN_SET() for init/uninit events. In some cases, convert modular components from using modevent to using sysinit (where appropriate). In some cases, do minor rejuggling of SYSINIT ordering to make room for or better manage events. Portions submitted by: jhb (VNET_SYSINIT), bz (cleanup) Discussed with: jhb, bz, julian, zec Reviewed by: bz Approved by: re (VIMAGE blanket)
2009-07-23 20:46:49 +00:00
static void
vnet_if_init(const void *unused __unused)
{
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
void *old;
Introduce and use a sysinit-based initialization scheme for virtual network stacks, VNET_SYSINIT: - Add VNET_SYSINIT and VNET_SYSUNINIT macros to declare events that will occur each time a network stack is instantiated and destroyed. In the !VIMAGE case, these are simply mapped into regular SYSINIT/SYSUNINIT. For the VIMAGE case, we instead use SYSINIT's to track their order and properties on registration, using them for each vnet when created/ destroyed, or immediately on module load for already-started vnets. - Remove vnet_modinfo mechanism that existed to serve this purpose previously, as well as its dependency scheme: we now just use the SYSINIT ordering scheme. - Implement VNET_DOMAIN_SET() to allow protocol domains to declare that they want init functions to be called for each virtual network stack rather than just once at boot, compiling down to DOMAIN_SET() in the non-VIMAGE case. - Walk all virtualized kernel subsystems and make use of these instead of modinfo or DOMAIN_SET() for init/uninit events. In some cases, convert modular components from using modevent to using sysinit (where appropriate). In some cases, do minor rejuggling of SYSINIT ordering to make room for or better manage events. Portions submitted by: jhb (VNET_SYSINIT), bz (cleanup) Discussed with: jhb, bz, julian, zec Reviewed by: bz Approved by: re (VIMAGE blanket)
2009-07-23 20:46:49 +00:00
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
CK_STAILQ_INIT(&V_ifnet);
CK_STAILQ_INIT(&V_ifg_head);
Fix a LOR by dropping the global ifnet locks while allocating a new ifnet table in if_grow(). The order of the SYSINIT's for ifnet state were swapped so that the various locks were initialized before being used. Reviewed by: pluknet, bz MFC after: 2 weeks
2011-01-24 22:21:58 +00:00
IFNET_WLOCK();
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
old = if_grow(); /* create initial table */
Fix a LOR by dropping the global ifnet locks while allocating a new ifnet table in if_grow(). The order of the SYSINIT's for ifnet state were swapped so that the various locks were initialized before being used. Reviewed by: pluknet, bz MFC after: 2 weeks
2011-01-24 22:21:58 +00:00
IFNET_WUNLOCK();
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
epoch_wait_preempt(net_epoch_preempt);
free(old, M_IFNET);
Introduce and use a sysinit-based initialization scheme for virtual network stacks, VNET_SYSINIT: - Add VNET_SYSINIT and VNET_SYSUNINIT macros to declare events that will occur each time a network stack is instantiated and destroyed. In the !VIMAGE case, these are simply mapped into regular SYSINIT/SYSUNINIT. For the VIMAGE case, we instead use SYSINIT's to track their order and properties on registration, using them for each vnet when created/ destroyed, or immediately on module load for already-started vnets. - Remove vnet_modinfo mechanism that existed to serve this purpose previously, as well as its dependency scheme: we now just use the SYSINIT ordering scheme. - Implement VNET_DOMAIN_SET() to allow protocol domains to declare that they want init functions to be called for each virtual network stack rather than just once at boot, compiling down to DOMAIN_SET() in the non-VIMAGE case. - Walk all virtualized kernel subsystems and make use of these instead of modinfo or DOMAIN_SET() for init/uninit events. In some cases, convert modular components from using modevent to using sysinit (where appropriate). In some cases, do minor rejuggling of SYSINIT ordering to make room for or better manage events. Portions submitted by: jhb (VNET_SYSINIT), bz (cleanup) Discussed with: jhb, bz, julian, zec Reviewed by: bz Approved by: re (VIMAGE blanket)
2009-07-23 20:46:49 +00:00
vnet_if_clone_init();
}
Fix a LOR by dropping the global ifnet locks while allocating a new ifnet table in if_grow(). The order of the SYSINIT's for ifnet state were swapped so that the various locks were initialized before being used. Reviewed by: pluknet, bz MFC after: 2 weeks
2011-01-24 22:21:58 +00:00
VNET_SYSINIT(vnet_if_init, SI_SUB_INIT_IF, SI_ORDER_SECOND, vnet_if_init,
Introduce and use a sysinit-based initialization scheme for virtual network stacks, VNET_SYSINIT: - Add VNET_SYSINIT and VNET_SYSUNINIT macros to declare events that will occur each time a network stack is instantiated and destroyed. In the !VIMAGE case, these are simply mapped into regular SYSINIT/SYSUNINIT. For the VIMAGE case, we instead use SYSINIT's to track their order and properties on registration, using them for each vnet when created/ destroyed, or immediately on module load for already-started vnets. - Remove vnet_modinfo mechanism that existed to serve this purpose previously, as well as its dependency scheme: we now just use the SYSINIT ordering scheme. - Implement VNET_DOMAIN_SET() to allow protocol domains to declare that they want init functions to be called for each virtual network stack rather than just once at boot, compiling down to DOMAIN_SET() in the non-VIMAGE case. - Walk all virtualized kernel subsystems and make use of these instead of modinfo or DOMAIN_SET() for init/uninit events. In some cases, convert modular components from using modevent to using sysinit (where appropriate). In some cases, do minor rejuggling of SYSINIT ordering to make room for or better manage events. Portions submitted by: jhb (VNET_SYSINIT), bz (cleanup) Discussed with: jhb, bz, julian, zec Reviewed by: bz Approved by: re (VIMAGE blanket)
2009-07-23 20:46:49 +00:00
NULL);
Introduce an infrastructure for dismantling vnet instances. Vnet modules and protocol domains may now register destructor functions to clean up and release per-module state. The destructor mechanisms can be triggered by invoking "vimage -d", or a future equivalent command which will be provided via the new jail framework. While this patch introduces numerous placeholder destructor functions, many of those are currently incomplete, thus leaking memory or (even worse) failing to stop all running timers. Many of such issues are already known and will be incrementaly fixed over the next weeks in smaller incremental commits. Apart from introducing new fields in structs ifnet, domain, protosw and vnet_net, which requires the kernel and modules to be rebuilt, this change should have no impact on nooptions VIMAGE builds, since vnet destructors can only be called in VIMAGE kernels. Moreover, destructor functions should be in general compiled in only in options VIMAGE builds, except for kernel modules which can be safely kldunloaded at run time. Bump __FreeBSD_version to 800097. Reviewed by: bz, julian Approved by: rwatson, kib (re), julian (mentor)
2009-06-08 17:15:40 +00:00
#ifdef VIMAGE
Introduce and use a sysinit-based initialization scheme for virtual network stacks, VNET_SYSINIT: - Add VNET_SYSINIT and VNET_SYSUNINIT macros to declare events that will occur each time a network stack is instantiated and destroyed. In the !VIMAGE case, these are simply mapped into regular SYSINIT/SYSUNINIT. For the VIMAGE case, we instead use SYSINIT's to track their order and properties on registration, using them for each vnet when created/ destroyed, or immediately on module load for already-started vnets. - Remove vnet_modinfo mechanism that existed to serve this purpose previously, as well as its dependency scheme: we now just use the SYSINIT ordering scheme. - Implement VNET_DOMAIN_SET() to allow protocol domains to declare that they want init functions to be called for each virtual network stack rather than just once at boot, compiling down to DOMAIN_SET() in the non-VIMAGE case. - Walk all virtualized kernel subsystems and make use of these instead of modinfo or DOMAIN_SET() for init/uninit events. In some cases, convert modular components from using modevent to using sysinit (where appropriate). In some cases, do minor rejuggling of SYSINIT ordering to make room for or better manage events. Portions submitted by: jhb (VNET_SYSINIT), bz (cleanup) Discussed with: jhb, bz, julian, zec Reviewed by: bz Approved by: re (VIMAGE blanket)
2009-07-23 20:46:49 +00:00
static void
vnet_if_uninit(const void *unused __unused)
Introduce an infrastructure for dismantling vnet instances. Vnet modules and protocol domains may now register destructor functions to clean up and release per-module state. The destructor mechanisms can be triggered by invoking "vimage -d", or a future equivalent command which will be provided via the new jail framework. While this patch introduces numerous placeholder destructor functions, many of those are currently incomplete, thus leaking memory or (even worse) failing to stop all running timers. Many of such issues are already known and will be incrementaly fixed over the next weeks in smaller incremental commits. Apart from introducing new fields in structs ifnet, domain, protosw and vnet_net, which requires the kernel and modules to be rebuilt, this change should have no impact on nooptions VIMAGE builds, since vnet destructors can only be called in VIMAGE kernels. Moreover, destructor functions should be in general compiled in only in options VIMAGE builds, except for kernel modules which can be safely kldunloaded at run time. Bump __FreeBSD_version to 800097. Reviewed by: bz, julian Approved by: rwatson, kib (re), julian (mentor)
2009-06-08 17:15:40 +00:00
{
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
VNET_ASSERT(CK_STAILQ_EMPTY(&V_ifnet), ("%s:%d tailq &V_ifnet=%p "
Mfp4 CH=177255: Make VNET_ASSERT() available with either VNET_DEBUG or INVARIANTS. Change the syntax to match KASSERT() to allow more flexible panic messages rather than having a printf with hardcoded arguments before panic. Adjust the few assertions we have to the new format (and enhance the output). Sponsored by: The FreeBSD Foundation Sponsored by: CK Software GmbH Reviewed by: jhb MFC after: 2 weeks
2011-02-11 13:27:00 +00:00
"not empty", __func__, __LINE__, &V_ifnet));
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
VNET_ASSERT(CK_STAILQ_EMPTY(&V_ifg_head), ("%s:%d tailq &V_ifg_head=%p "
Mfp4 CH=177255: Make VNET_ASSERT() available with either VNET_DEBUG or INVARIANTS. Change the syntax to match KASSERT() to allow more flexible panic messages rather than having a printf with hardcoded arguments before panic. Adjust the few assertions we have to the new format (and enhance the output). Sponsored by: The FreeBSD Foundation Sponsored by: CK Software GmbH Reviewed by: jhb MFC after: 2 weeks
2011-02-11 13:27:00 +00:00
"not empty", __func__, __LINE__, &V_ifg_head));
Introduce an infrastructure for dismantling vnet instances. Vnet modules and protocol domains may now register destructor functions to clean up and release per-module state. The destructor mechanisms can be triggered by invoking "vimage -d", or a future equivalent command which will be provided via the new jail framework. While this patch introduces numerous placeholder destructor functions, many of those are currently incomplete, thus leaking memory or (even worse) failing to stop all running timers. Many of such issues are already known and will be incrementaly fixed over the next weeks in smaller incremental commits. Apart from introducing new fields in structs ifnet, domain, protosw and vnet_net, which requires the kernel and modules to be rebuilt, this change should have no impact on nooptions VIMAGE builds, since vnet destructors can only be called in VIMAGE kernels. Moreover, destructor functions should be in general compiled in only in options VIMAGE builds, except for kernel modules which can be safely kldunloaded at run time. Bump __FreeBSD_version to 800097. Reviewed by: bz, julian Approved by: rwatson, kib (re), julian (mentor)
2009-06-08 17:15:40 +00:00
free((caddr_t)V_ifindex_table, M_IFNET);
}
Introduce and use a sysinit-based initialization scheme for virtual network stacks, VNET_SYSINIT: - Add VNET_SYSINIT and VNET_SYSUNINIT macros to declare events that will occur each time a network stack is instantiated and destroyed. In the !VIMAGE case, these are simply mapped into regular SYSINIT/SYSUNINIT. For the VIMAGE case, we instead use SYSINIT's to track their order and properties on registration, using them for each vnet when created/ destroyed, or immediately on module load for already-started vnets. - Remove vnet_modinfo mechanism that existed to serve this purpose previously, as well as its dependency scheme: we now just use the SYSINIT ordering scheme. - Implement VNET_DOMAIN_SET() to allow protocol domains to declare that they want init functions to be called for each virtual network stack rather than just once at boot, compiling down to DOMAIN_SET() in the non-VIMAGE case. - Walk all virtualized kernel subsystems and make use of these instead of modinfo or DOMAIN_SET() for init/uninit events. In some cases, convert modular components from using modevent to using sysinit (where appropriate). In some cases, do minor rejuggling of SYSINIT ordering to make room for or better manage events. Portions submitted by: jhb (VNET_SYSINIT), bz (cleanup) Discussed with: jhb, bz, julian, zec Reviewed by: bz Approved by: re (VIMAGE blanket)
2009-07-23 20:46:49 +00:00
VNET_SYSUNINIT(vnet_if_uninit, SI_SUB_INIT_IF, SI_ORDER_FIRST,
vnet_if_uninit, NULL);
Rather than having the if_vmove() code intermixed in the vnet_destroy() function in vnet.c move it to if.c where it logically belongs and put it under a VNET_SYSUNINIT() call. To not change the current behaviour make sure it runs first thing during teardown. In the future this will allow us more flexibility on changing the order on when we want to get rid of interfaces. Stop exporting if_vmove() and make it file static. Reviewed by: gnn Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D6438
2016-05-18 20:06:45 +00:00
static void
vnet_if_return(const void *unused __unused)
{
struct ifnet *ifp, *nifp;
/* Return all inherited interfaces to their parent vnets. */
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
CK_STAILQ_FOREACH_SAFE(ifp, &V_ifnet, if_link, nifp) {
Rather than having the if_vmove() code intermixed in the vnet_destroy() function in vnet.c move it to if.c where it logically belongs and put it under a VNET_SYSUNINIT() call. To not change the current behaviour make sure it runs first thing during teardown. In the future this will allow us more flexibility on changing the order on when we want to get rid of interfaces. Stop exporting if_vmove() and make it file static. Reviewed by: gnn Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D6438
2016-05-18 20:06:45 +00:00
if (ifp->if_home_vnet != ifp->if_vnet)
if_vmove(ifp, ifp->if_home_vnet);
}
}
VNET_SYSUNINIT(vnet_if_return, SI_SUB_VNET_DONE, SI_ORDER_ANY,
vnet_if_return, NULL);
Introduce an infrastructure for dismantling vnet instances. Vnet modules and protocol domains may now register destructor functions to clean up and release per-module state. The destructor mechanisms can be triggered by invoking "vimage -d", or a future equivalent command which will be provided via the new jail framework. While this patch introduces numerous placeholder destructor functions, many of those are currently incomplete, thus leaking memory or (even worse) failing to stop all running timers. Many of such issues are already known and will be incrementaly fixed over the next weeks in smaller incremental commits. Apart from introducing new fields in structs ifnet, domain, protosw and vnet_net, which requires the kernel and modules to be rebuilt, this change should have no impact on nooptions VIMAGE builds, since vnet destructors can only be called in VIMAGE kernels. Moreover, destructor functions should be in general compiled in only in options VIMAGE builds, except for kernel modules which can be safely kldunloaded at run time. Bump __FreeBSD_version to 800097. Reviewed by: bz, julian Approved by: rwatson, kib (re), julian (mentor)
2009-06-08 17:15:40 +00:00
#endif
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
static void *
Wrap array accesses in macros, which also happen to be lvalues: ifnet_addrs[i - 1] -> ifaddr_byindex(i) ifindex2ifnet[i] -> ifnet_byindex(i) This is intended to ease the conversion to SMPng.
2001-09-06 02:40:43 +00:00
if_grow(void)
{
Fix a LOR by dropping the global ifnet locks while allocating a new ifnet table in if_grow(). The order of the SYSINIT's for ifnet state were swapped so that the various locks were initialized before being used. Reviewed by: pluknet, bz MFC after: 2 weeks
2011-01-24 22:21:58 +00:00
int oldlim;
Wrap array accesses in macros, which also happen to be lvalues: ifnet_addrs[i - 1] -> ifaddr_byindex(i) ifindex2ifnet[i] -> ifnet_byindex(i) This is intended to ease the conversion to SMPng.
2001-09-06 02:40:43 +00:00
u_int n;
Remove useless structure ifindex_entry. Sponsored by: Nginx, Inc. Sponsored by: Netflix
2014-11-07 09:15:39 +00:00
struct ifnet **e;
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
void *old;
Wrap array accesses in macros, which also happen to be lvalues: ifnet_addrs[i - 1] -> ifaddr_byindex(i) ifindex2ifnet[i] -> ifnet_byindex(i) This is intended to ease the conversion to SMPng.
2001-09-06 02:40:43 +00:00
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
old = NULL;
Fix a LOR by dropping the global ifnet locks while allocating a new ifnet table in if_grow(). The order of the SYSINIT's for ifnet state were swapped so that the various locks were initialized before being used. Reviewed by: pluknet, bz MFC after: 2 weeks
2011-01-24 22:21:58 +00:00
IFNET_WLOCK_ASSERT();
oldlim = V_if_indexlim;
IFNET_WUNLOCK();
n = (oldlim << 1) * sizeof(*e);
Stop embedding struct ifnet at the top of driver softcs. Instead the struct ifnet or the layer 2 common structure it was embedded in have been replaced with a struct ifnet pointer to be filled by a call to the new function, if_alloc(). The layer 2 common structure is also allocated via if_alloc() based on the interface type. It is hung off the new struct ifnet member, if_l2com. This change removes the size of these structures from the kernel ABI and will allow us to better manage them as interfaces come and go. Other changes of note: - Struct arpcom is no longer referenced in normal interface code. Instead the Ethernet address is accessed via the IFP2ENADDR() macro. To enforce this ac_enaddr has been renamed to _ac_enaddr. - The second argument to ether_ifattach is now always the mac address from driver private storage rather than sometimes being ac_enaddr. Reviewed by: sobomax, sam
2005-06-10 16:49:24 +00:00
e = malloc(n, M_IFNET, M_WAITOK | M_ZERO);
Fix a LOR by dropping the global ifnet locks while allocating a new ifnet table in if_grow(). The order of the SYSINIT's for ifnet state were swapped so that the various locks were initialized before being used. Reviewed by: pluknet, bz MFC after: 2 weeks
2011-01-24 22:21:58 +00:00
IFNET_WLOCK();
if (V_if_indexlim != oldlim) {
free(e, M_IFNET);
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
return (NULL);
Fix a LOR by dropping the global ifnet locks while allocating a new ifnet table in if_grow(). The order of the SYSINIT's for ifnet state were swapped so that the various locks were initialized before being used. Reviewed by: pluknet, bz MFC after: 2 weeks
2011-01-24 22:21:58 +00:00
}
Commit step 1 of the vimage project, (network stack) virtualization work done by Marko Zec (zec@). This is the first in a series of commits over the course of the next few weeks. Mark all uses of global variables to be virtualized with a V_ prefix. Use macros to map them back to their global names for now, so this is a NOP change only. We hope to have caught at least 85-90% of what is needed so we do not invalidate a lot of outstanding patches again. Obtained from: //depot/projects/vimage-commit2/... Reviewed by: brooks, des, ed, mav, julian, jamie, kris, rwatson, zec, ... (various people I forgot, different versions) md5 (with a bit of help) Sponsored by: NLnet Foundation, The FreeBSD Foundation X-MFC after: never V_Commit_Message_Reviewed_By: more people than the patch
2008-08-17 23:27:27 +00:00
if (V_ifindex_table != NULL) {
memcpy((caddr_t)e, (caddr_t)V_ifindex_table, n/2);
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
old = V_ifindex_table;
Wrap array accesses in macros, which also happen to be lvalues: ifnet_addrs[i - 1] -> ifaddr_byindex(i) ifindex2ifnet[i] -> ifnet_byindex(i) This is intended to ease the conversion to SMPng.
2001-09-06 02:40:43 +00:00
}
Fix a LOR by dropping the global ifnet locks while allocating a new ifnet table in if_grow(). The order of the SYSINIT's for ifnet state were swapped so that the various locks were initialized before being used. Reviewed by: pluknet, bz MFC after: 2 weeks
2011-01-24 22:21:58 +00:00
V_if_indexlim <<= 1;
Commit step 1 of the vimage project, (network stack) virtualization work done by Marko Zec (zec@). This is the first in a series of commits over the course of the next few weeks. Mark all uses of global variables to be virtualized with a V_ prefix. Use macros to map them back to their global names for now, so this is a NOP change only. We hope to have caught at least 85-90% of what is needed so we do not invalidate a lot of outstanding patches again. Obtained from: //depot/projects/vimage-commit2/... Reviewed by: brooks, des, ed, mav, julian, jamie, kris, rwatson, zec, ... (various people I forgot, different versions) md5 (with a bit of help) Sponsored by: NLnet Foundation, The FreeBSD Foundation X-MFC after: never V_Commit_Message_Reviewed_By: more people than the patch
2008-08-17 23:27:27 +00:00
V_ifindex_table = e;
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
return (old);
Wrap array accesses in macros, which also happen to be lvalues: ifnet_addrs[i - 1] -> ifaddr_byindex(i) ifindex2ifnet[i] -> ifnet_byindex(i) This is intended to ease the conversion to SMPng.
2001-09-06 02:40:43 +00:00
}
Stop embedding struct ifnet at the top of driver softcs. Instead the struct ifnet or the layer 2 common structure it was embedded in have been replaced with a struct ifnet pointer to be filled by a call to the new function, if_alloc(). The layer 2 common structure is also allocated via if_alloc() based on the interface type. It is hung off the new struct ifnet member, if_l2com. This change removes the size of these structures from the kernel ABI and will allow us to better manage them as interfaces come and go. Other changes of note: - Struct arpcom is no longer referenced in normal interface code. Instead the Ethernet address is accessed via the IFP2ENADDR() macro. To enforce this ac_enaddr has been renamed to _ac_enaddr. - The second argument to ether_ifattach is now always the mac address from driver private storage rather than sometimes being ac_enaddr. Reviewed by: sobomax, sam
2005-06-10 16:49:24 +00:00
/*
Update the comments on if_alloc(), if_free(), if_free_type(), and if_attach. Remove a comment about pre-3.0 network drivers from if_attach(). Be a bit more consistant about whitespace near comments.
2007-05-16 19:59:01 +00:00
* Allocate a struct ifnet and an index for an interface. A layer 2
* common structure will also be allocated if an allocation routine is
* registered for the passed type.
Stop embedding struct ifnet at the top of driver softcs. Instead the struct ifnet or the layer 2 common structure it was embedded in have been replaced with a struct ifnet pointer to be filled by a call to the new function, if_alloc(). The layer 2 common structure is also allocated via if_alloc() based on the interface type. It is hung off the new struct ifnet member, if_l2com. This change removes the size of these structures from the kernel ABI and will allow us to better manage them as interfaces come and go. Other changes of note: - Struct arpcom is no longer referenced in normal interface code. Instead the Ethernet address is accessed via the IFP2ENADDR() macro. To enforce this ac_enaddr has been renamed to _ac_enaddr. - The second argument to ether_ifattach is now always the mac address from driver private storage rather than sometimes being ac_enaddr. Reviewed by: sobomax, sam
2005-06-10 16:49:24 +00:00
*/
Move portions of data structure initialization from if_attach() to if_alloc(), and portions of data structure destruction from if_detach() to if_free(). These changes leave more of the struct ifnet in a safe-to-access condition between alloc and attach, and between detach and free, and focus on attach/detach as stack usage events rather than data structure initialization. Affected fields include the linkstate task queue, if_afdata lock, address lists, kqueue state, and MAC labels. ifq_attach() ifq_detach() are not moved as ifq_attach() may use a queue length set by the device driver between if_alloc() and if_attach(). MFC after: 3 weeks
2009-04-23 10:59:40 +00:00
struct ifnet *
Track device's NUMA domain in ifnet & alloc ifnet from NUMA local memory This commit adds new if_alloc_domain() and if_alloc_dev() methods to allocate ifnets. When called with a domain on a NUMA machine, ifalloc_domain() will record the NUMA domain in the ifnet, and it will allocate the ifnet struct from memory which is local to that NUMA node. Similarly, if_alloc_dev() is a wrapper for if_alloc_domain which uses a driver supplied device_t to call ifalloc_domain() with the appropriate domain. Note that the new if_numa_domain field fits in an alignment pad in struct ifnet, and so does not alter the size of the structure. Reviewed by: glebius, kib, markj Sponsored by: Netflix Differential Revision: https://reviews.freebsd.org/D19930
2019-04-22 19:24:21 +00:00
if_alloc_domain(u_char type, int numa_domain)
Stop embedding struct ifnet at the top of driver softcs. Instead the struct ifnet or the layer 2 common structure it was embedded in have been replaced with a struct ifnet pointer to be filled by a call to the new function, if_alloc(). The layer 2 common structure is also allocated via if_alloc() based on the interface type. It is hung off the new struct ifnet member, if_l2com. This change removes the size of these structures from the kernel ABI and will allow us to better manage them as interfaces come and go. Other changes of note: - Struct arpcom is no longer referenced in normal interface code. Instead the Ethernet address is accessed via the IFP2ENADDR() macro. To enforce this ac_enaddr has been renamed to _ac_enaddr. - The second argument to ether_ifattach is now always the mac address from driver private storage rather than sometimes being ac_enaddr. Reviewed by: sobomax, sam
2005-06-10 16:49:24 +00:00
{
struct ifnet *ifp;
Break out allocation of new ifindex values from if_alloc() and if_vmove(), and centralize in a single function ifindex_alloc(). Assert the IFNET_WLOCK, and add missing IFNET_WLOCK in if_alloc(). This does not close all known races in this code. Reviewed by: bz MFC after: 3 days
2009-08-25 20:21:16 +00:00
u_short idx;
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
void *old;
Stop embedding struct ifnet at the top of driver softcs. Instead the struct ifnet or the layer 2 common structure it was embedded in have been replaced with a struct ifnet pointer to be filled by a call to the new function, if_alloc(). The layer 2 common structure is also allocated via if_alloc() based on the interface type. It is hung off the new struct ifnet member, if_l2com. This change removes the size of these structures from the kernel ABI and will allow us to better manage them as interfaces come and go. Other changes of note: - Struct arpcom is no longer referenced in normal interface code. Instead the Ethernet address is accessed via the IFP2ENADDR() macro. To enforce this ac_enaddr has been renamed to _ac_enaddr. - The second argument to ether_ifattach is now always the mac address from driver private storage rather than sometimes being ac_enaddr. Reviewed by: sobomax, sam
2005-06-10 16:49:24 +00:00
Track device's NUMA domain in ifnet & alloc ifnet from NUMA local memory This commit adds new if_alloc_domain() and if_alloc_dev() methods to allocate ifnets. When called with a domain on a NUMA machine, ifalloc_domain() will record the NUMA domain in the ifnet, and it will allocate the ifnet struct from memory which is local to that NUMA node. Similarly, if_alloc_dev() is a wrapper for if_alloc_domain which uses a driver supplied device_t to call ifalloc_domain() with the appropriate domain. Note that the new if_numa_domain field fits in an alignment pad in struct ifnet, and so does not alter the size of the structure. Reviewed by: glebius, kib, markj Sponsored by: Netflix Differential Revision: https://reviews.freebsd.org/D19930
2019-04-22 19:24:21 +00:00
KASSERT(numa_domain <= IF_NODOM, ("numa_domain too large"));
if (numa_domain == IF_NODOM)
ifp = malloc(sizeof(struct ifnet), M_IFNET,
M_WAITOK | M_ZERO);
else
ifp = malloc_domainset(sizeof(struct ifnet), M_IFNET,
DOMAINSET_PREF(numa_domain), M_WAITOK | M_ZERO);
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
restart:
Break out allocation of new ifindex values from if_alloc() and if_vmove(), and centralize in a single function ifindex_alloc(). Assert the IFNET_WLOCK, and add missing IFNET_WLOCK in if_alloc(). This does not close all known races in this code. Reviewed by: bz MFC after: 3 days
2009-08-25 20:21:16 +00:00
IFNET_WLOCK();
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
idx = ifindex_alloc(&old);
if (__predict_false(idx == USHRT_MAX)) {
IFNET_WUNLOCK();
epoch_wait_preempt(net_epoch_preempt);
free(old, M_IFNET);
goto restart;
}
ifnet_setbyindex(idx, IFNET_HOLD);
Break out allocation of new ifindex values from if_alloc() and if_vmove(), and centralize in a single function ifindex_alloc(). Assert the IFNET_WLOCK, and add missing IFNET_WLOCK in if_alloc(). This does not close all known races in this code. Reviewed by: bz MFC after: 3 days
2009-08-25 20:21:16 +00:00
IFNET_WUNLOCK();
ifp->if_index = idx;
Stop embedding struct ifnet at the top of driver softcs. Instead the struct ifnet or the layer 2 common structure it was embedded in have been replaced with a struct ifnet pointer to be filled by a call to the new function, if_alloc(). The layer 2 common structure is also allocated via if_alloc() based on the interface type. It is hung off the new struct ifnet member, if_l2com. This change removes the size of these structures from the kernel ABI and will allow us to better manage them as interfaces come and go. Other changes of note: - Struct arpcom is no longer referenced in normal interface code. Instead the Ethernet address is accessed via the IFP2ENADDR() macro. To enforce this ac_enaddr has been renamed to _ac_enaddr. - The second argument to ether_ifattach is now always the mac address from driver private storage rather than sometimes being ac_enaddr. Reviewed by: sobomax, sam
2005-06-10 16:49:24 +00:00
ifp->if_type = type;
Start to address a number of races relating to use of ifnet pointers after the corresponding interface has been destroyed: (1) Add an ifnet refcount, ifp->if_refcount. Initialize it to 1 in if_alloc(), and modify if_free_type() to decrement and check the refcount. (2) Add new if_ref() and if_rele() interfaces to allow kernel code walking global interface lists to release IFNET_[RW]LOCK() yet keep the ifnet stable. Currently, if_rele() is a no-op wrapper around if_free(), but this may change in the future. (3) Add new ifnet field, if_alloctype, which caches the type passed to if_alloc(), but unlike if_type, won't be changed by drivers. This allows asynchronous free's of the interface after the driver has released it to still use the right type. Use that instead of the type passed to if_free_type(), but assert that they are the same (might have to rethink this if that doesn't work out). (4) Add a new ifnet_byindex_ref(), which looks up an interface by index and returns a reference rather than a pointer to it. (5) Fix if_alloc() to fully initialize the if_addr_mtx before hooking up the ifnet to global lists. (6) Modify sysctls in if_mib.c to use ifnet_byindex_ref() and release the ifnet when done. When this change is MFC'd, it will need to replace if_ispare fields rather than adding new fields in order to avoid breaking the binary interface. Once this change is MFC'd, if_free_type() should be removed, as its 'type' argument is now optional. This refcount is not appropriate for counting mbuf pkthdr references, and also not for counting entry into the device driver via ifnet function pointers. An rmlock may be appropriate for the latter. Rather, this is about ensuring data structure stability when reaching an ifnet via global ifnet lists and tables followed by copy in or out of userspace. MFC after: 3 weeks Reported by: mdtancsa Reviewed by: brooks
2009-04-21 22:43:32 +00:00
ifp->if_alloctype = type;
Track device's NUMA domain in ifnet & alloc ifnet from NUMA local memory This commit adds new if_alloc_domain() and if_alloc_dev() methods to allocate ifnets. When called with a domain on a NUMA machine, ifalloc_domain() will record the NUMA domain in the ifnet, and it will allocate the ifnet struct from memory which is local to that NUMA node. Similarly, if_alloc_dev() is a wrapper for if_alloc_domain which uses a driver supplied device_t to call ifalloc_domain() with the appropriate domain. Note that the new if_numa_domain field fits in an alignment pad in struct ifnet, and so does not alter the size of the structure. Reviewed by: glebius, kib, markj Sponsored by: Netflix Differential Revision: https://reviews.freebsd.org/D19930
2019-04-22 19:24:21 +00:00
ifp->if_numa_domain = numa_domain;
Several device drivers call if_alloc() and then do further checks and will cal if_free() in case of conflict, error, .. if_free() however sets the VNET instance from the ifp->if_vnet which was not yet initialized but would only in if_attach(). Fix this by setting the curvnet from where we allocate the interface in if_alloc(). if_attach() will later overwrite this as needed. We do not set the home_vnet early on as we only want to prevent the if_free() panic but not change any of the other housekeeping, e.g., triggered through ifioctl()s. Reviewed by: brooks Approved by: re (gjb) MFC after: 2 weeks Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D7010
2016-06-29 05:21:25 +00:00
#ifdef VIMAGE
ifp->if_vnet = curvnet;
#endif
Stop embedding struct ifnet at the top of driver softcs. Instead the struct ifnet or the layer 2 common structure it was embedded in have been replaced with a struct ifnet pointer to be filled by a call to the new function, if_alloc(). The layer 2 common structure is also allocated via if_alloc() based on the interface type. It is hung off the new struct ifnet member, if_l2com. This change removes the size of these structures from the kernel ABI and will allow us to better manage them as interfaces come and go. Other changes of note: - Struct arpcom is no longer referenced in normal interface code. Instead the Ethernet address is accessed via the IFP2ENADDR() macro. To enforce this ac_enaddr has been renamed to _ac_enaddr. - The second argument to ether_ifattach is now always the mac address from driver private storage rather than sometimes being ac_enaddr. Reviewed by: sobomax, sam
2005-06-10 16:49:24 +00:00
if (if_com_alloc[type] != NULL) {
ifp->if_l2com = if_com_alloc[type](type, ifp);
Return NULL instead of a bogus pointer from if_alloc when if_com_alloc fails. Move detaching the ifnet from the ifindex_table into if_free so we can both keep the sanity checks and actually delete the ifnets. [0] Reported by: gallatin [0] Approved by: re (blanket)
2005-06-12 00:53:03 +00:00
if (ifp->if_l2com == NULL) {
Stop embedding struct ifnet at the top of driver softcs. Instead the struct ifnet or the layer 2 common structure it was embedded in have been replaced with a struct ifnet pointer to be filled by a call to the new function, if_alloc(). The layer 2 common structure is also allocated via if_alloc() based on the interface type. It is hung off the new struct ifnet member, if_l2com. This change removes the size of these structures from the kernel ABI and will allow us to better manage them as interfaces come and go. Other changes of note: - Struct arpcom is no longer referenced in normal interface code. Instead the Ethernet address is accessed via the IFP2ENADDR() macro. To enforce this ac_enaddr has been renamed to _ac_enaddr. - The second argument to ether_ifattach is now always the mac address from driver private storage rather than sometimes being ac_enaddr. Reviewed by: sobomax, sam
2005-06-10 16:49:24 +00:00
free(ifp, M_IFNET);
Add IFNET_HOLD reserved pointer value for the ifindex ifnet array, which allows an index to be reserved for an ifnet without making the ifnet available for management operations. Use this in if_alloc() while the ifnet lock is released between initial index allocation and completion of ifnet initialization. Add ifindex_free() to centralize the implementation of releasing an ifindex value. Use in if_free() and if_vmove(), as well as when releasing a held index in if_alloc(). Reviewed by: bz MFC after: 3 days
2009-08-26 11:13:10 +00:00
ifindex_free(idx);
Return NULL instead of a bogus pointer from if_alloc when if_com_alloc fails. Move detaching the ifnet from the ifindex_table into if_free so we can both keep the sanity checks and actually delete the ifnets. [0] Reported by: gallatin [0] Approved by: re (blanket)
2005-06-12 00:53:03 +00:00
return (NULL);
}
Stop embedding struct ifnet at the top of driver softcs. Instead the struct ifnet or the layer 2 common structure it was embedded in have been replaced with a struct ifnet pointer to be filled by a call to the new function, if_alloc(). The layer 2 common structure is also allocated via if_alloc() based on the interface type. It is hung off the new struct ifnet member, if_l2com. This change removes the size of these structures from the kernel ABI and will allow us to better manage them as interfaces come and go. Other changes of note: - Struct arpcom is no longer referenced in normal interface code. Instead the Ethernet address is accessed via the IFP2ENADDR() macro. To enforce this ac_enaddr has been renamed to _ac_enaddr. - The second argument to ether_ifattach is now always the mac address from driver private storage rather than sometimes being ac_enaddr. Reviewed by: sobomax, sam
2005-06-10 16:49:24 +00:00
}
Start to address a number of races relating to use of ifnet pointers after the corresponding interface has been destroyed: (1) Add an ifnet refcount, ifp->if_refcount. Initialize it to 1 in if_alloc(), and modify if_free_type() to decrement and check the refcount. (2) Add new if_ref() and if_rele() interfaces to allow kernel code walking global interface lists to release IFNET_[RW]LOCK() yet keep the ifnet stable. Currently, if_rele() is a no-op wrapper around if_free(), but this may change in the future. (3) Add new ifnet field, if_alloctype, which caches the type passed to if_alloc(), but unlike if_type, won't be changed by drivers. This allows asynchronous free's of the interface after the driver has released it to still use the right type. Use that instead of the type passed to if_free_type(), but assert that they are the same (might have to rethink this if that doesn't work out). (4) Add a new ifnet_byindex_ref(), which looks up an interface by index and returns a reference rather than a pointer to it. (5) Fix if_alloc() to fully initialize the if_addr_mtx before hooking up the ifnet to global lists. (6) Modify sysctls in if_mib.c to use ifnet_byindex_ref() and release the ifnet when done. When this change is MFC'd, it will need to replace if_ispare fields rather than adding new fields in order to avoid breaking the binary interface. Once this change is MFC'd, if_free_type() should be removed, as its 'type' argument is now optional. This refcount is not appropriate for counting mbuf pkthdr references, and also not for counting entry into the device driver via ifnet function pointers. An rmlock may be appropriate for the latter. Rather, this is about ensuring data structure stability when reaching an ifnet via global ifnet lists and tables followed by copy in or out of userspace. MFC after: 3 weeks Reported by: mdtancsa Reviewed by: brooks
2009-04-21 22:43:32 +00:00
IF_ADDR_LOCK_INIT(ifp);
Move portions of data structure initialization from if_attach() to if_alloc(), and portions of data structure destruction from if_detach() to if_free(). These changes leave more of the struct ifnet in a safe-to-access condition between alloc and attach, and between detach and free, and focus on attach/detach as stack usage events rather than data structure initialization. Affected fields include the linkstate task queue, if_afdata lock, address lists, kqueue state, and MAC labels. ifq_attach() ifq_detach() are not moved as ifq_attach() may use a queue length set by the device driver between if_alloc() and if_attach(). MFC after: 3 weeks
2009-04-23 10:59:40 +00:00
TASK_INIT(&ifp->if_linktask, 0, do_link_state_change, ifp);
ifp->if_afdata_initialized = 0;
Introduce the if_vmove() function, which will be used in the future for reassigning ifnets from one vnet to another. if_vmove() works by calling a restricted subset of actions normally executed by if_detach() on an ifnet in the current vnet, and then switches to the target vnet and executes an appropriate subset of if_attach() actions there. if_attach() and if_detach() have become wrapper functions around if_attach_internal() and if_detach_internal(), where the later variants have an additional argument, a flag indicating whether a full attach or detach sequence is to be executed, or only a restricted subset suitable for moving an ifnet from one vnet to another. Hence, if_vmove() will not call if_detach() and if_attach() directly, but will call the if_detach_internal() and if_attach_internal() variants instead, with the vmove flag set. While here, staticize ifnet_setbyindex() since it is not referenced from outside of sys/net/if.c. Also rename ifccnt field in struct vimage to ifcnt, and do some minor whitespace garbage collection where appropriate. This change should have no functional impact on nooptions VIMAGE kernel builds. Reviewed by: bz, rwatson, brooks? Approved by: julian (mentor)
2009-05-22 22:09:00 +00:00
IF_AFDATA_LOCK_INIT(ifp);
ifnet: Replace if_addr_lock rwlock with epoch + mutex Run on LLNW canaries and tested by pho@ gallatin: Using a 14-core, 28-HTT single socket E5-2697 v3 with a 40GbE MLX5 based ConnectX 4-LX NIC, I see an almost 12% improvement in received packet rate, and a larger improvement in bytes delivered all the way to userspace. When the host receiving 64 streams of netperf -H $DUT -t UDP_STREAM -- -m 1, I see, using nstat -I mce0 1 before the patch: InMpps OMpps InGbs OGbs err TCP Est %CPU syscalls csw irq GBfree 4.98 0.00 4.42 0.00 4235592 33 83.80 4720653 2149771 1235 247.32 4.73 0.00 4.20 0.00 4025260 33 82.99 4724900 2139833 1204 247.32 4.72 0.00 4.20 0.00 4035252 33 82.14 4719162 2132023 1264 247.32 4.71 0.00 4.21 0.00 4073206 33 83.68 4744973 2123317 1347 247.32 4.72 0.00 4.21 0.00 4061118 33 80.82 4713615 2188091 1490 247.32 4.72 0.00 4.21 0.00 4051675 33 85.29 4727399 2109011 1205 247.32 4.73 0.00 4.21 0.00 4039056 33 84.65 4724735 2102603 1053 247.32 After the patch InMpps OMpps InGbs OGbs err TCP Est %CPU syscalls csw irq GBfree 5.43 0.00 4.20 0.00 3313143 33 84.96 5434214 1900162 2656 245.51 5.43 0.00 4.20 0.00 3308527 33 85.24 5439695 1809382 2521 245.51 5.42 0.00 4.19 0.00 3316778 33 87.54 5416028 1805835 2256 245.51 5.42 0.00 4.19 0.00 3317673 33 90.44 5426044 1763056 2332 245.51 5.42 0.00 4.19 0.00 3314839 33 88.11 5435732 1792218 2499 245.52 5.44 0.00 4.19 0.00 3293228 33 91.84 5426301 1668597 2121 245.52 Similarly, netperf reports 230Mb/s before the patch, and 270Mb/s after the patch Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15366
2018-05-18 20:13:34 +00:00
CK_STAILQ_INIT(&ifp->if_addrhead);
CK_STAILQ_INIT(&ifp->if_multiaddrs);
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
CK_STAILQ_INIT(&ifp->if_groups);
Move portions of data structure initialization from if_attach() to if_alloc(), and portions of data structure destruction from if_detach() to if_free(). These changes leave more of the struct ifnet in a safe-to-access condition between alloc and attach, and between detach and free, and focus on attach/detach as stack usage events rather than data structure initialization. Affected fields include the linkstate task queue, if_afdata lock, address lists, kqueue state, and MAC labels. ifq_attach() ifq_detach() are not moved as ifq_attach() may use a queue length set by the device driver between if_alloc() and if_attach(). MFC after: 3 weeks
2009-04-23 10:59:40 +00:00
#ifdef MAC
mac_ifnet_init(ifp);
#endif
r193336 moved ifq_detach to if_free which broke if_alloc followed by if_free (w/o doing if_attach); move ifq_attach to if_alloc and rename ifq_attach/detach to ifq_init/ifq_delete to better identify their purpose Reviewed by: jhb, kmacy
2009-06-15 19:50:03 +00:00
ifq_init(&ifp->if_snd, ifp);
Move portions of data structure initialization from if_attach() to if_alloc(), and portions of data structure destruction from if_detach() to if_free(). These changes leave more of the struct ifnet in a safe-to-access condition between alloc and attach, and between detach and free, and focus on attach/detach as stack usage events rather than data structure initialization. Affected fields include the linkstate task queue, if_afdata lock, address lists, kqueue state, and MAC labels. ifq_attach() ifq_detach() are not moved as ifq_attach() may use a queue length set by the device driver between if_alloc() and if_attach(). MFC after: 3 weeks
2009-04-23 10:59:40 +00:00
Start to address a number of races relating to use of ifnet pointers after the corresponding interface has been destroyed: (1) Add an ifnet refcount, ifp->if_refcount. Initialize it to 1 in if_alloc(), and modify if_free_type() to decrement and check the refcount. (2) Add new if_ref() and if_rele() interfaces to allow kernel code walking global interface lists to release IFNET_[RW]LOCK() yet keep the ifnet stable. Currently, if_rele() is a no-op wrapper around if_free(), but this may change in the future. (3) Add new ifnet field, if_alloctype, which caches the type passed to if_alloc(), but unlike if_type, won't be changed by drivers. This allows asynchronous free's of the interface after the driver has released it to still use the right type. Use that instead of the type passed to if_free_type(), but assert that they are the same (might have to rethink this if that doesn't work out). (4) Add a new ifnet_byindex_ref(), which looks up an interface by index and returns a reference rather than a pointer to it. (5) Fix if_alloc() to fully initialize the if_addr_mtx before hooking up the ifnet to global lists. (6) Modify sysctls in if_mib.c to use ifnet_byindex_ref() and release the ifnet when done. When this change is MFC'd, it will need to replace if_ispare fields rather than adding new fields in order to avoid breaking the binary interface. Once this change is MFC'd, if_free_type() should be removed, as its 'type' argument is now optional. This refcount is not appropriate for counting mbuf pkthdr references, and also not for counting entry into the device driver via ifnet function pointers. An rmlock may be appropriate for the latter. Rather, this is about ensuring data structure stability when reaching an ifnet via global ifnet lists and tables followed by copy in or out of userspace. MFC after: 3 weeks Reported by: mdtancsa Reviewed by: brooks
2009-04-21 22:43:32 +00:00
refcount_init(&ifp->if_refcount, 1); /* Index reference. */
Finally, convert counters in struct ifnet to counter(9). Sponsored by: Netflix Sponsored by: Nginx, Inc.
2014-09-28 08:57:07 +00:00
for (int i = 0; i < IFCOUNTERS; i++)
ifp->if_counters[i] = counter_u64_alloc(M_WAITOK);
Move if_get_counter initialization from if_attach into if_alloc. Also, initialize all counters before ifnet will become available in the system. This fixes possible access to uninitialized ifned fields. PR: 194550
2014-10-23 14:29:52 +00:00
ifp->if_get_counter = if_get_counter_default;
Allow to specify PCP on packets not belonging to any VLAN. According to 802.1Q-2014, VLAN tagged packets with VLAN id 0 should be considered as untagged, and only PCP and DEI values from the VLAN tag are meaningful. See for instance https://www.cisco.com/c/en/us/td/docs/switches/connectedgrid/cg-switch-sw-master/software/configuration/guide/vlan0/b_vlan_0.html. Make it possible to specify PCP value for outgoing packets on an ethernet interface. When PCP is supplied, the tag is appended, VLAN id set to 0, and PCP is filled by the supplied value. The code to do VLAN tag encapsulation is refactored from the if_vlan.c and moved into if_ethersubr.c. Drivers might have issues with filtering VID 0 packets on receive. This bug should be fixed for each driver. Reviewed by: ae (previous version), hselasky, melifaro Sponsored by: Mellanox Technologies MFC after: 2 weeks Differential revision: https://reviews.freebsd.org/D14702
2018-03-27 15:29:32 +00:00
ifp->if_pcp = IFNET_PCP_NONE;
Move if_get_counter initialization from if_attach into if_alloc. Also, initialize all counters before ifnet will become available in the system. This fixes possible access to uninitialized ifned fields. PR: 194550
2014-10-23 14:29:52 +00:00
ifnet_setbyindex(ifp->if_index, ifp);
Stop embedding struct ifnet at the top of driver softcs. Instead the struct ifnet or the layer 2 common structure it was embedded in have been replaced with a struct ifnet pointer to be filled by a call to the new function, if_alloc(). The layer 2 common structure is also allocated via if_alloc() based on the interface type. It is hung off the new struct ifnet member, if_l2com. This change removes the size of these structures from the kernel ABI and will allow us to better manage them as interfaces come and go. Other changes of note: - Struct arpcom is no longer referenced in normal interface code. Instead the Ethernet address is accessed via the IFP2ENADDR() macro. To enforce this ac_enaddr has been renamed to _ac_enaddr. - The second argument to ether_ifattach is now always the mac address from driver private storage rather than sometimes being ac_enaddr. Reviewed by: sobomax, sam
2005-06-10 16:49:24 +00:00
return (ifp);
}
Track device's NUMA domain in ifnet & alloc ifnet from NUMA local memory This commit adds new if_alloc_domain() and if_alloc_dev() methods to allocate ifnets. When called with a domain on a NUMA machine, ifalloc_domain() will record the NUMA domain in the ifnet, and it will allocate the ifnet struct from memory which is local to that NUMA node. Similarly, if_alloc_dev() is a wrapper for if_alloc_domain which uses a driver supplied device_t to call ifalloc_domain() with the appropriate domain. Note that the new if_numa_domain field fits in an alignment pad in struct ifnet, and so does not alter the size of the structure. Reviewed by: glebius, kib, markj Sponsored by: Netflix Differential Revision: https://reviews.freebsd.org/D19930
2019-04-22 19:24:21 +00:00
struct ifnet *
if_alloc_dev(u_char type, device_t dev)
{
int numa_domain;
if (dev == NULL || bus_get_domain(dev, &numa_domain) != 0)
return (if_alloc_domain(type, IF_NODOM));
return (if_alloc_domain(type, numa_domain));
}
struct ifnet *
if_alloc(u_char type)
{
return (if_alloc_domain(type, IF_NODOM));
}
Update the comments on if_alloc(), if_free(), if_free_type(), and if_attach. Remove a comment about pre-3.0 network drivers from if_attach(). Be a bit more consistant about whitespace near comments.
2007-05-16 19:59:01 +00:00
/*
When removing ifnets, we should first remove the reference to ifnet from the interface index, then decrease refcount, not vice versa. Otherwise there is a race (reproducible) when if_free_internal() contests on IFNET_WLOCK(), and we got a zero-refed ifnet in the index for a long time. It may be picked by some other thread, that runs ifnet_byindex_ref(), who takes the ifnet from index, and bumps refcount. When reader drops the lock, if_free_internal() proceeds with free. Then reader tries to free it a second time.
2011-04-04 07:45:08 +00:00
* Do the actual work of freeing a struct ifnet, and layer 2 common
* structure. This call is made when the last reference to an
Add a new interface flag, IFF_DYING, which is set when a device driver calls if_free(), and remains set if the refcount is elevated. IF_DYING skips the bit in the if_flags bitmask previously used by IFF_NEEDSGIANT, so that an MFC can be done without changing which bit is used, as IFF_NEEDSGIANT is still present in 7.x. ifnet_byindex_ref() checks for IFF_DYING and returns NULL if it is set, preventing new references from by acquired by index, preventing monitoring sysctls from seeing it. Other lookup mechanisms currently do not check IFF_DYING, but may need to in the future. MFC after: 3 weeks
2009-04-23 09:32:30 +00:00
* interface is released.
Update the comments on if_alloc(), if_free(), if_free_type(), and if_attach. Remove a comment about pre-3.0 network drivers from if_attach(). Be a bit more consistant about whitespace near comments.
2007-05-16 19:59:01 +00:00
*/
Add a new interface flag, IFF_DYING, which is set when a device driver calls if_free(), and remains set if the refcount is elevated. IF_DYING skips the bit in the if_flags bitmask previously used by IFF_NEEDSGIANT, so that an MFC can be done without changing which bit is used, as IFF_NEEDSGIANT is still present in 7.x. ifnet_byindex_ref() checks for IFF_DYING and returns NULL if it is set, preventing new references from by acquired by index, preventing monitoring sysctls from seeing it. Other lookup mechanisms currently do not check IFF_DYING, but may need to in the future. MFC after: 3 weeks
2009-04-23 09:32:30 +00:00
static void
if_free_internal(struct ifnet *ifp)
Stop embedding struct ifnet at the top of driver softcs. Instead the struct ifnet or the layer 2 common structure it was embedded in have been replaced with a struct ifnet pointer to be filled by a call to the new function, if_alloc(). The layer 2 common structure is also allocated via if_alloc() based on the interface type. It is hung off the new struct ifnet member, if_l2com. This change removes the size of these structures from the kernel ABI and will allow us to better manage them as interfaces come and go. Other changes of note: - Struct arpcom is no longer referenced in normal interface code. Instead the Ethernet address is accessed via the IFP2ENADDR() macro. To enforce this ac_enaddr has been renamed to _ac_enaddr. - The second argument to ether_ifattach is now always the mac address from driver private storage rather than sometimes being ac_enaddr. Reviewed by: sobomax, sam
2005-06-10 16:49:24 +00:00
{
Add a new interface flag, IFF_DYING, which is set when a device driver calls if_free(), and remains set if the refcount is elevated. IF_DYING skips the bit in the if_flags bitmask previously used by IFF_NEEDSGIANT, so that an MFC can be done without changing which bit is used, as IFF_NEEDSGIANT is still present in 7.x. ifnet_byindex_ref() checks for IFF_DYING and returns NULL if it is set, preventing new references from by acquired by index, preventing monitoring sysctls from seeing it. Other lookup mechanisms currently do not check IFF_DYING, but may need to in the future. MFC after: 3 weeks
2009-04-23 09:32:30 +00:00
KASSERT((ifp->if_flags & IFF_DYING),
("if_free_internal: interface not dying"));
if (if_com_free[ifp->if_alloctype] != NULL)
if_com_free[ifp->if_alloctype](ifp->if_l2com,
ifp->if_alloctype);
Move portions of data structure initialization from if_attach() to if_alloc(), and portions of data structure destruction from if_detach() to if_free(). These changes leave more of the struct ifnet in a safe-to-access condition between alloc and attach, and between detach and free, and focus on attach/detach as stack usage events rather than data structure initialization. Affected fields include the linkstate task queue, if_afdata lock, address lists, kqueue state, and MAC labels. ifq_attach() ifq_detach() are not moved as ifq_attach() may use a queue length set by the device driver between if_alloc() and if_attach(). MFC after: 3 weeks
2009-04-23 10:59:40 +00:00
#ifdef MAC
mac_ifnet_destroy(ifp);
#endif /* MAC */
IF_AFDATA_DESTROY(ifp);
Add a new interface flag, IFF_DYING, which is set when a device driver calls if_free(), and remains set if the refcount is elevated. IF_DYING skips the bit in the if_flags bitmask previously used by IFF_NEEDSGIANT, so that an MFC can be done without changing which bit is used, as IFF_NEEDSGIANT is still present in 7.x. ifnet_byindex_ref() checks for IFF_DYING and returns NULL if it is set, preventing new references from by acquired by index, preventing monitoring sysctls from seeing it. Other lookup mechanisms currently do not check IFF_DYING, but may need to in the future. MFC after: 3 weeks
2009-04-23 09:32:30 +00:00
IF_ADDR_LOCK_DESTROY(ifp);
r193336 moved ifq_detach to if_free which broke if_alloc followed by if_free (w/o doing if_attach); move ifq_attach to if_alloc and rename ifq_attach/detach to ifq_init/ifq_delete to better identify their purpose Reviewed by: jhb, kmacy
2009-06-15 19:50:03 +00:00
ifq_delete(&ifp->if_snd);
Finally, convert counters in struct ifnet to counter(9). Sponsored by: Netflix Sponsored by: Nginx, Inc.
2014-09-28 08:57:07 +00:00
for (int i = 0; i < IFCOUNTERS; i++)
counter_u64_free(ifp->if_counters[i]);
Adapt the fix in r341008 to correctly work with EBR. IFNET_RLOCK_NOSLEEP() is epoch_enter_preempt() in FreeBSD 12+. Holding it in sysctl_rtsock() doesn't protect us from ifnet unlinking, because unlinking occurs with IFNET_WLOCK(), that is rw_wlock+sx_xlock, and it doesn check that concurrent code is running in epoch section. But while we are in epoch section, we should be able to do access to ifnet's fields, even it was unlinked. Thus do not change if_addr and if_hw_addr fields in ifnet_detach_internal() to NULL, since rtsock code can do access to these fields and this is allowed while it is running in epoch section. This should fix the race, when ifnet_detach_internal() unlinks ifnet after we checked it for IFF_DYING in sysctl_dumpentry. Move free(ifp->if_hw_addr) into ifnet_free_internal(). Also remove the NULL check for ifp->if_description, since free(9) can correctly handle NULL pointer. MFC after: 1 week
2018-11-30 10:36:14 +00:00
free(ifp->if_description, M_IFDESCR);
free(ifp->if_hw_addr, M_IFADDR);
Track device's NUMA domain in ifnet & alloc ifnet from NUMA local memory This commit adds new if_alloc_domain() and if_alloc_dev() methods to allocate ifnets. When called with a domain on a NUMA machine, ifalloc_domain() will record the NUMA domain in the ifnet, and it will allocate the ifnet struct from memory which is local to that NUMA node. Similarly, if_alloc_dev() is a wrapper for if_alloc_domain which uses a driver supplied device_t to call ifalloc_domain() with the appropriate domain. Note that the new if_numa_domain field fits in an alignment pad in struct ifnet, and so does not alter the size of the structure. Reviewed by: glebius, kib, markj Sponsored by: Netflix Differential Revision: https://reviews.freebsd.org/D19930
2019-04-22 19:24:21 +00:00
if (ifp->if_numa_domain == IF_NODOM)
free(ifp, M_IFNET);
else
free_domain(ifp, M_IFNET);
Stop embedding struct ifnet at the top of driver softcs. Instead the struct ifnet or the layer 2 common structure it was embedded in have been replaced with a struct ifnet pointer to be filled by a call to the new function, if_alloc(). The layer 2 common structure is also allocated via if_alloc() based on the interface type. It is hung off the new struct ifnet member, if_l2com. This change removes the size of these structures from the kernel ABI and will allow us to better manage them as interfaces come and go. Other changes of note: - Struct arpcom is no longer referenced in normal interface code. Instead the Ethernet address is accessed via the IFP2ENADDR() macro. To enforce this ac_enaddr has been renamed to _ac_enaddr. - The second argument to ether_ifattach is now always the mac address from driver private storage rather than sometimes being ac_enaddr. Reviewed by: sobomax, sam
2005-06-10 16:49:24 +00:00
}
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
static void
if_destroy(epoch_context_t ctx)
{
struct ifnet *ifp;
ifp = __containerof(ctx, struct ifnet, if_epoch_ctx);
if_free_internal(ifp);
}
Update the comments on if_alloc(), if_free(), if_free_type(), and if_attach. Remove a comment about pre-3.0 network drivers from if_attach(). Be a bit more consistant about whitespace near comments.
2007-05-16 19:59:01 +00:00
/*
Remove the unused if_free_type() function. X-MFC after: never
2011-12-09 23:26:28 +00:00
* Deregister an interface and free the associated storage.
Update the comments on if_alloc(), if_free(), if_free_type(), and if_attach. Remove a comment about pre-3.0 network drivers from if_attach(). Be a bit more consistant about whitespace near comments.
2007-05-16 19:59:01 +00:00
*/
Stop embedding struct ifnet at the top of driver softcs. Instead the struct ifnet or the layer 2 common structure it was embedded in have been replaced with a struct ifnet pointer to be filled by a call to the new function, if_alloc(). The layer 2 common structure is also allocated via if_alloc() based on the interface type. It is hung off the new struct ifnet member, if_l2com. This change removes the size of these structures from the kernel ABI and will allow us to better manage them as interfaces come and go. Other changes of note: - Struct arpcom is no longer referenced in normal interface code. Instead the Ethernet address is accessed via the IFP2ENADDR() macro. To enforce this ac_enaddr has been renamed to _ac_enaddr. - The second argument to ether_ifattach is now always the mac address from driver private storage rather than sometimes being ac_enaddr. Reviewed by: sobomax, sam
2005-06-10 16:49:24 +00:00
void
Remove the unused if_free_type() function. X-MFC after: never
2011-12-09 23:26:28 +00:00
if_free(struct ifnet *ifp)
Stop embedding struct ifnet at the top of driver softcs. Instead the struct ifnet or the layer 2 common structure it was embedded in have been replaced with a struct ifnet pointer to be filled by a call to the new function, if_alloc(). The layer 2 common structure is also allocated via if_alloc() based on the interface type. It is hung off the new struct ifnet member, if_l2com. This change removes the size of these structures from the kernel ABI and will allow us to better manage them as interfaces come and go. Other changes of note: - Struct arpcom is no longer referenced in normal interface code. Instead the Ethernet address is accessed via the IFP2ENADDR() macro. To enforce this ac_enaddr has been renamed to _ac_enaddr. - The second argument to ether_ifattach is now always the mac address from driver private storage rather than sometimes being ac_enaddr. Reviewed by: sobomax, sam
2005-06-10 16:49:24 +00:00
{
Add a new interface flag, IFF_DYING, which is set when a device driver calls if_free(), and remains set if the refcount is elevated. IF_DYING skips the bit in the if_flags bitmask previously used by IFF_NEEDSGIANT, so that an MFC can be done without changing which bit is used, as IFF_NEEDSGIANT is still present in 7.x. ifnet_byindex_ref() checks for IFF_DYING and returns NULL if it is set, preventing new references from by acquired by index, preventing monitoring sysctls from seeing it. Other lookup mechanisms currently do not check IFF_DYING, but may need to in the future. MFC after: 3 weeks
2009-04-23 09:32:30 +00:00
ifp->if_flags |= IFF_DYING; /* XXX: Locking */
When removing ifnets, we should first remove the reference to ifnet from the interface index, then decrease refcount, not vice versa. Otherwise there is a race (reproducible) when if_free_internal() contests on IFNET_WLOCK(), and we got a zero-refed ifnet in the index for a long time. It may be picked by some other thread, that runs ifnet_byindex_ref(), who takes the ifnet from index, and bumps refcount. When reader drops the lock, if_free_internal() proceeds with free. Then reader tries to free it a second time.
2011-04-04 07:45:08 +00:00
PR: 168520 170096 Submitted by: adrian, zec Fix multiple kernel panics when VIMAGE is enabled in the kernel. These fixes are based on patches submitted by Adrian Chadd and Marko Zec. (1) Set curthread->td_vnet to vnet0 in device_probe_and_attach() just before calling device_attach(). This fixes multiple VIMAGE related kernel panics when trying to attach Bluetooth or USB Ethernet devices because curthread->td_vnet is NULL. (2) Set curthread->td_vnet in if_detach(). This fixes kernel panics when detaching networking interfaces, especially USB Ethernet devices. (3) Use VNET_DOMAIN_SET() in ng_btsocket.c (4) In ng_unref_node() set curthread->td_vnet. This fixes kernel panics when detaching Netgraph nodes.
2013-07-15 01:32:55 +00:00
CURVNET_SET_QUIET(ifp->if_vnet);
When removing ifnets, we should first remove the reference to ifnet from the interface index, then decrease refcount, not vice versa. Otherwise there is a race (reproducible) when if_free_internal() contests on IFNET_WLOCK(), and we got a zero-refed ifnet in the index for a long time. It may be picked by some other thread, that runs ifnet_byindex_ref(), who takes the ifnet from index, and bumps refcount. When reader drops the lock, if_free_internal() proceeds with free. Then reader tries to free it a second time.
2011-04-04 07:45:08 +00:00
IFNET_WLOCK();
The two functions ifnet_byindex() and ifnet_byindex_locked() are exactly the same after the network stack was epochified. Merge the two into one function and cleanup all uses of ifnet_byindex_locked(). While at it: - Add branch prediction macros. - Make sure the ifnet pointer is only deferred once, also when code optimisation is disabled. Sponsored by: Mellanox Technologies
2019-10-15 12:08:09 +00:00
KASSERT(ifp == ifnet_byindex(ifp->if_index),
When removing ifnets, we should first remove the reference to ifnet from the interface index, then decrease refcount, not vice versa. Otherwise there is a race (reproducible) when if_free_internal() contests on IFNET_WLOCK(), and we got a zero-refed ifnet in the index for a long time. It may be picked by some other thread, that runs ifnet_byindex_ref(), who takes the ifnet from index, and bumps refcount. When reader drops the lock, if_free_internal() proceeds with free. Then reader tries to free it a second time.
2011-04-04 07:45:08 +00:00
("%s: freeing unallocated ifnet", ifp->if_xname));
ifindex_free_locked(ifp->if_index);
IFNET_WUNLOCK();
PR: 168520 170096 Submitted by: adrian, zec Fix multiple kernel panics when VIMAGE is enabled in the kernel. These fixes are based on patches submitted by Adrian Chadd and Marko Zec. (1) Set curthread->td_vnet to vnet0 in device_probe_and_attach() just before calling device_attach(). This fixes multiple VIMAGE related kernel panics when trying to attach Bluetooth or USB Ethernet devices because curthread->td_vnet is NULL. (2) Set curthread->td_vnet in if_detach(). This fixes kernel panics when detaching networking interfaces, especially USB Ethernet devices. (3) Use VNET_DOMAIN_SET() in ng_btsocket.c (4) In ng_unref_node() set curthread->td_vnet. This fixes kernel panics when detaching Netgraph nodes.
2013-07-15 01:32:55 +00:00
if (refcount_release(&ifp->if_refcount))
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
epoch_call(net_epoch_preempt, &ifp->if_epoch_ctx, if_destroy);
PR: 168520 170096 Submitted by: adrian, zec Fix multiple kernel panics when VIMAGE is enabled in the kernel. These fixes are based on patches submitted by Adrian Chadd and Marko Zec. (1) Set curthread->td_vnet to vnet0 in device_probe_and_attach() just before calling device_attach(). This fixes multiple VIMAGE related kernel panics when trying to attach Bluetooth or USB Ethernet devices because curthread->td_vnet is NULL. (2) Set curthread->td_vnet in if_detach(). This fixes kernel panics when detaching networking interfaces, especially USB Ethernet devices. (3) Use VNET_DOMAIN_SET() in ng_btsocket.c (4) In ng_unref_node() set curthread->td_vnet. This fixes kernel panics when detaching Netgraph nodes.
2013-07-15 01:32:55 +00:00
CURVNET_RESTORE();
Add a new interface flag, IFF_DYING, which is set when a device driver calls if_free(), and remains set if the refcount is elevated. IF_DYING skips the bit in the if_flags bitmask previously used by IFF_NEEDSGIANT, so that an MFC can be done without changing which bit is used, as IFF_NEEDSGIANT is still present in 7.x. ifnet_byindex_ref() checks for IFF_DYING and returns NULL if it is set, preventing new references from by acquired by index, preventing monitoring sysctls from seeing it. Other lookup mechanisms currently do not check IFF_DYING, but may need to in the future. MFC after: 3 weeks
2009-04-23 09:32:30 +00:00
}
Stop embedding struct ifnet at the top of driver softcs. Instead the struct ifnet or the layer 2 common structure it was embedded in have been replaced with a struct ifnet pointer to be filled by a call to the new function, if_alloc(). The layer 2 common structure is also allocated via if_alloc() based on the interface type. It is hung off the new struct ifnet member, if_l2com. This change removes the size of these structures from the kernel ABI and will allow us to better manage them as interfaces come and go. Other changes of note: - Struct arpcom is no longer referenced in normal interface code. Instead the Ethernet address is accessed via the IFP2ENADDR() macro. To enforce this ac_enaddr has been renamed to _ac_enaddr. - The second argument to ether_ifattach is now always the mac address from driver private storage rather than sometimes being ac_enaddr. Reviewed by: sobomax, sam
2005-06-10 16:49:24 +00:00
Add a new interface flag, IFF_DYING, which is set when a device driver calls if_free(), and remains set if the refcount is elevated. IF_DYING skips the bit in the if_flags bitmask previously used by IFF_NEEDSGIANT, so that an MFC can be done without changing which bit is used, as IFF_NEEDSGIANT is still present in 7.x. ifnet_byindex_ref() checks for IFF_DYING and returns NULL if it is set, preventing new references from by acquired by index, preventing monitoring sysctls from seeing it. Other lookup mechanisms currently do not check IFF_DYING, but may need to in the future. MFC after: 3 weeks
2009-04-23 09:32:30 +00:00
/*
* Interfaces to keep an ifnet type-stable despite the possibility of the
* driver calling if_free(). If there are additional references, we defer
* freeing the underlying data structure.
*/
Start to address a number of races relating to use of ifnet pointers after the corresponding interface has been destroyed: (1) Add an ifnet refcount, ifp->if_refcount. Initialize it to 1 in if_alloc(), and modify if_free_type() to decrement and check the refcount. (2) Add new if_ref() and if_rele() interfaces to allow kernel code walking global interface lists to release IFNET_[RW]LOCK() yet keep the ifnet stable. Currently, if_rele() is a no-op wrapper around if_free(), but this may change in the future. (3) Add new ifnet field, if_alloctype, which caches the type passed to if_alloc(), but unlike if_type, won't be changed by drivers. This allows asynchronous free's of the interface after the driver has released it to still use the right type. Use that instead of the type passed to if_free_type(), but assert that they are the same (might have to rethink this if that doesn't work out). (4) Add a new ifnet_byindex_ref(), which looks up an interface by index and returns a reference rather than a pointer to it. (5) Fix if_alloc() to fully initialize the if_addr_mtx before hooking up the ifnet to global lists. (6) Modify sysctls in if_mib.c to use ifnet_byindex_ref() and release the ifnet when done. When this change is MFC'd, it will need to replace if_ispare fields rather than adding new fields in order to avoid breaking the binary interface. Once this change is MFC'd, if_free_type() should be removed, as its 'type' argument is now optional. This refcount is not appropriate for counting mbuf pkthdr references, and also not for counting entry into the device driver via ifnet function pointers. An rmlock may be appropriate for the latter. Rather, this is about ensuring data structure stability when reaching an ifnet via global ifnet lists and tables followed by copy in or out of userspace. MFC after: 3 weeks Reported by: mdtancsa Reviewed by: brooks
2009-04-21 22:43:32 +00:00
void
if_ref(struct ifnet *ifp)
{
/* We don't assert the ifnet list lock here, but arguably should. */
refcount_acquire(&ifp->if_refcount);
}
void
if_rele(struct ifnet *ifp)
{
Add a new interface flag, IFF_DYING, which is set when a device driver calls if_free(), and remains set if the refcount is elevated. IF_DYING skips the bit in the if_flags bitmask previously used by IFF_NEEDSGIANT, so that an MFC can be done without changing which bit is used, as IFF_NEEDSGIANT is still present in 7.x. ifnet_byindex_ref() checks for IFF_DYING and returns NULL if it is set, preventing new references from by acquired by index, preventing monitoring sysctls from seeing it. Other lookup mechanisms currently do not check IFF_DYING, but may need to in the future. MFC after: 3 weeks
2009-04-23 09:32:30 +00:00
if (!refcount_release(&ifp->if_refcount))
return;
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
epoch_call(net_epoch_preempt, &ifp->if_epoch_ctx, if_destroy);
Start to address a number of races relating to use of ifnet pointers after the corresponding interface has been destroyed: (1) Add an ifnet refcount, ifp->if_refcount. Initialize it to 1 in if_alloc(), and modify if_free_type() to decrement and check the refcount. (2) Add new if_ref() and if_rele() interfaces to allow kernel code walking global interface lists to release IFNET_[RW]LOCK() yet keep the ifnet stable. Currently, if_rele() is a no-op wrapper around if_free(), but this may change in the future. (3) Add new ifnet field, if_alloctype, which caches the type passed to if_alloc(), but unlike if_type, won't be changed by drivers. This allows asynchronous free's of the interface after the driver has released it to still use the right type. Use that instead of the type passed to if_free_type(), but assert that they are the same (might have to rethink this if that doesn't work out). (4) Add a new ifnet_byindex_ref(), which looks up an interface by index and returns a reference rather than a pointer to it. (5) Fix if_alloc() to fully initialize the if_addr_mtx before hooking up the ifnet to global lists. (6) Modify sysctls in if_mib.c to use ifnet_byindex_ref() and release the ifnet when done. When this change is MFC'd, it will need to replace if_ispare fields rather than adding new fields in order to avoid breaking the binary interface. Once this change is MFC'd, if_free_type() should be removed, as its 'type' argument is now optional. This refcount is not appropriate for counting mbuf pkthdr references, and also not for counting entry into the device driver via ifnet function pointers. An rmlock may be appropriate for the latter. Rather, this is about ensuring data structure stability when reaching an ifnet via global ifnet lists and tables followed by copy in or out of userspace. MFC after: 3 weeks Reported by: mdtancsa Reviewed by: brooks
2009-04-21 22:43:32 +00:00
}
- bump __FreeBSD version to reflect added buf_ring, memory barriers, and ifnet functions - add memory barriers to <machine/atomic.h> - update drivers to only conditionally define their own - add lockless producer / consumer ring buffer - remove ring buffer implementation from cxgb and update its callers - add if_transmit(struct ifnet *ifp, struct mbuf *m) to ifnet to allow drivers to efficiently manage multiple hardware queues (i.e. not serialize all packets through one ifq) - expose if_qflush to allow drivers to flush any driver managed queues This work was supported by Bitgravity Inc. and Chelsio Inc.
2008-11-22 05:55:56 +00:00
void
r193336 moved ifq_detach to if_free which broke if_alloc followed by if_free (w/o doing if_attach); move ifq_attach to if_alloc and rename ifq_attach/detach to ifq_init/ifq_delete to better identify their purpose Reviewed by: jhb, kmacy
2009-06-15 19:50:03 +00:00
ifq_init(struct ifaltq *ifq, struct ifnet *ifp)
- bump __FreeBSD version to reflect added buf_ring, memory barriers, and ifnet functions - add memory barriers to <machine/atomic.h> - update drivers to only conditionally define their own - add lockless producer / consumer ring buffer - remove ring buffer implementation from cxgb and update its callers - add if_transmit(struct ifnet *ifp, struct mbuf *m) to ifnet to allow drivers to efficiently manage multiple hardware queues (i.e. not serialize all packets through one ifq) - expose if_qflush to allow drivers to flush any driver managed queues This work was supported by Bitgravity Inc. and Chelsio Inc.
2008-11-22 05:55:56 +00:00
{
mtx_init(&ifq->ifq_mtx, ifp->if_xname, "if send queue", MTX_DEF);
if (ifq->ifq_maxlen == 0)
ifq->ifq_maxlen = ifqmaxlen;
ifq->altq_type = 0;
ifq->altq_disc = NULL;
ifq->altq_flags &= ALTQF_CANTCHANGE;
ifq->altq_tbr = NULL;
ifq->altq_ifp = ifp;
}
void
r193336 moved ifq_detach to if_free which broke if_alloc followed by if_free (w/o doing if_attach); move ifq_attach to if_alloc and rename ifq_attach/detach to ifq_init/ifq_delete to better identify their purpose Reviewed by: jhb, kmacy
2009-06-15 19:50:03 +00:00
ifq_delete(struct ifaltq *ifq)
- bump __FreeBSD version to reflect added buf_ring, memory barriers, and ifnet functions - add memory barriers to <machine/atomic.h> - update drivers to only conditionally define their own - add lockless producer / consumer ring buffer - remove ring buffer implementation from cxgb and update its callers - add if_transmit(struct ifnet *ifp, struct mbuf *m) to ifnet to allow drivers to efficiently manage multiple hardware queues (i.e. not serialize all packets through one ifq) - expose if_qflush to allow drivers to flush any driver managed queues This work was supported by Bitgravity Inc. and Chelsio Inc.
2008-11-22 05:55:56 +00:00
{
mtx_destroy(&ifq->ifq_mtx);
}
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
/*
sys/net*: minor spelling fixes. No functional change.
2016-05-03 18:05:43 +00:00
* Perform generic interface initialization tasks and attach the interface
Introduce the if_vmove() function, which will be used in the future for reassigning ifnets from one vnet to another. if_vmove() works by calling a restricted subset of actions normally executed by if_detach() on an ifnet in the current vnet, and then switches to the target vnet and executes an appropriate subset of if_attach() actions there. if_attach() and if_detach() have become wrapper functions around if_attach_internal() and if_detach_internal(), where the later variants have an additional argument, a flag indicating whether a full attach or detach sequence is to be executed, or only a restricted subset suitable for moving an ifnet from one vnet to another. Hence, if_vmove() will not call if_detach() and if_attach() directly, but will call the if_detach_internal() and if_attach_internal() variants instead, with the vmove flag set. While here, staticize ifnet_setbyindex() since it is not referenced from outside of sys/net/if.c. Also rename ifccnt field in struct vimage to ifcnt, and do some minor whitespace garbage collection where appropriate. This change should have no functional impact on nooptions VIMAGE kernel builds. Reviewed by: bz, rwatson, brooks? Approved by: julian (mentor)
2009-05-22 22:09:00 +00:00
* to the list of "active" interfaces. If vmove flag is set on entry
* to if_attach_internal(), perform only a limited subset of initialization
* tasks, given that we are moving from one vnet to another an ifnet which
* has already been fully initialized.
Update the comments on if_alloc(), if_free(), if_free_type(), and if_attach. Remove a comment about pre-3.0 network drivers from if_attach(). Be a bit more consistant about whitespace near comments.
2007-05-16 19:59:01 +00:00
*
Fix group membership of cloned interfaces when one is moved by if_vmove(). In if_vmove(), if_detach_internal() and if_attach_internal() were called in series to detach and reattach the interface. When detaching, if_delgroup() was called and the interface leaves all of the group membership. And then upon attachment, if_addgroup(ifp, IFG_ALL) was called and it joined only "all" group again. This had a problem. Normally, a cloned interface automatically joins a group whose name is ifc_name of the cloner in addition to "all" upon creation. However, if_vmove() removed the membership and did not restore upon attachment. Differential Revision: https://reviews.freebsd.org/D1859
2015-03-02 20:00:03 +00:00
* Note that if_detach_internal() removes group membership unconditionally
* even when vmove flag is set, and if_attach_internal() adds only IFG_ALL.
* Thus, when if_vmove() is applied to a cloned interface, group membership
* is lost while a cloned one always joins a group whose name is
* ifc->ifc_name. To recover this after if_detach_internal() and
* if_attach_internal(), the cloner should be specified to
* if_attach_internal() via ifc. If it is non-NULL, if_attach_internal()
* attempts to join a group whose name is ifc->ifc_name.
*
Update the comments on if_alloc(), if_free(), if_free_type(), and if_attach. Remove a comment about pre-3.0 network drivers from if_attach(). Be a bit more consistant about whitespace near comments.
2007-05-16 19:59:01 +00:00
* XXX:
* - The decision to return void and thus require this function to
* succeed is questionable.
* - We should probably do more sanity checking. For instance we don't
* do anything to insure if_xname is unique or non-empty.
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
*/
void
Clean up whitespace, remove "register" keyword, ANSIfy. No functional changes.
2003-10-23 13:49:10 +00:00
if_attach(struct ifnet *ifp)
Introduce the if_vmove() function, which will be used in the future for reassigning ifnets from one vnet to another. if_vmove() works by calling a restricted subset of actions normally executed by if_detach() on an ifnet in the current vnet, and then switches to the target vnet and executes an appropriate subset of if_attach() actions there. if_attach() and if_detach() have become wrapper functions around if_attach_internal() and if_detach_internal(), where the later variants have an additional argument, a flag indicating whether a full attach or detach sequence is to be executed, or only a restricted subset suitable for moving an ifnet from one vnet to another. Hence, if_vmove() will not call if_detach() and if_attach() directly, but will call the if_detach_internal() and if_attach_internal() variants instead, with the vmove flag set. While here, staticize ifnet_setbyindex() since it is not referenced from outside of sys/net/if.c. Also rename ifccnt field in struct vimage to ifcnt, and do some minor whitespace garbage collection where appropriate. This change should have no functional impact on nooptions VIMAGE kernel builds. Reviewed by: bz, rwatson, brooks? Approved by: julian (mentor)
2009-05-22 22:09:00 +00:00
{
Fix group membership of cloned interfaces when one is moved by if_vmove(). In if_vmove(), if_detach_internal() and if_attach_internal() were called in series to detach and reattach the interface. When detaching, if_delgroup() was called and the interface leaves all of the group membership. And then upon attachment, if_addgroup(ifp, IFG_ALL) was called and it joined only "all" group again. This had a problem. Normally, a cloned interface automatically joins a group whose name is ifc_name of the cloner in addition to "all" upon creation. However, if_vmove() removed the membership and did not restore upon attachment. Differential Revision: https://reviews.freebsd.org/D1859
2015-03-02 20:00:03 +00:00
if_attach_internal(ifp, 0, NULL);
Introduce the if_vmove() function, which will be used in the future for reassigning ifnets from one vnet to another. if_vmove() works by calling a restricted subset of actions normally executed by if_detach() on an ifnet in the current vnet, and then switches to the target vnet and executes an appropriate subset of if_attach() actions there. if_attach() and if_detach() have become wrapper functions around if_attach_internal() and if_detach_internal(), where the later variants have an additional argument, a flag indicating whether a full attach or detach sequence is to be executed, or only a restricted subset suitable for moving an ifnet from one vnet to another. Hence, if_vmove() will not call if_detach() and if_attach() directly, but will call the if_detach_internal() and if_attach_internal() variants instead, with the vmove flag set. While here, staticize ifnet_setbyindex() since it is not referenced from outside of sys/net/if.c. Also rename ifccnt field in struct vimage to ifcnt, and do some minor whitespace garbage collection where appropriate. This change should have no functional impact on nooptions VIMAGE kernel builds. Reviewed by: bz, rwatson, brooks? Approved by: julian (mentor)
2009-05-22 22:09:00 +00:00
}
Improve transmit sending offload, TSO, algorithm in general. The current TSO limitation feature only takes the total number of bytes in an mbuf chain into account and does not limit by the number of mbufs in a chain. Some kinds of hardware is limited by two factors. One is the fragment length and the second is the fragment count. Both of these limits need to be taken into account when doing TSO. Else some kinds of hardware might have to drop completely valid mbuf chains because they cannot loaded into the given hardware's DMA engine. The new way of doing TSO limitation has been made backwards compatible as input from other FreeBSD developers and will use defaults for values not set. Reviewed by: adrian, rmacklem Sponsored by: Mellanox Technologies MFC after: 1 week
2014-09-22 08:27:27 +00:00
/*
* Compute the least common TSO limit.
*/
void
if_hw_tsomax_common(if_t ifp, struct ifnet_hw_tsomax *pmax)
{
/*
* 1) If there is no limit currently, take the limit from
* the network adapter.
*
* 2) If the network adapter has a limit below the current
* limit, apply it.
*/
if (pmax->tsomaxbytes == 0 || (ifp->if_hw_tsomax != 0 &&
ifp->if_hw_tsomax < pmax->tsomaxbytes)) {
pmax->tsomaxbytes = ifp->if_hw_tsomax;
}
if (pmax->tsomaxsegcount == 0 || (ifp->if_hw_tsomaxsegcount != 0 &&
ifp->if_hw_tsomaxsegcount < pmax->tsomaxsegcount)) {
pmax->tsomaxsegcount = ifp->if_hw_tsomaxsegcount;
}
if (pmax->tsomaxsegsize == 0 || (ifp->if_hw_tsomaxsegsize != 0 &&
ifp->if_hw_tsomaxsegsize < pmax->tsomaxsegsize)) {
pmax->tsomaxsegsize = ifp->if_hw_tsomaxsegsize;
}
}
/*
* Update TSO limit of a network adapter.
*
* Returns zero if no change. Else non-zero.
*/
int
if_hw_tsomax_update(if_t ifp, struct ifnet_hw_tsomax *pmax)
{
int retval = 0;
if (ifp->if_hw_tsomax != pmax->tsomaxbytes) {
ifp->if_hw_tsomax = pmax->tsomaxbytes;
retval++;
}
if (ifp->if_hw_tsomaxsegsize != pmax->tsomaxsegsize) {
ifp->if_hw_tsomaxsegsize = pmax->tsomaxsegsize;
retval++;
}
if (ifp->if_hw_tsomaxsegcount != pmax->tsomaxsegcount) {
ifp->if_hw_tsomaxsegcount = pmax->tsomaxsegcount;
retval++;
}
return (retval);
}
Introduce the if_vmove() function, which will be used in the future for reassigning ifnets from one vnet to another. if_vmove() works by calling a restricted subset of actions normally executed by if_detach() on an ifnet in the current vnet, and then switches to the target vnet and executes an appropriate subset of if_attach() actions there. if_attach() and if_detach() have become wrapper functions around if_attach_internal() and if_detach_internal(), where the later variants have an additional argument, a flag indicating whether a full attach or detach sequence is to be executed, or only a restricted subset suitable for moving an ifnet from one vnet to another. Hence, if_vmove() will not call if_detach() and if_attach() directly, but will call the if_detach_internal() and if_attach_internal() variants instead, with the vmove flag set. While here, staticize ifnet_setbyindex() since it is not referenced from outside of sys/net/if.c. Also rename ifccnt field in struct vimage to ifcnt, and do some minor whitespace garbage collection where appropriate. This change should have no functional impact on nooptions VIMAGE kernel builds. Reviewed by: bz, rwatson, brooks? Approved by: julian (mentor)
2009-05-22 22:09:00 +00:00
static void
Fix group membership of cloned interfaces when one is moved by if_vmove(). In if_vmove(), if_detach_internal() and if_attach_internal() were called in series to detach and reattach the interface. When detaching, if_delgroup() was called and the interface leaves all of the group membership. And then upon attachment, if_addgroup(ifp, IFG_ALL) was called and it joined only "all" group again. This had a problem. Normally, a cloned interface automatically joins a group whose name is ifc_name of the cloner in addition to "all" upon creation. However, if_vmove() removed the membership and did not restore upon attachment. Differential Revision: https://reviews.freebsd.org/D1859
2015-03-02 20:00:03 +00:00
if_attach_internal(struct ifnet *ifp, int vmove, struct if_clone *ifc)
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
{
unsigned socksize, ifasize;
Use new printf features rather than local kludges.
1996-01-24 21:12:23 +00:00
int namelen, masklen;
Clean up whitespace, remove "register" keyword, ANSIfy. No functional changes.
2003-10-23 13:49:10 +00:00
struct sockaddr_dl *sdl;
struct ifaddr *ifa;
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
Stop embedding struct ifnet at the top of driver softcs. Instead the struct ifnet or the layer 2 common structure it was embedded in have been replaced with a struct ifnet pointer to be filled by a call to the new function, if_alloc(). The layer 2 common structure is also allocated via if_alloc() based on the interface type. It is hung off the new struct ifnet member, if_l2com. This change removes the size of these structures from the kernel ABI and will allow us to better manage them as interfaces come and go. Other changes of note: - Struct arpcom is no longer referenced in normal interface code. Instead the Ethernet address is accessed via the IFP2ENADDR() macro. To enforce this ac_enaddr has been renamed to _ac_enaddr. - The second argument to ether_ifattach is now always the mac address from driver private storage rather than sometimes being ac_enaddr. Reviewed by: sobomax, sam
2005-06-10 16:49:24 +00:00
if (ifp->if_index == 0 || ifp != ifnet_byindex(ifp->if_index))
panic ("%s: BUG: if_attach called without if_alloc'd input()\n",
ifp->if_xname);
Permit buiding kernels with options VIMAGE, restricted to only a single active network stack instance. Turning on options VIMAGE at compile time yields the following changes relative to default kernel build: 1) V_ accessor macros for virtualized variables resolve to structure fields via base pointers, instead of being resolved as fields in global structs or plain global variables. As an example, V_ifnet becomes: options VIMAGE: ((struct vnet_net *) vnet_net)->_ifnet default build: vnet_net_0._ifnet options VIMAGE_GLOBALS: ifnet 2) INIT_VNET_* macros will declare and set up base pointers to be used by V_ accessor macros, instead of resolving to whitespace: INIT_VNET_NET(ifp->if_vnet); becomes struct vnet_net *vnet_net = (ifp->if_vnet)->mod_data[VNET_MOD_NET]; 3) Memory for vnet modules registered via vnet_mod_register() is now allocated at run time in sys/kern/kern_vimage.c, instead of per vnet module structs being declared as globals. If required, vnet modules can now request the framework to provide them with allocated bzeroed memory by filling in the vmi_size field in their vmi_modinfo structures. 4) structs socket, ifnet, inpcbinfo, tcpcb and syncache_head are extended to hold a pointer to the parent vnet. options VIMAGE builds will fill in those fields as required. 5) curvnet is introduced as a new global variable in options VIMAGE builds, always pointing to the default and only struct vnet. 6) struct sysctl_oid has been extended with additional two fields to store major and minor virtualization module identifiers, oid_v_subs and oid_v_mod. SYSCTL_V_* family of macros will fill in those fields accordingly, and store the offset in the appropriate vnet container struct in oid_arg1. In sysctl handlers dealing with virtualized sysctls, the SYSCTL_RESOLVE_V_ARG1() macro will compute the address of the target variable and make it available in arg1 variable for further processing. Unused fields in structs vnet_inet, vnet_inet6 and vnet_ipfw have been deleted. Reviewed by: bz, rwatson Approved by: julian (mentor)
2009-04-30 13:36:26 +00:00
#ifdef VIMAGE
ifp->if_vnet = curvnet;
Introduce an infrastructure for dismantling vnet instances. Vnet modules and protocol domains may now register destructor functions to clean up and release per-module state. The destructor mechanisms can be triggered by invoking "vimage -d", or a future equivalent command which will be provided via the new jail framework. While this patch introduces numerous placeholder destructor functions, many of those are currently incomplete, thus leaking memory or (even worse) failing to stop all running timers. Many of such issues are already known and will be incrementaly fixed over the next weeks in smaller incremental commits. Apart from introducing new fields in structs ifnet, domain, protosw and vnet_net, which requires the kernel and modules to be rebuilt, this change should have no impact on nooptions VIMAGE builds, since vnet destructors can only be called in VIMAGE kernels. Moreover, destructor functions should be in general compiled in only in options VIMAGE builds, except for kernel modules which can be safely kldunloaded at run time. Bump __FreeBSD_version to 800097. Reviewed by: bz, julian Approved by: rwatson, kib (re), julian (mentor)
2009-06-08 17:15:40 +00:00
if (ifp->if_home_vnet == NULL)
ifp->if_home_vnet = curvnet;
Permit buiding kernels with options VIMAGE, restricted to only a single active network stack instance. Turning on options VIMAGE at compile time yields the following changes relative to default kernel build: 1) V_ accessor macros for virtualized variables resolve to structure fields via base pointers, instead of being resolved as fields in global structs or plain global variables. As an example, V_ifnet becomes: options VIMAGE: ((struct vnet_net *) vnet_net)->_ifnet default build: vnet_net_0._ifnet options VIMAGE_GLOBALS: ifnet 2) INIT_VNET_* macros will declare and set up base pointers to be used by V_ accessor macros, instead of resolving to whitespace: INIT_VNET_NET(ifp->if_vnet); becomes struct vnet_net *vnet_net = (ifp->if_vnet)->mod_data[VNET_MOD_NET]; 3) Memory for vnet modules registered via vnet_mod_register() is now allocated at run time in sys/kern/kern_vimage.c, instead of per vnet module structs being declared as globals. If required, vnet modules can now request the framework to provide them with allocated bzeroed memory by filling in the vmi_size field in their vmi_modinfo structures. 4) structs socket, ifnet, inpcbinfo, tcpcb and syncache_head are extended to hold a pointer to the parent vnet. options VIMAGE builds will fill in those fields as required. 5) curvnet is introduced as a new global variable in options VIMAGE builds, always pointing to the default and only struct vnet. 6) struct sysctl_oid has been extended with additional two fields to store major and minor virtualization module identifiers, oid_v_subs and oid_v_mod. SYSCTL_V_* family of macros will fill in those fields accordingly, and store the offset in the appropriate vnet container struct in oid_arg1. In sysctl handlers dealing with virtualized sysctls, the SYSCTL_RESOLVE_V_ARG1() macro will compute the address of the target variable and make it available in arg1 variable for further processing. Unused fields in structs vnet_inet, vnet_inet6 and vnet_ipfw have been deleted. Reviewed by: bz, rwatson Approved by: julian (mentor)
2009-04-30 13:36:26 +00:00
#endif
Import interface groups from OpenBSD. This allows to group interfaces in order to - for example - apply firewall rules to a whole group of interfaces. This is required for importing pf from OpenBSD 3.9 Obtained from: OpenBSD (with changes) Discussed on: -net (back in April)
2006-06-19 22:20:45 +00:00
if_addgroup(ifp, IFG_ALL);
Fix group membership of cloned interfaces when one is moved by if_vmove(). In if_vmove(), if_detach_internal() and if_attach_internal() were called in series to detach and reattach the interface. When detaching, if_delgroup() was called and the interface leaves all of the group membership. And then upon attachment, if_addgroup(ifp, IFG_ALL) was called and it joined only "all" group again. This had a problem. Normally, a cloned interface automatically joins a group whose name is ifc_name of the cloner in addition to "all" upon creation. However, if_vmove() removed the membership and did not restore upon attachment. Differential Revision: https://reviews.freebsd.org/D1859
2015-03-02 20:00:03 +00:00
/* Restore group membership for cloned interfaces. */
if (vmove && ifc != NULL)
if_clone_addgroup(ifp, ifc);
Use getmicrotime() for if_lastchange, 10msec is plenty precision.
1998-04-06 11:43:12 +00:00
getmicrotime(&ifp->if_lastchange);
o Remove struct if_data from struct ifnet. Now it is merely API structure for route(4) socket and ifmib(4) sysctl. o Move fields from if_data to ifnet, but keep all statistic counters separate, since they should disappear later. o Provide function if_data_copy() to fill if_data, utilize it in routing socket and ifmib handler. o Provide overridable ifnet(9) method to fetch counters. If no provided, if_get_counters_compat() would be used, that returns old counters. Sponsored by: Netflix Sponsored by: Nginx, Inc.
2014-08-31 06:46:21 +00:00
ifp->if_epoch = time_uptime;
Move portions of data structure initialization from if_attach() to if_alloc(), and portions of data structure destruction from if_detach() to if_free(). These changes leave more of the struct ifnet in a safe-to-access condition between alloc and attach, and between detach and free, and focus on attach/detach as stack usage events rather than data structure initialization. Affected fields include the linkstate task queue, if_afdata lock, address lists, kqueue state, and MAC labels. ifq_attach() ifq_detach() are not moved as ifq_attach() may use a queue length set by the device driver between if_alloc() and if_attach(). MFC after: 3 weeks
2009-04-23 10:59:40 +00:00
export if_qflush for use by driver if_qflush routines only set ifp->if_{transmit, qflush} if not already set KASSERT that neither or both are set
2009-04-16 23:05:10 +00:00
KASSERT((ifp->if_transmit == NULL && ifp->if_qflush == NULL) ||
(ifp->if_transmit != NULL && ifp->if_qflush != NULL),
("transmit and qflush must both either be set or both be NULL"));
if (ifp->if_transmit == NULL) {
ifp->if_transmit = if_transmit;
ifp->if_qflush = if_qflush;
}
Add if_input_default() method, that will be used for if_input initialization, when no input method specified before if_attach(). This prevents panics when if_input() method called directly e.g. from bpf(4) code. PR: 192426 Reviewed by: glebius MFC after: 1 week
2015-03-12 14:55:33 +00:00
if (ifp->if_input == NULL)
ifp->if_input = if_input_default;
o Remove struct if_data from struct ifnet. Now it is merely API structure for route(4) socket and ifmib(4) sysctl. o Move fields from if_data to ifnet, but keep all statistic counters separate, since they should disappear later. o Provide function if_data_copy() to fill if_data, utilize it in routing socket and ifmib handler. o Provide overridable ifnet(9) method to fetch counters. If no provided, if_get_counters_compat() would be used, that returns old counters. Sponsored by: Netflix Sponsored by: Nginx, Inc.
2014-08-31 06:46:21 +00:00
Implement interface link header precomputation API. Add if_requestencap() interface method which is capable of calculating various link headers for given interface. Right now there is support for INET/INET6/ARP llheader calculation (IFENCAP_LL type request). Other types are planned to support more complex calculation (L2 multipath lagg nexthops, tunnel encap nexthops, etc..). Reshape 'struct route' to be able to pass additional data (with is length) to prepend to mbuf. These two changes permits routing code to pass pre-calculated nexthop data (like L2 header for route w/gateway) down to the stack eliminating the need for other lookups. It also brings us closer to more complex scenarios like transparently handling MPLS nexthops and tunnel interfaces. Last, but not least, it removes layering violation introduced by flowtable code (ro_lle) and simplifies handling of existing if_output consumers. ARP/ND changes: Make arp/ndp stack pre-calculate link header upon installing/updating lle record. Interface link address change are handled by re-calculating headers for all lles based on if_lladdr event. After these changes, arpresolve()/nd6_resolve() returns full pre-calculated header for supported interfaces thus simplifying if_output(). Move these lookups to separate ether_resolve_addr() function which ether returs error or fully-prepared link header. Add <arp|nd6_>resolve_addr() compat versions to return link addresses instead of pre-calculated data. BPF changes: Raw bpf writes occupied _two_ cases: AF_UNSPEC and pseudo_AF_HDRCMPLT. Despite the naming, both of there have ther header "complete". The only difference is that interface source mac has to be filled by OS for AF_UNSPEC (controlled via BIOCGHDRCMPLT). This logic has to stay inside BPF and not pollute if_output() routines. Convert BPF to pass prepend data via new 'struct route' mechanism. Note that it does not change non-optimized if_output(): ro_prepend handling is purely optional. Side note: hackish pseudo_AF_HDRCMPLT is supported for ethernet and FDDI. It is not needed for ethernet anymore. The only remaining FDDI user is dev/pdq mostly untouched since 2007. FDDI support was eliminated from OpenBSD in 2013 (sys/net/if_fddisubr.c rev 1.65). Flowtable changes: Flowtable violates layering by saving (and not correctly managing) rtes/lles. Instead of passing lle pointer, pass pointer to pre-calculated header data from that lle. Differential Revision: https://reviews.freebsd.org/D4102
2015-12-31 05:03:27 +00:00
if (ifp->if_requestencap == NULL)
ifp->if_requestencap = if_requestencap_default;
Introduce the if_vmove() function, which will be used in the future for reassigning ifnets from one vnet to another. if_vmove() works by calling a restricted subset of actions normally executed by if_detach() on an ifnet in the current vnet, and then switches to the target vnet and executes an appropriate subset of if_attach() actions there. if_attach() and if_detach() have become wrapper functions around if_attach_internal() and if_detach_internal(), where the later variants have an additional argument, a flag indicating whether a full attach or detach sequence is to be executed, or only a restricted subset suitable for moving an ifnet from one vnet to another. Hence, if_vmove() will not call if_detach() and if_attach() directly, but will call the if_detach_internal() and if_attach_internal() variants instead, with the vmove flag set. While here, staticize ifnet_setbyindex() since it is not referenced from outside of sys/net/if.c. Also rename ifccnt field in struct vimage to ifcnt, and do some minor whitespace garbage collection where appropriate. This change should have no functional impact on nooptions VIMAGE kernel builds. Reviewed by: bz, rwatson, brooks? Approved by: julian (mentor)
2009-05-22 22:09:00 +00:00
if (!vmove) {
Introduce support for Mandatory Access Control and extensible kernel access control. Instrument the interface management code so that MAC labels are properly maintained on network interfaces (struct ifnet). In particular, invoke entry points when interfaces are created and removed. MAC policies may initialized the label interface based on a variety of factors, including the interface name. Obtained from: TrustedBSD Project Sponsored by: DARPA, NAI Labs
2002-07-31 16:16:03 +00:00
#ifdef MAC
Introduce the if_vmove() function, which will be used in the future for reassigning ifnets from one vnet to another. if_vmove() works by calling a restricted subset of actions normally executed by if_detach() on an ifnet in the current vnet, and then switches to the target vnet and executes an appropriate subset of if_attach() actions there. if_attach() and if_detach() have become wrapper functions around if_attach_internal() and if_detach_internal(), where the later variants have an additional argument, a flag indicating whether a full attach or detach sequence is to be executed, or only a restricted subset suitable for moving an ifnet from one vnet to another. Hence, if_vmove() will not call if_detach() and if_attach() directly, but will call the if_detach_internal() and if_attach_internal() variants instead, with the vmove flag set. While here, staticize ifnet_setbyindex() since it is not referenced from outside of sys/net/if.c. Also rename ifccnt field in struct vimage to ifcnt, and do some minor whitespace garbage collection where appropriate. This change should have no functional impact on nooptions VIMAGE kernel builds. Reviewed by: bz, rwatson, brooks? Approved by: julian (mentor)
2009-05-22 22:09:00 +00:00
mac_ifnet_create(ifp);
Introduce support for Mandatory Access Control and extensible kernel access control. Instrument the interface management code so that MAC labels are properly maintained on network interfaces (struct ifnet). In particular, invoke entry points when interfaces are created and removed. MAC policies may initialized the label interface based on a variety of factors, including the interface name. Obtained from: TrustedBSD Project Sponsored by: DARPA, NAI Labs
2002-07-31 16:16:03 +00:00
#endif
Introduce the if_vmove() function, which will be used in the future for reassigning ifnets from one vnet to another. if_vmove() works by calling a restricted subset of actions normally executed by if_detach() on an ifnet in the current vnet, and then switches to the target vnet and executes an appropriate subset of if_attach() actions there. if_attach() and if_detach() have become wrapper functions around if_attach_internal() and if_detach_internal(), where the later variants have an additional argument, a flag indicating whether a full attach or detach sequence is to be executed, or only a restricted subset suitable for moving an ifnet from one vnet to another. Hence, if_vmove() will not call if_detach() and if_attach() directly, but will call the if_detach_internal() and if_attach_internal() variants instead, with the vmove flag set. While here, staticize ifnet_setbyindex() since it is not referenced from outside of sys/net/if.c. Also rename ifccnt field in struct vimage to ifcnt, and do some minor whitespace garbage collection where appropriate. This change should have no functional impact on nooptions VIMAGE kernel builds. Reviewed by: bz, rwatson, brooks? Approved by: julian (mentor)
2009-05-22 22:09:00 +00:00
/*
* Create a Link Level name for this device.
*/
namelen = strlen(ifp->if_xname);
/*
* Always save enough space for any possiable name so we
* can do a rename in place later.
*/
masklen = offsetof(struct sockaddr_dl, sdl_data[0]) + IFNAMSIZ;
socksize = masklen + ifp->if_addrlen;
if (socksize < sizeof(*sdl))
socksize = sizeof(*sdl);
socksize = roundup2(socksize, sizeof(long));
ifasize = sizeof(*ifa) + 2 * socksize;
Remove ifa_init() and provide ifa_alloc() that will allocate and setup struct ifaddr internally. Sponsored by: Netflix Sponsored by: Nginx, Inc.
2013-10-15 10:31:42 +00:00
ifa = ifa_alloc(ifasize, M_WAITOK);
Introduce the if_vmove() function, which will be used in the future for reassigning ifnets from one vnet to another. if_vmove() works by calling a restricted subset of actions normally executed by if_detach() on an ifnet in the current vnet, and then switches to the target vnet and executes an appropriate subset of if_attach() actions there. if_attach() and if_detach() have become wrapper functions around if_attach_internal() and if_detach_internal(), where the later variants have an additional argument, a flag indicating whether a full attach or detach sequence is to be executed, or only a restricted subset suitable for moving an ifnet from one vnet to another. Hence, if_vmove() will not call if_detach() and if_attach() directly, but will call the if_detach_internal() and if_attach_internal() variants instead, with the vmove flag set. While here, staticize ifnet_setbyindex() since it is not referenced from outside of sys/net/if.c. Also rename ifccnt field in struct vimage to ifcnt, and do some minor whitespace garbage collection where appropriate. This change should have no functional impact on nooptions VIMAGE kernel builds. Reviewed by: bz, rwatson, brooks? Approved by: julian (mentor)
2009-05-22 22:09:00 +00:00
sdl = (struct sockaddr_dl *)(ifa + 1);
sdl->sdl_len = socksize;
sdl->sdl_family = AF_LINK;
bcopy(ifp->if_xname, sdl->sdl_data, namelen);
sdl->sdl_nlen = namelen;
sdl->sdl_index = ifp->if_index;
sdl->sdl_type = ifp->if_type;
ifp->if_addr = ifa;
ifa->ifa_ifp = ifp;
ifa->ifa_addr = (struct sockaddr *)sdl;
sdl = (struct sockaddr_dl *)(socksize + (caddr_t)sdl);
ifa->ifa_netmask = (struct sockaddr *)sdl;
sdl->sdl_len = masklen;
while (namelen != 0)
sdl->sdl_data[--namelen] = 0xff;
ifnet: Replace if_addr_lock rwlock with epoch + mutex Run on LLNW canaries and tested by pho@ gallatin: Using a 14-core, 28-HTT single socket E5-2697 v3 with a 40GbE MLX5 based ConnectX 4-LX NIC, I see an almost 12% improvement in received packet rate, and a larger improvement in bytes delivered all the way to userspace. When the host receiving 64 streams of netperf -H $DUT -t UDP_STREAM -- -m 1, I see, using nstat -I mce0 1 before the patch: InMpps OMpps InGbs OGbs err TCP Est %CPU syscalls csw irq GBfree 4.98 0.00 4.42 0.00 4235592 33 83.80 4720653 2149771 1235 247.32 4.73 0.00 4.20 0.00 4025260 33 82.99 4724900 2139833 1204 247.32 4.72 0.00 4.20 0.00 4035252 33 82.14 4719162 2132023 1264 247.32 4.71 0.00 4.21 0.00 4073206 33 83.68 4744973 2123317 1347 247.32 4.72 0.00 4.21 0.00 4061118 33 80.82 4713615 2188091 1490 247.32 4.72 0.00 4.21 0.00 4051675 33 85.29 4727399 2109011 1205 247.32 4.73 0.00 4.21 0.00 4039056 33 84.65 4724735 2102603 1053 247.32 After the patch InMpps OMpps InGbs OGbs err TCP Est %CPU syscalls csw irq GBfree 5.43 0.00 4.20 0.00 3313143 33 84.96 5434214 1900162 2656 245.51 5.43 0.00 4.20 0.00 3308527 33 85.24 5439695 1809382 2521 245.51 5.42 0.00 4.19 0.00 3316778 33 87.54 5416028 1805835 2256 245.51 5.42 0.00 4.19 0.00 3317673 33 90.44 5426044 1763056 2332 245.51 5.42 0.00 4.19 0.00 3314839 33 88.11 5435732 1792218 2499 245.52 5.44 0.00 4.19 0.00 3293228 33 91.84 5426301 1668597 2121 245.52 Similarly, netperf reports 230Mb/s before the patch, and 270Mb/s after the patch Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15366
2018-05-18 20:13:34 +00:00
CK_STAILQ_INSERT_HEAD(&ifp->if_addrhead, ifa, ifa_link);
Introduce the if_vmove() function, which will be used in the future for reassigning ifnets from one vnet to another. if_vmove() works by calling a restricted subset of actions normally executed by if_detach() on an ifnet in the current vnet, and then switches to the target vnet and executes an appropriate subset of if_attach() actions there. if_attach() and if_detach() have become wrapper functions around if_attach_internal() and if_detach_internal(), where the later variants have an additional argument, a flag indicating whether a full attach or detach sequence is to be executed, or only a restricted subset suitable for moving an ifnet from one vnet to another. Hence, if_vmove() will not call if_detach() and if_attach() directly, but will call the if_detach_internal() and if_attach_internal() variants instead, with the vmove flag set. While here, staticize ifnet_setbyindex() since it is not referenced from outside of sys/net/if.c. Also rename ifccnt field in struct vimage to ifcnt, and do some minor whitespace garbage collection where appropriate. This change should have no functional impact on nooptions VIMAGE kernel builds. Reviewed by: bz, rwatson, brooks? Approved by: julian (mentor)
2009-05-22 22:09:00 +00:00
/* Reliably crash if used uninitialized. */
ifp->if_broadcastaddr = NULL;
Revert r271504. A new patch to solve this issue will be made. Suggested by: adrian @
2014-09-13 20:52:01 +00:00
Persistently store NIC's hardware MAC address, and add a way to retrive it The MAC address reported by `ifconfig ${nic} ether' does not always match the address in the hardware, as reported by the driver during attach. In particular, NICs which are components of a lagg(4) interface all report the same MAC. When attaching, the NIC driver passes the MAC address it read from the hardware as an argument to ether_ifattach(). Keep a second copy of it, and create ioctl(SIOCGHWADDR) to return it. Teach `ifconfig' to report it along with the active MAC address. PR: 194386 Reviewed by: glebius MFC after: 1 week Sponsored by: Panasas Differential Revision: https://reviews.freebsd.org/D10609
2017-05-10 22:13:47 +00:00
if (ifp->if_type == IFT_ETHER) {
ifp->if_hw_addr = malloc(ifp->if_addrlen, M_IFADDR,
M_WAITOK | M_ZERO);
}
Revert r271504. A new patch to solve this issue will be made. Suggested by: adrian @
2014-09-13 20:52:01 +00:00
#if defined(INET) || defined(INET6)
Improve transmit sending offload, TSO, algorithm in general. The current TSO limitation feature only takes the total number of bytes in an mbuf chain into account and does not limit by the number of mbufs in a chain. Some kinds of hardware is limited by two factors. One is the fragment length and the second is the fragment count. Both of these limits need to be taken into account when doing TSO. Else some kinds of hardware might have to drop completely valid mbuf chains because they cannot loaded into the given hardware's DMA engine. The new way of doing TSO limitation has been made backwards compatible as input from other FreeBSD developers and will use defaults for values not set. Reviewed by: adrian, rmacklem Sponsored by: Mellanox Technologies MFC after: 1 week
2014-09-22 08:27:27 +00:00
/* Use defaults for TSO, if nothing is set */
if (ifp->if_hw_tsomax == 0 &&
ifp->if_hw_tsomaxsegcount == 0 &&
ifp->if_hw_tsomaxsegsize == 0) {
/*
* The TSO defaults needs to be such that an
* NFS mbuf list of 35 mbufs totalling just
* below 64K works and that a chain of mbufs
* can be defragged into at most 32 segments:
*/
ifp->if_hw_tsomax = min(IP_MAXPACKET, (32 * MCLBYTES) -
Revert r271504. A new patch to solve this issue will be made. Suggested by: adrian @
2014-09-13 20:52:01 +00:00
(ETHER_HDR_LEN + ETHER_VLAN_ENCAP_LEN));
Improve transmit sending offload, TSO, algorithm in general. The current TSO limitation feature only takes the total number of bytes in an mbuf chain into account and does not limit by the number of mbufs in a chain. Some kinds of hardware is limited by two factors. One is the fragment length and the second is the fragment count. Both of these limits need to be taken into account when doing TSO. Else some kinds of hardware might have to drop completely valid mbuf chains because they cannot loaded into the given hardware's DMA engine. The new way of doing TSO limitation has been made backwards compatible as input from other FreeBSD developers and will use defaults for values not set. Reviewed by: adrian, rmacklem Sponsored by: Mellanox Technologies MFC after: 1 week
2014-09-22 08:27:27 +00:00
ifp->if_hw_tsomaxsegcount = 35;
ifp->if_hw_tsomaxsegsize = 2048; /* 2K */
/* XXX some drivers set IFCAP_TSO after ethernet attach */
if (ifp->if_capabilities & IFCAP_TSO) {
if_printf(ifp, "Using defaults for TSO: %u/%u/%u\n",
ifp->if_hw_tsomax,
ifp->if_hw_tsomaxsegcount,
ifp->if_hw_tsomaxsegsize);
}
}
Revert r271504. A new patch to solve this issue will be made. Suggested by: adrian @
2014-09-13 20:52:01 +00:00
#endif
Introduce the if_vmove() function, which will be used in the future for reassigning ifnets from one vnet to another. if_vmove() works by calling a restricted subset of actions normally executed by if_detach() on an ifnet in the current vnet, and then switches to the target vnet and executes an appropriate subset of if_attach() actions there. if_attach() and if_detach() have become wrapper functions around if_attach_internal() and if_detach_internal(), where the later variants have an additional argument, a flag indicating whether a full attach or detach sequence is to be executed, or only a restricted subset suitable for moving an ifnet from one vnet to another. Hence, if_vmove() will not call if_detach() and if_attach() directly, but will call the if_detach_internal() and if_attach_internal() variants instead, with the vmove flag set. While here, staticize ifnet_setbyindex() since it is not referenced from outside of sys/net/if.c. Also rename ifccnt field in struct vimage to ifcnt, and do some minor whitespace garbage collection where appropriate. This change should have no functional impact on nooptions VIMAGE kernel builds. Reviewed by: bz, rwatson, brooks? Approved by: julian (mentor)
2009-05-22 22:09:00 +00:00
}
When moving ifnets from one vnet to another, and the ifnet has ifaddresses of AF_LINK type which thus have an embedded if_index "backpointer", we must update that if_index backpointer to reflect the new if_index that our ifnet just got assigned. This change affects only options VIMAGE builds. Submitted by: bz Reviewed by: bz Approved by: re (rwatson), julian (mentor)
2009-08-24 10:14:09 +00:00
#ifdef VIMAGE
else {
/*
* Update the interface index in the link layer address
* of the interface.
*/
for (ifa = ifp->if_addr; ifa != NULL;
ifnet: Replace if_addr_lock rwlock with epoch + mutex Run on LLNW canaries and tested by pho@ gallatin: Using a 14-core, 28-HTT single socket E5-2697 v3 with a 40GbE MLX5 based ConnectX 4-LX NIC, I see an almost 12% improvement in received packet rate, and a larger improvement in bytes delivered all the way to userspace. When the host receiving 64 streams of netperf -H $DUT -t UDP_STREAM -- -m 1, I see, using nstat -I mce0 1 before the patch: InMpps OMpps InGbs OGbs err TCP Est %CPU syscalls csw irq GBfree 4.98 0.00 4.42 0.00 4235592 33 83.80 4720653 2149771 1235 247.32 4.73 0.00 4.20 0.00 4025260 33 82.99 4724900 2139833 1204 247.32 4.72 0.00 4.20 0.00 4035252 33 82.14 4719162 2132023 1264 247.32 4.71 0.00 4.21 0.00 4073206 33 83.68 4744973 2123317 1347 247.32 4.72 0.00 4.21 0.00 4061118 33 80.82 4713615 2188091 1490 247.32 4.72 0.00 4.21 0.00 4051675 33 85.29 4727399 2109011 1205 247.32 4.73 0.00 4.21 0.00 4039056 33 84.65 4724735 2102603 1053 247.32 After the patch InMpps OMpps InGbs OGbs err TCP Est %CPU syscalls csw irq GBfree 5.43 0.00 4.20 0.00 3313143 33 84.96 5434214 1900162 2656 245.51 5.43 0.00 4.20 0.00 3308527 33 85.24 5439695 1809382 2521 245.51 5.42 0.00 4.19 0.00 3316778 33 87.54 5416028 1805835 2256 245.51 5.42 0.00 4.19 0.00 3317673 33 90.44 5426044 1763056 2332 245.51 5.42 0.00 4.19 0.00 3314839 33 88.11 5435732 1792218 2499 245.52 5.44 0.00 4.19 0.00 3293228 33 91.84 5426301 1668597 2121 245.52 Similarly, netperf reports 230Mb/s before the patch, and 270Mb/s after the patch Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15366
2018-05-18 20:13:34 +00:00
ifa = CK_STAILQ_NEXT(ifa, ifa_link)) {
When moving ifnets from one vnet to another, and the ifnet has ifaddresses of AF_LINK type which thus have an embedded if_index "backpointer", we must update that if_index backpointer to reflect the new if_index that our ifnet just got assigned. This change affects only options VIMAGE builds. Submitted by: bz Reviewed by: bz Approved by: re (rwatson), julian (mentor)
2009-08-24 10:14:09 +00:00
if (ifa->ifa_addr->sa_family == AF_LINK) {
sdl = (struct sockaddr_dl *)ifa->ifa_addr;
sdl->sdl_index = ifp->if_index;
}
}
}
#endif
The if_check() function performed three actions: - verified that the ifp->if_snd.ifq_mtx was initalized for all attached interfaces. This was pointless because it was initalized for all interfaces in if_attach() so I've removed it. - Checked that ifp->if_snd.ifq_maxlen is initalized and set it to ifqmaxlen if unset. This makes more sense in if_attach() so I moved it there. - The first call of if_slowtimo(). Delete if_check() and call if_slowtimo() directly from the SYSINIT().
2008-05-17 03:38:13 +00:00
- First initialize ifnet, and then insert it into global list. - First remove from global list, then start destroying. PR: kern/97679 Submitted by: Alex Lyashkov <shadow itt.net.ru> Reviewed by: rwatson, brooks
2006-06-21 06:02:35 +00:00
IFNET_WLOCK();
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
CK_STAILQ_INSERT_TAIL(&V_ifnet, ifp, if_link);
Change the curvnet variable from a global const struct vnet *, previously always pointing to the default vnet context, to a dynamically changing thread-local one. The currvnet context should be set on entry to networking code via CURVNET_SET() macros, and reverted to previous state via CURVNET_RESTORE(). Recursions on curvnet are permitted, though strongly discuouraged. This change should have no functional impact on nooptions VIMAGE kernel builds, where CURVNET_* macros expand to whitespace. The curthread->td_vnet (aka curvnet) variable's purpose is to be an indicator of the vnet context in which the current network-related operation takes place, in case we cannot deduce the current vnet context from any other source, such as by looking at mbuf's m->m_pkthdr.rcvif->if_vnet, sockets's so->so_vnet etc. Moreover, so far curvnet has turned out to be an invaluable consistency checking aid: it helps to catch cases when sockets, ifnets or any other vnet-aware structures may have leaked from one vnet to another. The exact placement of the CURVNET_SET() / CURVNET_RESTORE() macros was a result of an empirical iterative process, whith an aim to reduce recursions on CURVNET_SET() to a minimum, while still reducing the scope of CURVNET_SET() to networking only operations - the alternative would be calling CURVNET_SET() on each system call entry. In general, curvnet has to be set in three typicall cases: when processing socket-related requests from userspace or from within the kernel; when processing inbound traffic flowing from device drivers to upper layers of the networking stack, and when executing timer-driven networking functions. This change also introduces a DDB subcommand to show the list of all vnet instances. Approved by: julian (mentor)
2009-05-05 10:56:12 +00:00
#ifdef VIMAGE
Normalize field naming for struct vnet, fix two debugging printfs that print them. Reviewed by: bz Approved by: re (kensmith, kib)
2009-07-19 17:40:45 +00:00
curvnet->vnet_ifcnt++;
Change the curvnet variable from a global const struct vnet *, previously always pointing to the default vnet context, to a dynamically changing thread-local one. The currvnet context should be set on entry to networking code via CURVNET_SET() macros, and reverted to previous state via CURVNET_RESTORE(). Recursions on curvnet are permitted, though strongly discuouraged. This change should have no functional impact on nooptions VIMAGE kernel builds, where CURVNET_* macros expand to whitespace. The curthread->td_vnet (aka curvnet) variable's purpose is to be an indicator of the vnet context in which the current network-related operation takes place, in case we cannot deduce the current vnet context from any other source, such as by looking at mbuf's m->m_pkthdr.rcvif->if_vnet, sockets's so->so_vnet etc. Moreover, so far curvnet has turned out to be an invaluable consistency checking aid: it helps to catch cases when sockets, ifnets or any other vnet-aware structures may have leaked from one vnet to another. The exact placement of the CURVNET_SET() / CURVNET_RESTORE() macros was a result of an empirical iterative process, whith an aim to reduce recursions on CURVNET_SET() to a minimum, while still reducing the scope of CURVNET_SET() to networking only operations - the alternative would be calling CURVNET_SET() on each system call entry. In general, curvnet has to be set in three typicall cases: when processing socket-related requests from userspace or from within the kernel; when processing inbound traffic flowing from device drivers to upper layers of the networking stack, and when executing timer-driven networking functions. This change also introduces a DDB subcommand to show the list of all vnet instances. Approved by: julian (mentor)
2009-05-05 10:56:12 +00:00
#endif
- First initialize ifnet, and then insert it into global list. - First remove from global list, then start destroying. PR: kern/97679 Submitted by: Alex Lyashkov <shadow itt.net.ru> Reviewed by: rwatson, brooks
2006-06-21 06:02:35 +00:00
IFNET_WUNLOCK();
Implement the check I was talking about in the previous message already. Introduce domain_init_status to keep track of the init status of the domains list (surprise). 0 = uninitialized, 1 = initialized/unpopulated, 2 = initialized/done. Higher values can be used to support late addition of domains which right now "works", but is potential dangerous. I choose to only give a warning when doing so. Use domain_init_status with if_attachdomain[1]() to ensure that we have a complete domains list when we init the if_afdata array. Store the current value of domain_init_status in if_afdata_initialized. This way we can update if_afdata after a new protocol has been added (once that is allowed). Submitted by: se (with changes) Reviewed by: julian, glebius, se PR: kern/73321 (partly)
2004-11-30 22:38:37 +00:00
if (domain_init_status >= 2)
- add dom_if{attach,detach} framework. - transition to use ifp->if_afdata. Obtained from: KAME
2003-10-17 15:46:31 +00:00
if_attachdomain1(ifp);
Bring eventhandler callbacks for pf. This enables pf to track dynamic address changes on interfaces (dailup) with the "on (<ifname>)"-syntax. This also brings hooks in anticipation of tracking cloned interfaces, which will be in future versions of pf. Approved by: bms(mentor)
2004-02-26 04:27:55 +00:00
EVENTHANDLER_INVOKE(ifnet_arrival_event, ifp);
Change the curvnet variable from a global const struct vnet *, previously always pointing to the default vnet context, to a dynamically changing thread-local one. The currvnet context should be set on entry to networking code via CURVNET_SET() macros, and reverted to previous state via CURVNET_RESTORE(). Recursions on curvnet are permitted, though strongly discuouraged. This change should have no functional impact on nooptions VIMAGE kernel builds, where CURVNET_* macros expand to whitespace. The curthread->td_vnet (aka curvnet) variable's purpose is to be an indicator of the vnet context in which the current network-related operation takes place, in case we cannot deduce the current vnet context from any other source, such as by looking at mbuf's m->m_pkthdr.rcvif->if_vnet, sockets's so->so_vnet etc. Moreover, so far curvnet has turned out to be an invaluable consistency checking aid: it helps to catch cases when sockets, ifnets or any other vnet-aware structures may have leaked from one vnet to another. The exact placement of the CURVNET_SET() / CURVNET_RESTORE() macros was a result of an empirical iterative process, whith an aim to reduce recursions on CURVNET_SET() to a minimum, while still reducing the scope of CURVNET_SET() to networking only operations - the alternative would be calling CURVNET_SET() on each system call entry. In general, curvnet has to be set in three typicall cases: when processing socket-related requests from userspace or from within the kernel; when processing inbound traffic flowing from device drivers to upper layers of the networking stack, and when executing timer-driven networking functions. This change also introduces a DDB subcommand to show the list of all vnet instances. Approved by: julian (mentor)
2009-05-05 10:56:12 +00:00
if (IS_DEFAULT_VNET(curvnet))
devctl_notify("IFNET", ifp->if_xname, "ATTACH", NULL);
Bring eventhandler callbacks for pf. This enables pf to track dynamic address changes on interfaces (dailup) with the "on (<ifname>)"-syntax. This also brings hooks in anticipation of tracking cloned interfaces, which will be in future versions of pf. Approved by: bms(mentor)
2004-02-26 04:27:55 +00:00
Introduce an interface announcement message for the routing socket so that routing daemons and other interested parties know when an interface is attached/detached. PR: kern/33747 Obtained from: NetBSD MFC after: 2 weeks
2002-01-18 14:33:04 +00:00
/* Announce the interface. */
rt_ifannouncemsg(ifp, IFAN_ARRIVAL);
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
}
Bring the 'new-bus' to the i386. This extensively changes the way the i386 platform boots, it is no longer ISA-centric, and is fully dynamic. Most old drivers compile and run without modification via 'compatability shims' to enable a smoother transition. eisa, isapnp and pccard* are not yet using the new resource manager. Once fully converted, all drivers will be loadable, including PCI and ISA. (Some other changes appear to have snuck in, including a port of Soren's ATA driver to the Alpha. Soren, back this out if you need to.) This is a checkpoint of work-in-progress, but is quite functional. The bulk of the work was done over the last few years by Doug Rabson and Garrett Wollman. Approved by: core
1999-04-16 21:22:55 +00:00
- add dom_if{attach,detach} framework. - transition to use ifp->if_afdata. Obtained from: KAME
2003-10-17 15:46:31 +00:00
static void
epoch(9): allocate net epochs earlier in boot
2018-05-18 18:48:00 +00:00
if_epochalloc(void *dummy __unused)
- add dom_if{attach,detach} framework. - transition to use ifp->if_afdata. Obtained from: KAME
2003-10-17 15:46:31 +00:00
{
Add debugging facility EPOCH_TRACE that checks that epochs entered are properly nested and warns about recursive entrances. Unlike with locks, there is nothing fundamentally wrong with such use, the intent of tracer is to help to review complex epoch-protected code paths, and we mean the network stack here. Reviewed by: hselasky Sponsored by: Netflix Pull Request: https://reviews.freebsd.org/D21610
2019-09-25 18:26:31 +00:00
net_epoch_preempt = epoch_alloc("Net preemptible", EPOCH_PREEMPT);
net_epoch = epoch_alloc("Net", 0);
epoch(9): allocate net epochs earlier in boot
2018-05-18 18:48:00 +00:00
}
SYSINIT(ifepochalloc, SI_SUB_TASKQ + 1, SI_ORDER_ANY,
if_epochalloc, NULL);
static void
if_attachdomain(void *dummy)
{
struct ifnet *ifp;
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
CK_STAILQ_FOREACH(ifp, &V_ifnet, if_link)
- add dom_if{attach,detach} framework. - transition to use ifp->if_afdata. Obtained from: KAME
2003-10-17 15:46:31 +00:00
if_attachdomain1(ifp);
}
Implement the check I was talking about in the previous message already. Introduce domain_init_status to keep track of the init status of the domains list (surprise). 0 = uninitialized, 1 = initialized/unpopulated, 2 = initialized/done. Higher values can be used to support late addition of domains which right now "works", but is potential dangerous. I choose to only give a warning when doing so. Use domain_init_status with if_attachdomain[1]() to ensure that we have a complete domains list when we init the if_afdata array. Store the current value of domain_init_status in if_afdata_initialized. This way we can update if_afdata after a new protocol has been added (once that is allowed). Submitted by: se (with changes) Reviewed by: julian, glebius, se PR: kern/73321 (partly)
2004-11-30 22:38:37 +00:00
SYSINIT(domainifattach, SI_SUB_PROTO_IFATTACHDOMAIN, SI_ORDER_SECOND,
- add dom_if{attach,detach} framework. - transition to use ifp->if_afdata. Obtained from: KAME
2003-10-17 15:46:31 +00:00
if_attachdomain, NULL);
static void
Clean up whitespace, remove "register" keyword, ANSIfy. No functional changes.
2003-10-23 13:49:10 +00:00
if_attachdomain1(struct ifnet *ifp)
- add dom_if{attach,detach} framework. - transition to use ifp->if_afdata. Obtained from: KAME
2003-10-17 15:46:31 +00:00
{
struct domain *dp;
Since dp->dom_ifattach calls malloc() with M_WAITOK, we cannot use mutex lock directly here. Protect ifp->if_afdata instead. Reported by: grehan
2003-10-24 16:57:59 +00:00
/*
* Since dp->dom_ifattach calls malloc() with M_WAITOK, we
* cannot lock ifp->if_afdata initialization, entirely.
*/
In if_attachdomain1() there does not seem to be any reason to use TRYLOCK rather than just acquire the lock, so just do that. Reviewed by: markj Obtained from: projects/vnet MFC after: 2 weeks Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D6578
2016-05-28 08:32:15 +00:00
IF_AFDATA_LOCK(ifp);
Implement the check I was talking about in the previous message already. Introduce domain_init_status to keep track of the init status of the domains list (surprise). 0 = uninitialized, 1 = initialized/unpopulated, 2 = initialized/done. Higher values can be used to support late addition of domains which right now "works", but is potential dangerous. I choose to only give a warning when doing so. Use domain_init_status with if_attachdomain[1]() to ensure that we have a complete domains list when we init the if_afdata array. Store the current value of domain_init_status in if_afdata_initialized. This way we can update if_afdata after a new protocol has been added (once that is allowed). Submitted by: se (with changes) Reviewed by: julian, glebius, se PR: kern/73321 (partly)
2004-11-30 22:38:37 +00:00
if (ifp->if_afdata_initialized >= domain_init_status) {
Since dp->dom_ifattach calls malloc() with M_WAITOK, we cannot use mutex lock directly here. Protect ifp->if_afdata instead. Reported by: grehan
2003-10-24 16:57:59 +00:00
IF_AFDATA_UNLOCK(ifp);
Update to previous r241688 to use __func__ instead of spelled out function name in log(9) message. Suggested by: glebius
2012-10-19 10:07:55 +00:00
log(LOG_WARNING, "%s called more than once on %s\n",
__func__, ifp->if_xname);
Since dp->dom_ifattach calls malloc() with M_WAITOK, we cannot use mutex lock directly here. Protect ifp->if_afdata instead. Reported by: grehan
2003-10-24 16:57:59 +00:00
return;
}
Implement the check I was talking about in the previous message already. Introduce domain_init_status to keep track of the init status of the domains list (surprise). 0 = uninitialized, 1 = initialized/unpopulated, 2 = initialized/done. Higher values can be used to support late addition of domains which right now "works", but is potential dangerous. I choose to only give a warning when doing so. Use domain_init_status with if_attachdomain[1]() to ensure that we have a complete domains list when we init the if_afdata array. Store the current value of domain_init_status in if_afdata_initialized. This way we can update if_afdata after a new protocol has been added (once that is allowed). Submitted by: se (with changes) Reviewed by: julian, glebius, se PR: kern/73321 (partly)
2004-11-30 22:38:37 +00:00
ifp->if_afdata_initialized = domain_init_status;
Since dp->dom_ifattach calls malloc() with M_WAITOK, we cannot use mutex lock directly here. Protect ifp->if_afdata instead. Reported by: grehan
2003-10-24 16:57:59 +00:00
IF_AFDATA_UNLOCK(ifp);
- add dom_if{attach,detach} framework. - transition to use ifp->if_afdata. Obtained from: KAME
2003-10-17 15:46:31 +00:00
/* address family dependent data region */
bzero(ifp->if_afdata, sizeof(ifp->if_afdata));
for (dp = domains; dp; dp = dp->dom_next) {
if (dp->dom_ifattach)
ifp->if_afdata[dp->dom_family] =
(*dp->dom_ifattach)(ifp);
}
}
Separate out address-detaching part of if_detach into if_purgeaddrs, so if_tap doesn't need to rely on locally-rolled code to do same. The observable symptom of if_tap's bzero'ing the address details was a crash in "ifconfig tap0" after an if_tap device was closed. Reported By: Matti Saarinen (mjsaarin at cc dot helsinki dot fi)
2005-05-25 13:52:03 +00:00
/*
Implement reference counting for ifmultiaddr, in_multi, and in6_multi structures. Detect when ifnet instances are detached from the network stack and perform appropriate cleanup to prevent memory leaks. This has been implemented in such a way as to be backwards ABI compatible. Kernel consumers are changed to use if_delmulti_ifma(); in_delmulti() is unable to detect interface removal by design, as it performs searches on structures which are removed with the interface. With this architectural change, the panics FreeBSD users have experienced with carp and pfsync should be resolved. Obtained from: p4 branch bms_netdev Reviewed by: andre Sponsored by: Garance A Drosehn Idea from: NetBSD MFC after: 1 month
2007-03-20 00:36:10 +00:00
* Remove any unicast or broadcast network addresses from an interface.
Separate out address-detaching part of if_detach into if_purgeaddrs, so if_tap doesn't need to rely on locally-rolled code to do same. The observable symptom of if_tap's bzero'ing the address details was a crash in "ifconfig tap0" after an if_tap device was closed. Reported By: Matti Saarinen (mjsaarin at cc dot helsinki dot fi)
2005-05-25 13:52:03 +00:00
*/
void
if_purgeaddrs(struct ifnet *ifp)
{
Resolve deadlock between epoch(9) and various network interface SX-locks, during if_purgeaddrs(), by not allowing to hold the epoch read lock over typical network IOCTL code paths. This is a regression issue after r334305. Reviewed by: ae (network) Differential revision: https://reviews.freebsd.org/D17647 MFC after: 1 week Sponsored by: Mellanox Technologies
2018-10-22 13:25:26 +00:00
struct ifaddr *ifa;
Separate out address-detaching part of if_detach into if_purgeaddrs, so if_tap doesn't need to rely on locally-rolled code to do same. The observable symptom of if_tap's bzero'ing the address details was a crash in "ifconfig tap0" after an if_tap device was closed. Reported By: Matti Saarinen (mjsaarin at cc dot helsinki dot fi)
2005-05-25 13:52:03 +00:00
Resolve deadlock between epoch(9) and various network interface SX-locks, during if_purgeaddrs(), by not allowing to hold the epoch read lock over typical network IOCTL code paths. This is a regression issue after r334305. Reviewed by: ae (network) Differential revision: https://reviews.freebsd.org/D17647 MFC after: 1 week Sponsored by: Mellanox Technologies
2018-10-22 13:25:26 +00:00
while (1) {
Mechanical cleanup of epoch(9) usage in network stack. - Remove macros that covertly create epoch_tracker on thread stack. Such macros a quite unsafe, e.g. will produce a buggy code if same macro is used in embedded scopes. Explicitly declare epoch_tracker always. - Unmask interface list IFNET_RLOCK_NOSLEEP(), interface address list IF_ADDR_RLOCK() and interface AF specific data IF_AFDATA_RLOCK() read locking macros to what they actually are - the net_epoch. Keeping them as is is very misleading. They all are named FOO_RLOCK(), while they no longer have lock semantics. Now they allow recursion and what's more important they now no longer guarantee protection against their companion WLOCK macros. Note: INP_HASH_RLOCK() has same problems, but not touched by this commit. This is non functional mechanical change. The only functionally changed functions are ni6_addrs() and ni6_store_addrs(), where we no longer enter epoch recursively. Discussed with: jtl, gallatin
2019-01-09 01:11:19 +00:00
struct epoch_tracker et;
NET_EPOCH_ENTER(et);
Resolve deadlock between epoch(9) and various network interface SX-locks, during if_purgeaddrs(), by not allowing to hold the epoch read lock over typical network IOCTL code paths. This is a regression issue after r334305. Reviewed by: ae (network) Differential revision: https://reviews.freebsd.org/D17647 MFC after: 1 week Sponsored by: Mellanox Technologies
2018-10-22 13:25:26 +00:00
CK_STAILQ_FOREACH(ifa, &ifp->if_addrhead, ifa_link) {
if (ifa->ifa_addr->sa_family != AF_LINK)
break;
}
Mechanical cleanup of epoch(9) usage in network stack. - Remove macros that covertly create epoch_tracker on thread stack. Such macros a quite unsafe, e.g. will produce a buggy code if same macro is used in embedded scopes. Explicitly declare epoch_tracker always. - Unmask interface list IFNET_RLOCK_NOSLEEP(), interface address list IF_ADDR_RLOCK() and interface AF specific data IF_AFDATA_RLOCK() read locking macros to what they actually are - the net_epoch. Keeping them as is is very misleading. They all are named FOO_RLOCK(), while they no longer have lock semantics. Now they allow recursion and what's more important they now no longer guarantee protection against their companion WLOCK macros. Note: INP_HASH_RLOCK() has same problems, but not touched by this commit. This is non functional mechanical change. The only functionally changed functions are ni6_addrs() and ni6_store_addrs(), where we no longer enter epoch recursively. Discussed with: jtl, gallatin
2019-01-09 01:11:19 +00:00
NET_EPOCH_EXIT(et);
Resolve deadlock between epoch(9) and various network interface SX-locks, during if_purgeaddrs(), by not allowing to hold the epoch read lock over typical network IOCTL code paths. This is a regression issue after r334305. Reviewed by: ae (network) Differential revision: https://reviews.freebsd.org/D17647 MFC after: 1 week Sponsored by: Mellanox Technologies
2018-10-22 13:25:26 +00:00
if (ifa == NULL)
break;
Separate out address-detaching part of if_detach into if_purgeaddrs, so if_tap doesn't need to rely on locally-rolled code to do same. The observable symptom of if_tap's bzero'ing the address details was a crash in "ifconfig tap0" after an if_tap device was closed. Reported By: Matti Saarinen (mjsaarin at cc dot helsinki dot fi)
2005-05-25 13:52:03 +00:00
#ifdef INET
/* XXX: Ugly!! ad hoc just for INET */
There is a consensus that ifaddr.ifa_addr should never be NULL, except in places dealing with ifaddr creation or destruction; and in such special places incomplete ifaddrs should never be linked to system-wide data structures. Therefore we can eliminate all the superfluous checks for "ifa->ifa_addr != NULL" and get ready to the system crashing honestly instead of masking possible bugs. Suggested by: glebius, jhb, ru
2006-06-29 19:22:05 +00:00
if (ifa->ifa_addr->sa_family == AF_INET) {
Separate out address-detaching part of if_detach into if_purgeaddrs, so if_tap doesn't need to rely on locally-rolled code to do same. The observable symptom of if_tap's bzero'ing the address details was a crash in "ifconfig tap0" after an if_tap device was closed. Reported By: Matti Saarinen (mjsaarin at cc dot helsinki dot fi)
2005-05-25 13:52:03 +00:00
struct ifaliasreq ifr;
bzero(&ifr, sizeof(ifr));
ifr.ifra_addr = *ifa->ifa_addr;
if (ifa->ifa_dstaddr)
ifr.ifra_broadaddr = *ifa->ifa_dstaddr;
if (in_control(NULL, SIOCDIFADDR, (caddr_t)&ifr, ifp,
NULL) == 0)
continue;
}
#endif /* INET */
#ifdef INET6
There is a consensus that ifaddr.ifa_addr should never be NULL, except in places dealing with ifaddr creation or destruction; and in such special places incomplete ifaddrs should never be linked to system-wide data structures. Therefore we can eliminate all the superfluous checks for "ifa->ifa_addr != NULL" and get ready to the system crashing honestly instead of masking possible bugs. Suggested by: glebius, jhb, ru
2006-06-29 19:22:05 +00:00
if (ifa->ifa_addr->sa_family == AF_INET6) {
Separate out address-detaching part of if_detach into if_purgeaddrs, so if_tap doesn't need to rely on locally-rolled code to do same. The observable symptom of if_tap's bzero'ing the address details was a crash in "ifconfig tap0" after an if_tap device was closed. Reported By: Matti Saarinen (mjsaarin at cc dot helsinki dot fi)
2005-05-25 13:52:03 +00:00
in6_purgeaddr(ifa);
/* ifp_addrhead is already updated */
continue;
}
#endif /* INET6 */
In if_purgeaddrs() we cannot hold the lock over the entire loop due to called functions (as in other parts of the stack, leave a comment). Put around a lock the removal of the ifa from the list however to reduce the possible race with other places. Obtained from: projects/vnet MFC after: 2 weeks Sponsored by: The FreeBSD Foundation
2016-06-06 13:17:25 +00:00
IF_ADDR_WLOCK(ifp);
ifnet: Replace if_addr_lock rwlock with epoch + mutex Run on LLNW canaries and tested by pho@ gallatin: Using a 14-core, 28-HTT single socket E5-2697 v3 with a 40GbE MLX5 based ConnectX 4-LX NIC, I see an almost 12% improvement in received packet rate, and a larger improvement in bytes delivered all the way to userspace. When the host receiving 64 streams of netperf -H $DUT -t UDP_STREAM -- -m 1, I see, using nstat -I mce0 1 before the patch: InMpps OMpps InGbs OGbs err TCP Est %CPU syscalls csw irq GBfree 4.98 0.00 4.42 0.00 4235592 33 83.80 4720653 2149771 1235 247.32 4.73 0.00 4.20 0.00 4025260 33 82.99 4724900 2139833 1204 247.32 4.72 0.00 4.20 0.00 4035252 33 82.14 4719162 2132023 1264 247.32 4.71 0.00 4.21 0.00 4073206 33 83.68 4744973 2123317 1347 247.32 4.72 0.00 4.21 0.00 4061118 33 80.82 4713615 2188091 1490 247.32 4.72 0.00 4.21 0.00 4051675 33 85.29 4727399 2109011 1205 247.32 4.73 0.00 4.21 0.00 4039056 33 84.65 4724735 2102603 1053 247.32 After the patch InMpps OMpps InGbs OGbs err TCP Est %CPU syscalls csw irq GBfree 5.43 0.00 4.20 0.00 3313143 33 84.96 5434214 1900162 2656 245.51 5.43 0.00 4.20 0.00 3308527 33 85.24 5439695 1809382 2521 245.51 5.42 0.00 4.19 0.00 3316778 33 87.54 5416028 1805835 2256 245.51 5.42 0.00 4.19 0.00 3317673 33 90.44 5426044 1763056 2332 245.51 5.42 0.00 4.19 0.00 3314839 33 88.11 5435732 1792218 2499 245.52 5.44 0.00 4.19 0.00 3293228 33 91.84 5426301 1668597 2121 245.52 Similarly, netperf reports 230Mb/s before the patch, and 270Mb/s after the patch Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15366
2018-05-18 20:13:34 +00:00
CK_STAILQ_REMOVE(&ifp->if_addrhead, ifa, ifaddr, ifa_link);
In if_purgeaddrs() we cannot hold the lock over the entire loop due to called functions (as in other parts of the stack, leave a comment). Put around a lock the removal of the ifa from the list however to reduce the possible race with other places. Obtained from: projects/vnet MFC after: 2 weeks Sponsored by: The FreeBSD Foundation
2016-06-06 13:17:25 +00:00
IF_ADDR_WUNLOCK(ifp);
Clean up common ifaddr management: - Unify reference count and lock initialization in a single function, ifa_init(). - Move tear-down from a macro (IFAFREE) to a function ifa_free(). - Move reference count bump from a macro (IFAREF) to a function ifa_ref(). - Instead of using a u_int protected by a mutex to refcount(9) for reference count management. The ifa_mtx is now used for exactly one ioctl, and possibly should be removed. MFC after: 3 weeks
2009-06-21 19:30:33 +00:00
ifa_free(ifa);
Separate out address-detaching part of if_detach into if_purgeaddrs, so if_tap doesn't need to rely on locally-rolled code to do same. The observable symptom of if_tap's bzero'ing the address details was a crash in "ifconfig tap0" after an if_tap device was closed. Reported By: Matti Saarinen (mjsaarin at cc dot helsinki dot fi)
2005-05-25 13:52:03 +00:00
}
}
Implement reference counting for ifmultiaddr, in_multi, and in6_multi structures. Detect when ifnet instances are detached from the network stack and perform appropriate cleanup to prevent memory leaks. This has been implemented in such a way as to be backwards ABI compatible. Kernel consumers are changed to use if_delmulti_ifma(); in_delmulti() is unable to detect interface removal by design, as it performs searches on structures which are removed with the interface. With this architectural change, the panics FreeBSD users have experienced with carp and pfsync should be resolved. Obtained from: p4 branch bms_netdev Reviewed by: andre Sponsored by: Garance A Drosehn Idea from: NetBSD MFC after: 1 month
2007-03-20 00:36:10 +00:00
/*
While flushing the multicast filter of an interface, do not zero the relevant ifmultiaddr structures' reference to the parent interface, unless the parent interface is really detaching. While here, program only link layer multicast filters to a wlan's hardware parent interface. PR: kern/142391, kern/142392 Reviewed by: sam, rpaolo, bms MFC after: 1 week
2010-01-24 16:17:58 +00:00
* Remove any multicast network addresses from an interface when an ifnet
* is going away.
Implement reference counting for ifmultiaddr, in_multi, and in6_multi structures. Detect when ifnet instances are detached from the network stack and perform appropriate cleanup to prevent memory leaks. This has been implemented in such a way as to be backwards ABI compatible. Kernel consumers are changed to use if_delmulti_ifma(); in_delmulti() is unable to detect interface removal by design, as it performs searches on structures which are removed with the interface. With this architectural change, the panics FreeBSD users have experienced with carp and pfsync should be resolved. Obtained from: p4 branch bms_netdev Reviewed by: andre Sponsored by: Garance A Drosehn Idea from: NetBSD MFC after: 1 month
2007-03-20 00:36:10 +00:00
*/
While flushing the multicast filter of an interface, do not zero the relevant ifmultiaddr structures' reference to the parent interface, unless the parent interface is really detaching. While here, program only link layer multicast filters to a wlan's hardware parent interface. PR: kern/142391, kern/142392 Reviewed by: sam, rpaolo, bms MFC after: 1 week
2010-01-24 16:17:58 +00:00
static void
Implement reference counting for ifmultiaddr, in_multi, and in6_multi structures. Detect when ifnet instances are detached from the network stack and perform appropriate cleanup to prevent memory leaks. This has been implemented in such a way as to be backwards ABI compatible. Kernel consumers are changed to use if_delmulti_ifma(); in_delmulti() is unable to detect interface removal by design, as it performs searches on structures which are removed with the interface. With this architectural change, the panics FreeBSD users have experienced with carp and pfsync should be resolved. Obtained from: p4 branch bms_netdev Reviewed by: andre Sponsored by: Garance A Drosehn Idea from: NetBSD MFC after: 1 month
2007-03-20 00:36:10 +00:00
if_purgemaddrs(struct ifnet *ifp)
{
struct ifmultiaddr *ifma;
Convert all users of IF_ADDR_LOCK to use new locking macros that specify either a read lock or write lock. Reviewed by: bz MFC after: 2 weeks
2012-01-05 19:00:36 +00:00
IF_ADDR_WLOCK(ifp);
ifnet: Replace if_addr_lock rwlock with epoch + mutex Run on LLNW canaries and tested by pho@ gallatin: Using a 14-core, 28-HTT single socket E5-2697 v3 with a 40GbE MLX5 based ConnectX 4-LX NIC, I see an almost 12% improvement in received packet rate, and a larger improvement in bytes delivered all the way to userspace. When the host receiving 64 streams of netperf -H $DUT -t UDP_STREAM -- -m 1, I see, using nstat -I mce0 1 before the patch: InMpps OMpps InGbs OGbs err TCP Est %CPU syscalls csw irq GBfree 4.98 0.00 4.42 0.00 4235592 33 83.80 4720653 2149771 1235 247.32 4.73 0.00 4.20 0.00 4025260 33 82.99 4724900 2139833 1204 247.32 4.72 0.00 4.20 0.00 4035252 33 82.14 4719162 2132023 1264 247.32 4.71 0.00 4.21 0.00 4073206 33 83.68 4744973 2123317 1347 247.32 4.72 0.00 4.21 0.00 4061118 33 80.82 4713615 2188091 1490 247.32 4.72 0.00 4.21 0.00 4051675 33 85.29 4727399 2109011 1205 247.32 4.73 0.00 4.21 0.00 4039056 33 84.65 4724735 2102603 1053 247.32 After the patch InMpps OMpps InGbs OGbs err TCP Est %CPU syscalls csw irq GBfree 5.43 0.00 4.20 0.00 3313143 33 84.96 5434214 1900162 2656 245.51 5.43 0.00 4.20 0.00 3308527 33 85.24 5439695 1809382 2521 245.51 5.42 0.00 4.19 0.00 3316778 33 87.54 5416028 1805835 2256 245.51 5.42 0.00 4.19 0.00 3317673 33 90.44 5426044 1763056 2332 245.51 5.42 0.00 4.19 0.00 3314839 33 88.11 5435732 1792218 2499 245.52 5.44 0.00 4.19 0.00 3293228 33 91.84 5426301 1668597 2121 245.52 Similarly, netperf reports 230Mb/s before the patch, and 270Mb/s after the patch Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15366
2018-05-18 20:13:34 +00:00
while (!CK_STAILQ_EMPTY(&ifp->if_multiaddrs)) {
ifma = CK_STAILQ_FIRST(&ifp->if_multiaddrs);
CK_STAILQ_REMOVE(&ifp->if_multiaddrs, ifma, ifmultiaddr, ifma_link);
Implement reference counting for ifmultiaddr, in_multi, and in6_multi structures. Detect when ifnet instances are detached from the network stack and perform appropriate cleanup to prevent memory leaks. This has been implemented in such a way as to be backwards ABI compatible. Kernel consumers are changed to use if_delmulti_ifma(); in_delmulti() is unable to detect interface removal by design, as it performs searches on structures which are removed with the interface. With this architectural change, the panics FreeBSD users have experienced with carp and pfsync should be resolved. Obtained from: p4 branch bms_netdev Reviewed by: andre Sponsored by: Garance A Drosehn Idea from: NetBSD MFC after: 1 month
2007-03-20 00:36:10 +00:00
if_delmulti_locked(ifp, ifma, 1);
Separate list manipulation locking from state change in multicast Multicast incorrectly calls in to drivers with a mutex held causing drivers to have to go through all manner of contortions to use a non sleepable lock. Serialize multicast updates instead. Submitted by: mmacy <mmacy@mattmacy.io> Reviewed by: shurd, sbruno Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D14969
2018-05-02 19:36:29 +00:00
}
Convert all users of IF_ADDR_LOCK to use new locking macros that specify either a read lock or write lock. Reviewed by: bz MFC after: 2 weeks
2012-01-05 19:00:36 +00:00
IF_ADDR_WUNLOCK(ifp);
Implement reference counting for ifmultiaddr, in_multi, and in6_multi structures. Detect when ifnet instances are detached from the network stack and perform appropriate cleanup to prevent memory leaks. This has been implemented in such a way as to be backwards ABI compatible. Kernel consumers are changed to use if_delmulti_ifma(); in_delmulti() is unable to detect interface removal by design, as it performs searches on structures which are removed with the interface. With this architectural change, the panics FreeBSD users have experienced with carp and pfsync should be resolved. Obtained from: p4 branch bms_netdev Reviewed by: andre Sponsored by: Garance A Drosehn Idea from: NetBSD MFC after: 1 month
2007-03-20 00:36:10 +00:00
}
Bring the 'new-bus' to the i386. This extensively changes the way the i386 platform boots, it is no longer ISA-centric, and is fully dynamic. Most old drivers compile and run without modification via 'compatability shims' to enable a smoother transition. eisa, isapnp and pccard* are not yet using the new resource manager. Once fully converted, all drivers will be loadable, including PCI and ISA. (Some other changes appear to have snuck in, including a port of Soren's ATA driver to the Alpha. Soren, back this out if you need to.) This is a checkpoint of work-in-progress, but is quite functional. The bulk of the work was done over the last few years by Doug Rabson and Garrett Wollman. Approved by: core
1999-04-16 21:22:55 +00:00
/*
Introduce the if_vmove() function, which will be used in the future for reassigning ifnets from one vnet to another. if_vmove() works by calling a restricted subset of actions normally executed by if_detach() on an ifnet in the current vnet, and then switches to the target vnet and executes an appropriate subset of if_attach() actions there. if_attach() and if_detach() have become wrapper functions around if_attach_internal() and if_detach_internal(), where the later variants have an additional argument, a flag indicating whether a full attach or detach sequence is to be executed, or only a restricted subset suitable for moving an ifnet from one vnet to another. Hence, if_vmove() will not call if_detach() and if_attach() directly, but will call the if_detach_internal() and if_attach_internal() variants instead, with the vmove flag set. While here, staticize ifnet_setbyindex() since it is not referenced from outside of sys/net/if.c. Also rename ifccnt field in struct vimage to ifcnt, and do some minor whitespace garbage collection where appropriate. This change should have no functional impact on nooptions VIMAGE kernel builds. Reviewed by: bz, rwatson, brooks? Approved by: julian (mentor)
2009-05-22 22:09:00 +00:00
* Detach an interface, removing it from the list of "active" interfaces.
* If vmove flag is set on entry to if_detach_internal(), perform only a
* limited subset of cleanup tasks, given that we are moving an ifnet from
* one vnet to another, where it must be fully operational.
Take a first cut at cleaning up ifnet removal and multicast socket panics, which occur when stale ifnet pointers are left in struct moptions hung off of inpcbs: - Add in_ifdetach(), which matches in6_ifdetach(), and allows the protocol to perform early tear-down on the interface early in if_detach(). - Annotate that if_detach() needs careful consideration. - Remove calls to in_pcbpurgeif0() in the handling of SIOCDIFADDR -- this is not the place to detect interface removal! This also removes what is basically a nasty (and now unnecessary) hack. - Invoke in_pcbpurgeif0() from in_ifdetach(), in both raw and UDP IPv4 sockets. It is now possible to run the msocket_ifnet_remove regression test using HEAD without panicking. MFC after: 3 days
2005-09-18 17:36:28 +00:00
*
* XXXRW: There are some significant questions about event ordering, and
* how to prevent things from starting to use the interface during detach.
Bring the 'new-bus' to the i386. This extensively changes the way the i386 platform boots, it is no longer ISA-centric, and is fully dynamic. Most old drivers compile and run without modification via 'compatability shims' to enable a smoother transition. eisa, isapnp and pccard* are not yet using the new resource manager. Once fully converted, all drivers will be loadable, including PCI and ISA. (Some other changes appear to have snuck in, including a port of Soren's ATA driver to the Alpha. Soren, back this out if you need to.) This is a checkpoint of work-in-progress, but is quite functional. The bulk of the work was done over the last few years by Doug Rabson and Garrett Wollman. Approved by: core
1999-04-16 21:22:55 +00:00
*/
void
Clean up whitespace, remove "register" keyword, ANSIfy. No functional changes.
2003-10-23 13:49:10 +00:00
if_detach(struct ifnet *ifp)
Introduce the if_vmove() function, which will be used in the future for reassigning ifnets from one vnet to another. if_vmove() works by calling a restricted subset of actions normally executed by if_detach() on an ifnet in the current vnet, and then switches to the target vnet and executes an appropriate subset of if_attach() actions there. if_attach() and if_detach() have become wrapper functions around if_attach_internal() and if_detach_internal(), where the later variants have an additional argument, a flag indicating whether a full attach or detach sequence is to be executed, or only a restricted subset suitable for moving an ifnet from one vnet to another. Hence, if_vmove() will not call if_detach() and if_attach() directly, but will call the if_detach_internal() and if_attach_internal() variants instead, with the vmove flag set. While here, staticize ifnet_setbyindex() since it is not referenced from outside of sys/net/if.c. Also rename ifccnt field in struct vimage to ifcnt, and do some minor whitespace garbage collection where appropriate. This change should have no functional impact on nooptions VIMAGE kernel builds. Reviewed by: bz, rwatson, brooks? Approved by: julian (mentor)
2009-05-22 22:09:00 +00:00
{
PR: 168520 170096 Submitted by: adrian, zec Fix multiple kernel panics when VIMAGE is enabled in the kernel. These fixes are based on patches submitted by Adrian Chadd and Marko Zec. (1) Set curthread->td_vnet to vnet0 in device_probe_and_attach() just before calling device_attach(). This fixes multiple VIMAGE related kernel panics when trying to attach Bluetooth or USB Ethernet devices because curthread->td_vnet is NULL. (2) Set curthread->td_vnet in if_detach(). This fixes kernel panics when detaching networking interfaces, especially USB Ethernet devices. (3) Use VNET_DOMAIN_SET() in ng_btsocket.c (4) In ng_unref_node() set curthread->td_vnet. This fixes kernel panics when detaching Netgraph nodes.
2013-07-15 01:32:55 +00:00
CURVNET_SET_QUIET(ifp->if_vnet);
Fix group membership of cloned interfaces when one is moved by if_vmove(). In if_vmove(), if_detach_internal() and if_attach_internal() were called in series to detach and reattach the interface. When detaching, if_delgroup() was called and the interface leaves all of the group membership. And then upon attachment, if_addgroup(ifp, IFG_ALL) was called and it joined only "all" group again. This had a problem. Normally, a cloned interface automatically joins a group whose name is ifc_name of the cloner in addition to "all" upon creation. However, if_vmove() removed the membership and did not restore upon attachment. Differential Revision: https://reviews.freebsd.org/D1859
2015-03-02 20:00:03 +00:00
if_detach_internal(ifp, 0, NULL);
PR: 168520 170096 Submitted by: adrian, zec Fix multiple kernel panics when VIMAGE is enabled in the kernel. These fixes are based on patches submitted by Adrian Chadd and Marko Zec. (1) Set curthread->td_vnet to vnet0 in device_probe_and_attach() just before calling device_attach(). This fixes multiple VIMAGE related kernel panics when trying to attach Bluetooth or USB Ethernet devices because curthread->td_vnet is NULL. (2) Set curthread->td_vnet in if_detach(). This fixes kernel panics when detaching networking interfaces, especially USB Ethernet devices. (3) Use VNET_DOMAIN_SET() in ng_btsocket.c (4) In ng_unref_node() set curthread->td_vnet. This fixes kernel panics when detaching Netgraph nodes.
2013-07-15 01:32:55 +00:00
CURVNET_RESTORE();
Introduce the if_vmove() function, which will be used in the future for reassigning ifnets from one vnet to another. if_vmove() works by calling a restricted subset of actions normally executed by if_detach() on an ifnet in the current vnet, and then switches to the target vnet and executes an appropriate subset of if_attach() actions there. if_attach() and if_detach() have become wrapper functions around if_attach_internal() and if_detach_internal(), where the later variants have an additional argument, a flag indicating whether a full attach or detach sequence is to be executed, or only a restricted subset suitable for moving an ifnet from one vnet to another. Hence, if_vmove() will not call if_detach() and if_attach() directly, but will call the if_detach_internal() and if_attach_internal() variants instead, with the vmove flag set. While here, staticize ifnet_setbyindex() since it is not referenced from outside of sys/net/if.c. Also rename ifccnt field in struct vimage to ifcnt, and do some minor whitespace garbage collection where appropriate. This change should have no functional impact on nooptions VIMAGE kernel builds. Reviewed by: bz, rwatson, brooks? Approved by: julian (mentor)
2009-05-22 22:09:00 +00:00
}
Get closer to a VIMAGE network stack teardown from top to bottom rather than removing the network interfaces first. This change is rather larger and convoluted as the ordering requirements cannot be separated. Move the pfil(9) framework to SI_SUB_PROTO_PFIL, move Firewalls and related modules to their own SI_SUB_PROTO_FIREWALL. Move initialization of "physical" interfaces to SI_SUB_DRIVERS, move virtual (cloned) interfaces to SI_SUB_PSEUDO. Move Multicast to SI_SUB_PROTO_MC. Re-work parts of multicast initialisation and teardown, not taking the huge amount of memory into account if used as a module yet. For interface teardown we try to do as many of them as we can on SI_SUB_INIT_IF, but for some this makes no sense, e.g., when tunnelling over a higher layer protocol such as IP. In that case the interface has to go along (or before) the higher layer protocol is shutdown. Kernel hhooks need to go last on teardown as they may be used at various higher layers and we cannot remove them before we cleaned up the higher layers. For interface teardown there are multiple paths: (a) a cloned interface is destroyed (inside a VIMAGE or in the base system), (b) any interface is moved from a virtual network stack to a different network stack ("vmove"), or (c) a virtual network stack is being shut down. All code paths go through if_detach_internal() where we, depending on the vmove flag or the vnet state, make a decision on how much to shut down; in case we are destroying a VNET the individual protocol layers will cleanup their own parts thus we cannot do so again for each interface as we end up with, e.g., double-frees, destroying locks twice or acquiring already destroyed locks. When calling into protocol cleanups we equally have to tell them whether they need to detach upper layer protocols ("ulp") or not (e.g., in6_ifdetach()). Provide or enahnce helper functions to do proper cleanup at a protocol rather than at an interface level. Approved by: re (hrs) Obtained from: projects/vnet Reviewed by: gnn, jhb Sponsored by: The FreeBSD Foundation MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D6747
2016-06-21 13:48:49 +00:00
/*
* The vmove flag, if set, indicates that we are called from a callpath
* that is moving an interface to a different vnet instance.
*
* The shutdown flag, if set, indicates that we are called in the
* process of shutting down a vnet instance. Currently only the
* vnet_if_return SYSUNINIT function sets it. Note: we can be called
* on a vnet instance shutdown without this flag being set, e.g., when
* the cloned interfaces are destoyed as first thing of teardown.
*/
If vnets are torn down while ifconfig runs an ioctl to say, destroy an epair(4), we may hit if_detach_internal() without holding a lock and by the time we aquire it the interface might be gone. We should not panic() in this case as it is our fault for not holding the lock all the way. It is not ideal to return silently without error to user space, but other callers will all ignore the return values so do not change the entire KPI for little benefit for now. The ifp will be dealt with one way or another still. Sponsored by: The FreeBSD Foundation MFC after: 2 weeks Reviewed by: gnn Differential Revision: https://reviews.freebsd.org/D4529
2015-12-22 15:03:45 +00:00
static int
Fix group membership of cloned interfaces when one is moved by if_vmove(). In if_vmove(), if_detach_internal() and if_attach_internal() were called in series to detach and reattach the interface. When detaching, if_delgroup() was called and the interface leaves all of the group membership. And then upon attachment, if_addgroup(ifp, IFG_ALL) was called and it joined only "all" group again. This had a problem. Normally, a cloned interface automatically joins a group whose name is ifc_name of the cloner in addition to "all" upon creation. However, if_vmove() removed the membership and did not restore upon attachment. Differential Revision: https://reviews.freebsd.org/D1859
2015-03-02 20:00:03 +00:00
if_detach_internal(struct ifnet *ifp, int vmove, struct if_clone **ifcp)
Bring the 'new-bus' to the i386. This extensively changes the way the i386 platform boots, it is no longer ISA-centric, and is fully dynamic. Most old drivers compile and run without modification via 'compatability shims' to enable a smoother transition. eisa, isapnp and pccard* are not yet using the new resource manager. Once fully converted, all drivers will be loadable, including PCI and ISA. (Some other changes appear to have snuck in, including a port of Soren's ATA driver to the Alpha. Soren, back this out if you need to.) This is a checkpoint of work-in-progress, but is quite functional. The bulk of the work was done over the last few years by Doug Rabson and Garrett Wollman. Approved by: core
1999-04-16 21:22:55 +00:00
{
Separate out address-detaching part of if_detach into if_purgeaddrs, so if_tap doesn't need to rely on locally-rolled code to do same. The observable symptom of if_tap's bzero'ing the address details was a crash in "ifconfig tap0" after an if_tap device was closed. Reported By: Matti Saarinen (mjsaarin at cc dot helsinki dot fi)
2005-05-25 13:52:03 +00:00
struct ifaddr *ifa;
MFP r274295: * Move interface route cleanup to route.c:rt_flushifroutes() * Convert most of "for (fibnum = 0; fibnum < rt_numfibs; fibnum++)" users to use new rt_foreach_fib() instead of hand-rolling cycles.
2015-08-08 18:14:59 +00:00
int i;
- add dom_if{attach,detach} framework. - transition to use ifp->if_afdata. Obtained from: KAME
2003-10-17 15:46:31 +00:00
struct domain *dp;
Do not attempt to clean up data that has not been initialized yet. This fixes two kernel panics on boot when the xl driver fails to allocate bus/port/memory resources. Reviewed by: silence on -net
2004-08-06 09:08:33 +00:00
struct ifnet *iter;
After r302054 unloading an network interface driver on a kernel without VIMAGE support would dereference a NULL point unconditionally leading to a panic. Wrap the entire VIMAGE related code with #ifdefs rather than just the decision making part to save an extra bit of resources. Reported by: np Sponsored by: The FreeBSD Foundation MFC After: 13 days Approved by: re (marius)
2016-06-22 11:45:30 +00:00
int found = 0;
#ifdef VIMAGE
Factor out VNET shutdown check into an own vnet structure field. Remove the now obsolete vnet_state field. This greatly simplifies the detection of VNET shutdown and avoids code duplication. Discussed with: bz@ MFC after: 1 week Sponsored by: Mellanox Technologies
2019-10-07 14:15:41 +00:00
bool shutdown;
- First initialize ifnet, and then insert it into global list. - First remove from global list, then start destroying. PR: kern/97679 Submitted by: Alex Lyashkov <shadow itt.net.ru> Reviewed by: rwatson, brooks
2006-06-21 06:02:35 +00:00
Factor out VNET shutdown check into an own vnet structure field. Remove the now obsolete vnet_state field. This greatly simplifies the detection of VNET shutdown and avoids code duplication. Discussed with: bz@ MFC after: 1 week Sponsored by: Mellanox Technologies
2019-10-07 14:15:41 +00:00
shutdown = ifp->if_vnet->vnet_shutdown;
After r302054 unloading an network interface driver on a kernel without VIMAGE support would dereference a NULL point unconditionally leading to a panic. Wrap the entire VIMAGE related code with #ifdefs rather than just the decision making part to save an extra bit of resources. Reported by: np Sponsored by: The FreeBSD Foundation MFC After: 13 days Approved by: re (marius)
2016-06-22 11:45:30 +00:00
#endif
- First initialize ifnet, and then insert it into global list. - First remove from global list, then start destroying. PR: kern/97679 Submitted by: Alex Lyashkov <shadow itt.net.ru> Reviewed by: rwatson, brooks
2006-06-21 06:02:35 +00:00
IFNET_WLOCK();
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
CK_STAILQ_FOREACH(iter, &V_ifnet, if_link)
- First initialize ifnet, and then insert it into global list. - First remove from global list, then start destroying. PR: kern/97679 Submitted by: Alex Lyashkov <shadow itt.net.ru> Reviewed by: rwatson, brooks
2006-06-21 06:02:35 +00:00
if (iter == ifp) {
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
CK_STAILQ_REMOVE(&V_ifnet, ifp, ifnet, if_link);
Fix possible panic during ifnet detach in rtsock. The panic can happen, when some application does dump of routing table using sysctl interface. To prevent this, set IFF_DYING flag in if_detach_internal() function, when ifnet under lock is removed from the chain. In sysctl_rtsock() take IFNET_RLOCK_NOSLEEP() to prevent ifnet detach during routes enumeration. In case, if some interface was detached in the time before we take the lock, add the check, that ifnet is not DYING. This prevents access to memory that could be freed after ifnet is unlinked. PR: 227720, 230498, 233306 Reviewed by: bz, eugen MFC after: 1 week Sponsored by: Yandex LLC Differential Revision: https://reviews.freebsd.org/D18338
2018-11-27 09:04:06 +00:00
if (!vmove)
ifp->if_flags |= IFF_DYING;
- First initialize ifnet, and then insert it into global list. - First remove from global list, then start destroying. PR: kern/97679 Submitted by: Alex Lyashkov <shadow itt.net.ru> Reviewed by: rwatson, brooks
2006-06-21 06:02:35 +00:00
found = 1;
break;
}
IFNET_WUNLOCK();
Introduce the if_vmove() function, which will be used in the future for reassigning ifnets from one vnet to another. if_vmove() works by calling a restricted subset of actions normally executed by if_detach() on an ifnet in the current vnet, and then switches to the target vnet and executes an appropriate subset of if_attach() actions there. if_attach() and if_detach() have become wrapper functions around if_attach_internal() and if_detach_internal(), where the later variants have an additional argument, a flag indicating whether a full attach or detach sequence is to be executed, or only a restricted subset suitable for moving an ifnet from one vnet to another. Hence, if_vmove() will not call if_detach() and if_attach() directly, but will call the if_detach_internal() and if_attach_internal() variants instead, with the vmove flag set. While here, staticize ifnet_setbyindex() since it is not referenced from outside of sys/net/if.c. Also rename ifccnt field in struct vimage to ifcnt, and do some minor whitespace garbage collection where appropriate. This change should have no functional impact on nooptions VIMAGE kernel builds. Reviewed by: bz, rwatson, brooks? Approved by: julian (mentor)
2009-05-22 22:09:00 +00:00
if (!found) {
If vnets are torn down while ifconfig runs an ioctl to say, destroy an epair(4), we may hit if_detach_internal() without holding a lock and by the time we aquire it the interface might be gone. We should not panic() in this case as it is our fault for not holding the lock all the way. It is not ideal to return silently without error to user space, but other callers will all ignore the return values so do not change the entire KPI for little benefit for now. The ifp will be dealt with one way or another still. Sponsored by: The FreeBSD Foundation MFC after: 2 weeks Reviewed by: gnn Differential Revision: https://reviews.freebsd.org/D4529
2015-12-22 15:03:45 +00:00
/*
* While we would want to panic here, we cannot
* guarantee that the interface is indeed still on
* the list given we don't hold locks all the way.
*/
return (ENOENT);
#if 0
Introduce the if_vmove() function, which will be used in the future for reassigning ifnets from one vnet to another. if_vmove() works by calling a restricted subset of actions normally executed by if_detach() on an ifnet in the current vnet, and then switches to the target vnet and executes an appropriate subset of if_attach() actions there. if_attach() and if_detach() have become wrapper functions around if_attach_internal() and if_detach_internal(), where the later variants have an additional argument, a flag indicating whether a full attach or detach sequence is to be executed, or only a restricted subset suitable for moving an ifnet from one vnet to another. Hence, if_vmove() will not call if_detach() and if_attach() directly, but will call the if_detach_internal() and if_attach_internal() variants instead, with the vmove flag set. While here, staticize ifnet_setbyindex() since it is not referenced from outside of sys/net/if.c. Also rename ifccnt field in struct vimage to ifcnt, and do some minor whitespace garbage collection where appropriate. This change should have no functional impact on nooptions VIMAGE kernel builds. Reviewed by: bz, rwatson, brooks? Approved by: julian (mentor)
2009-05-22 22:09:00 +00:00
if (vmove)
Enhance a panic string to contain more useful debugging information. Sponsored by: ISPsystem Reviewed by: rwatson MFC after: 5 days
2010-02-20 21:43:36 +00:00
panic("%s: ifp=%p not on the ifnet tailq %p",
__func__, ifp, &V_ifnet);
Introduce the if_vmove() function, which will be used in the future for reassigning ifnets from one vnet to another. if_vmove() works by calling a restricted subset of actions normally executed by if_detach() on an ifnet in the current vnet, and then switches to the target vnet and executes an appropriate subset of if_attach() actions there. if_attach() and if_detach() have become wrapper functions around if_attach_internal() and if_detach_internal(), where the later variants have an additional argument, a flag indicating whether a full attach or detach sequence is to be executed, or only a restricted subset suitable for moving an ifnet from one vnet to another. Hence, if_vmove() will not call if_detach() and if_attach() directly, but will call the if_detach_internal() and if_attach_internal() variants instead, with the vmove flag set. While here, staticize ifnet_setbyindex() since it is not referenced from outside of sys/net/if.c. Also rename ifccnt field in struct vimage to ifcnt, and do some minor whitespace garbage collection where appropriate. This change should have no functional impact on nooptions VIMAGE kernel builds. Reviewed by: bz, rwatson, brooks? Approved by: julian (mentor)
2009-05-22 22:09:00 +00:00
else
return; /* XXX this should panic as well? */
If vnets are torn down while ifconfig runs an ioctl to say, destroy an epair(4), we may hit if_detach_internal() without holding a lock and by the time we aquire it the interface might be gone. We should not panic() in this case as it is our fault for not holding the lock all the way. It is not ideal to return silently without error to user space, but other callers will all ignore the return values so do not change the entire KPI for little benefit for now. The ifp will be dealt with one way or another still. Sponsored by: The FreeBSD Foundation MFC after: 2 weeks Reviewed by: gnn Differential Revision: https://reviews.freebsd.org/D4529
2015-12-22 15:03:45 +00:00
#endif
Introduce the if_vmove() function, which will be used in the future for reassigning ifnets from one vnet to another. if_vmove() works by calling a restricted subset of actions normally executed by if_detach() on an ifnet in the current vnet, and then switches to the target vnet and executes an appropriate subset of if_attach() actions there. if_attach() and if_detach() have become wrapper functions around if_attach_internal() and if_detach_internal(), where the later variants have an additional argument, a flag indicating whether a full attach or detach sequence is to be executed, or only a restricted subset suitable for moving an ifnet from one vnet to another. Hence, if_vmove() will not call if_detach() and if_attach() directly, but will call the if_detach_internal() and if_attach_internal() variants instead, with the vmove flag set. While here, staticize ifnet_setbyindex() since it is not referenced from outside of sys/net/if.c. Also rename ifccnt field in struct vimage to ifcnt, and do some minor whitespace garbage collection where appropriate. This change should have no functional impact on nooptions VIMAGE kernel builds. Reviewed by: bz, rwatson, brooks? Approved by: julian (mentor)
2009-05-22 22:09:00 +00:00
}
Bring the 'new-bus' to the i386. This extensively changes the way the i386 platform boots, it is no longer ISA-centric, and is fully dynamic. Most old drivers compile and run without modification via 'compatability shims' to enable a smoother transition. eisa, isapnp and pccard* are not yet using the new resource manager. Once fully converted, all drivers will be loadable, including PCI and ISA. (Some other changes appear to have snuck in, including a port of Soren's ATA driver to the Alpha. Soren, back this out if you need to.) This is a checkpoint of work-in-progress, but is quite functional. The bulk of the work was done over the last few years by Doug Rabson and Garrett Wollman. Approved by: core
1999-04-16 21:22:55 +00:00
Get closer to a VIMAGE network stack teardown from top to bottom rather than removing the network interfaces first. This change is rather larger and convoluted as the ordering requirements cannot be separated. Move the pfil(9) framework to SI_SUB_PROTO_PFIL, move Firewalls and related modules to their own SI_SUB_PROTO_FIREWALL. Move initialization of "physical" interfaces to SI_SUB_DRIVERS, move virtual (cloned) interfaces to SI_SUB_PSEUDO. Move Multicast to SI_SUB_PROTO_MC. Re-work parts of multicast initialisation and teardown, not taking the huge amount of memory into account if used as a module yet. For interface teardown we try to do as many of them as we can on SI_SUB_INIT_IF, but for some this makes no sense, e.g., when tunnelling over a higher layer protocol such as IP. In that case the interface has to go along (or before) the higher layer protocol is shutdown. Kernel hhooks need to go last on teardown as they may be used at various higher layers and we cannot remove them before we cleaned up the higher layers. For interface teardown there are multiple paths: (a) a cloned interface is destroyed (inside a VIMAGE or in the base system), (b) any interface is moved from a virtual network stack to a different network stack ("vmove"), or (c) a virtual network stack is being shut down. All code paths go through if_detach_internal() where we, depending on the vmove flag or the vnet state, make a decision on how much to shut down; in case we are destroying a VNET the individual protocol layers will cleanup their own parts thus we cannot do so again for each interface as we end up with, e.g., double-frees, destroying locks twice or acquiring already destroyed locks. When calling into protocol cleanups we equally have to tell them whether they need to detach upper layer protocols ("ulp") or not (e.g., in6_ifdetach()). Provide or enahnce helper functions to do proper cleanup at a protocol rather than at an interface level. Approved by: re (hrs) Obtained from: projects/vnet Reviewed by: gnn, jhb Sponsored by: The FreeBSD Foundation MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D6747
2016-06-21 13:48:49 +00:00
/*
* At this point we know the interface still was on the ifnet list
* and we removed it so we are in a stable state.
*/
#ifdef VIMAGE
curvnet->vnet_ifcnt--;
#endif
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
epoch_wait_preempt(net_epoch_preempt);
Need to wait for epoch callbacks to complete before detaching a network interface. This particularly manifests itself when an INP has multicast options attached during a network interface detach. Then the IPv4 and IPv6 leave group call which results from freeing the multicast address, may access a freed ifnet structure. These are the steps to reproduce: service mdnsd onestart # installed from ports ifconfig epair create ifconfig epair0a 0/24 up ifconfig epair0a destroy Tested by: pho @ MFC after: 1 week Sponsored by: Mellanox Technologies
2019-06-28 10:49:04 +00:00
/*
* Ensure all pending EPOCH(9) callbacks have been executed. This
* fixes issues about late destruction of multicast options
* which lead to leave group calls, which in turn access the
* belonging ifnet structure:
*/
epoch_drain_callbacks(net_epoch_preempt);
Get closer to a VIMAGE network stack teardown from top to bottom rather than removing the network interfaces first. This change is rather larger and convoluted as the ordering requirements cannot be separated. Move the pfil(9) framework to SI_SUB_PROTO_PFIL, move Firewalls and related modules to their own SI_SUB_PROTO_FIREWALL. Move initialization of "physical" interfaces to SI_SUB_DRIVERS, move virtual (cloned) interfaces to SI_SUB_PSEUDO. Move Multicast to SI_SUB_PROTO_MC. Re-work parts of multicast initialisation and teardown, not taking the huge amount of memory into account if used as a module yet. For interface teardown we try to do as many of them as we can on SI_SUB_INIT_IF, but for some this makes no sense, e.g., when tunnelling over a higher layer protocol such as IP. In that case the interface has to go along (or before) the higher layer protocol is shutdown. Kernel hhooks need to go last on teardown as they may be used at various higher layers and we cannot remove them before we cleaned up the higher layers. For interface teardown there are multiple paths: (a) a cloned interface is destroyed (inside a VIMAGE or in the base system), (b) any interface is moved from a virtual network stack to a different network stack ("vmove"), or (c) a virtual network stack is being shut down. All code paths go through if_detach_internal() where we, depending on the vmove flag or the vnet state, make a decision on how much to shut down; in case we are destroying a VNET the individual protocol layers will cleanup their own parts thus we cannot do so again for each interface as we end up with, e.g., double-frees, destroying locks twice or acquiring already destroyed locks. When calling into protocol cleanups we equally have to tell them whether they need to detach upper layer protocols ("ulp") or not (e.g., in6_ifdetach()). Provide or enahnce helper functions to do proper cleanup at a protocol rather than at an interface level. Approved by: re (hrs) Obtained from: projects/vnet Reviewed by: gnn, jhb Sponsored by: The FreeBSD Foundation MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D6747
2016-06-21 13:48:49 +00:00
/*
* In any case (destroy or vmove) detach us from the groups
* and remove/wait for pending events on the taskq.
* XXX-BZ in theory an interface could still enqueue a taskq change?
*/
if_delgroups(ifp);
taskqueue_drain(taskqueue_swi, &ifp->if_linktask);
/*
* Check if this is a cloned interface or not. Must do even if
* shutting down as a if_vmove_reclaim() would move the ifp and
* the if_clone_addgroup() will have a corrupted string overwise
* from a gibberish pointer.
*/
Fix group membership of cloned interfaces when one is moved by if_vmove(). In if_vmove(), if_detach_internal() and if_attach_internal() were called in series to detach and reattach the interface. When detaching, if_delgroup() was called and the interface leaves all of the group membership. And then upon attachment, if_addgroup(ifp, IFG_ALL) was called and it joined only "all" group again. This had a problem. Normally, a cloned interface automatically joins a group whose name is ifc_name of the cloner in addition to "all" upon creation. However, if_vmove() removed the membership and did not restore upon attachment. Differential Revision: https://reviews.freebsd.org/D1859
2015-03-02 20:00:03 +00:00
if (vmove && ifcp != NULL)
*ifcp = if_clone_findifc(ifp);
Get closer to a VIMAGE network stack teardown from top to bottom rather than removing the network interfaces first. This change is rather larger and convoluted as the ordering requirements cannot be separated. Move the pfil(9) framework to SI_SUB_PROTO_PFIL, move Firewalls and related modules to their own SI_SUB_PROTO_FIREWALL. Move initialization of "physical" interfaces to SI_SUB_DRIVERS, move virtual (cloned) interfaces to SI_SUB_PSEUDO. Move Multicast to SI_SUB_PROTO_MC. Re-work parts of multicast initialisation and teardown, not taking the huge amount of memory into account if used as a module yet. For interface teardown we try to do as many of them as we can on SI_SUB_INIT_IF, but for some this makes no sense, e.g., when tunnelling over a higher layer protocol such as IP. In that case the interface has to go along (or before) the higher layer protocol is shutdown. Kernel hhooks need to go last on teardown as they may be used at various higher layers and we cannot remove them before we cleaned up the higher layers. For interface teardown there are multiple paths: (a) a cloned interface is destroyed (inside a VIMAGE or in the base system), (b) any interface is moved from a virtual network stack to a different network stack ("vmove"), or (c) a virtual network stack is being shut down. All code paths go through if_detach_internal() where we, depending on the vmove flag or the vnet state, make a decision on how much to shut down; in case we are destroying a VNET the individual protocol layers will cleanup their own parts thus we cannot do so again for each interface as we end up with, e.g., double-frees, destroying locks twice or acquiring already destroyed locks. When calling into protocol cleanups we equally have to tell them whether they need to detach upper layer protocols ("ulp") or not (e.g., in6_ifdetach()). Provide or enahnce helper functions to do proper cleanup at a protocol rather than at an interface level. Approved by: re (hrs) Obtained from: projects/vnet Reviewed by: gnn, jhb Sponsored by: The FreeBSD Foundation MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D6747
2016-06-21 13:48:49 +00:00
if_down(ifp);
After r302054 unloading an network interface driver on a kernel without VIMAGE support would dereference a NULL point unconditionally leading to a panic. Wrap the entire VIMAGE related code with #ifdefs rather than just the decision making part to save an extra bit of resources. Reported by: np Sponsored by: The FreeBSD Foundation MFC After: 13 days Approved by: re (marius)
2016-06-22 11:45:30 +00:00
#ifdef VIMAGE
Do not call all link state callbacks directly, but schedule a taskqueue(9) task. This fixes LORs and adds possibility to serve such events pseudorecursively, when link state change of interface causes subsequent change on other interfaces. Sponsored by: Rambler Reviewed by: sam, brooks, mux
2005-04-20 09:30:54 +00:00
/*
Get closer to a VIMAGE network stack teardown from top to bottom rather than removing the network interfaces first. This change is rather larger and convoluted as the ordering requirements cannot be separated. Move the pfil(9) framework to SI_SUB_PROTO_PFIL, move Firewalls and related modules to their own SI_SUB_PROTO_FIREWALL. Move initialization of "physical" interfaces to SI_SUB_DRIVERS, move virtual (cloned) interfaces to SI_SUB_PSEUDO. Move Multicast to SI_SUB_PROTO_MC. Re-work parts of multicast initialisation and teardown, not taking the huge amount of memory into account if used as a module yet. For interface teardown we try to do as many of them as we can on SI_SUB_INIT_IF, but for some this makes no sense, e.g., when tunnelling over a higher layer protocol such as IP. In that case the interface has to go along (or before) the higher layer protocol is shutdown. Kernel hhooks need to go last on teardown as they may be used at various higher layers and we cannot remove them before we cleaned up the higher layers. For interface teardown there are multiple paths: (a) a cloned interface is destroyed (inside a VIMAGE or in the base system), (b) any interface is moved from a virtual network stack to a different network stack ("vmove"), or (c) a virtual network stack is being shut down. All code paths go through if_detach_internal() where we, depending on the vmove flag or the vnet state, make a decision on how much to shut down; in case we are destroying a VNET the individual protocol layers will cleanup their own parts thus we cannot do so again for each interface as we end up with, e.g., double-frees, destroying locks twice or acquiring already destroyed locks. When calling into protocol cleanups we equally have to tell them whether they need to detach upper layer protocols ("ulp") or not (e.g., in6_ifdetach()). Provide or enahnce helper functions to do proper cleanup at a protocol rather than at an interface level. Approved by: re (hrs) Obtained from: projects/vnet Reviewed by: gnn, jhb Sponsored by: The FreeBSD Foundation MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D6747
2016-06-21 13:48:49 +00:00
* On VNET shutdown abort here as the stack teardown will do all
* the work top-down for us.
*/
if (shutdown) {
Notify that the ifnet will go away, even on vnet shutdown pf subscribes to ifnet_departure_event events, so it can clean up the ifg_pf_kif and if_pf_kif pointers in the ifnet. During vnet shutdown interfaces could go away without sending the event, so pf ends up cleaning these up as part of its shutdown sequence, which happens after the ifnet has already been freed. Send the ifnet_departure_event during vnet shutdown, allowing pf to clean up correctly. MFC after: 2 weeks Sponsored by: Orange Business Services Differential Revision: https://reviews.freebsd.org/D17500
2018-11-02 16:50:17 +00:00
/* Give interface users the chance to clean up. */
EVENTHANDLER_INVOKE(ifnet_departure_event, ifp);
Get closer to a VIMAGE network stack teardown from top to bottom rather than removing the network interfaces first. This change is rather larger and convoluted as the ordering requirements cannot be separated. Move the pfil(9) framework to SI_SUB_PROTO_PFIL, move Firewalls and related modules to their own SI_SUB_PROTO_FIREWALL. Move initialization of "physical" interfaces to SI_SUB_DRIVERS, move virtual (cloned) interfaces to SI_SUB_PSEUDO. Move Multicast to SI_SUB_PROTO_MC. Re-work parts of multicast initialisation and teardown, not taking the huge amount of memory into account if used as a module yet. For interface teardown we try to do as many of them as we can on SI_SUB_INIT_IF, but for some this makes no sense, e.g., when tunnelling over a higher layer protocol such as IP. In that case the interface has to go along (or before) the higher layer protocol is shutdown. Kernel hhooks need to go last on teardown as they may be used at various higher layers and we cannot remove them before we cleaned up the higher layers. For interface teardown there are multiple paths: (a) a cloned interface is destroyed (inside a VIMAGE or in the base system), (b) any interface is moved from a virtual network stack to a different network stack ("vmove"), or (c) a virtual network stack is being shut down. All code paths go through if_detach_internal() where we, depending on the vmove flag or the vnet state, make a decision on how much to shut down; in case we are destroying a VNET the individual protocol layers will cleanup their own parts thus we cannot do so again for each interface as we end up with, e.g., double-frees, destroying locks twice or acquiring already destroyed locks. When calling into protocol cleanups we equally have to tell them whether they need to detach upper layer protocols ("ulp") or not (e.g., in6_ifdetach()). Provide or enahnce helper functions to do proper cleanup at a protocol rather than at an interface level. Approved by: re (hrs) Obtained from: projects/vnet Reviewed by: gnn, jhb Sponsored by: The FreeBSD Foundation MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D6747
2016-06-21 13:48:49 +00:00
/*
* In case of a vmove we are done here without error.
* If we would signal an error it would lead to the same
* abort as if we did not find the ifnet anymore.
* if_detach() calls us in void context and does not care
* about an early abort notification, so life is splendid :)
*/
goto finish_vnet_shutdown;
}
After r302054 unloading an network interface driver on a kernel without VIMAGE support would dereference a NULL point unconditionally leading to a panic. Wrap the entire VIMAGE related code with #ifdefs rather than just the decision making part to save an extra bit of resources. Reported by: np Sponsored by: The FreeBSD Foundation MFC After: 13 days Approved by: re (marius)
2016-06-22 11:45:30 +00:00
#endif
Get closer to a VIMAGE network stack teardown from top to bottom rather than removing the network interfaces first. This change is rather larger and convoluted as the ordering requirements cannot be separated. Move the pfil(9) framework to SI_SUB_PROTO_PFIL, move Firewalls and related modules to their own SI_SUB_PROTO_FIREWALL. Move initialization of "physical" interfaces to SI_SUB_DRIVERS, move virtual (cloned) interfaces to SI_SUB_PSEUDO. Move Multicast to SI_SUB_PROTO_MC. Re-work parts of multicast initialisation and teardown, not taking the huge amount of memory into account if used as a module yet. For interface teardown we try to do as many of them as we can on SI_SUB_INIT_IF, but for some this makes no sense, e.g., when tunnelling over a higher layer protocol such as IP. In that case the interface has to go along (or before) the higher layer protocol is shutdown. Kernel hhooks need to go last on teardown as they may be used at various higher layers and we cannot remove them before we cleaned up the higher layers. For interface teardown there are multiple paths: (a) a cloned interface is destroyed (inside a VIMAGE or in the base system), (b) any interface is moved from a virtual network stack to a different network stack ("vmove"), or (c) a virtual network stack is being shut down. All code paths go through if_detach_internal() where we, depending on the vmove flag or the vnet state, make a decision on how much to shut down; in case we are destroying a VNET the individual protocol layers will cleanup their own parts thus we cannot do so again for each interface as we end up with, e.g., double-frees, destroying locks twice or acquiring already destroyed locks. When calling into protocol cleanups we equally have to tell them whether they need to detach upper layer protocols ("ulp") or not (e.g., in6_ifdetach()). Provide or enahnce helper functions to do proper cleanup at a protocol rather than at an interface level. Approved by: re (hrs) Obtained from: projects/vnet Reviewed by: gnn, jhb Sponsored by: The FreeBSD Foundation MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D6747
2016-06-21 13:48:49 +00:00
/*
* At this point we are not tearing down a VNET and are either
* going to destroy or vmove the interface and have to cleanup
* accordingly.
Do not call all link state callbacks directly, but schedule a taskqueue(9) task. This fixes LORs and adds possibility to serve such events pseudorecursively, when link state change of interface causes subsequent change on other interfaces. Sponsored by: Rambler Reviewed by: sam, brooks, mux
2005-04-20 09:30:54 +00:00
*/
Bring the 'new-bus' to the i386. This extensively changes the way the i386 platform boots, it is no longer ISA-centric, and is fully dynamic. Most old drivers compile and run without modification via 'compatability shims' to enable a smoother transition. eisa, isapnp and pccard* are not yet using the new resource manager. Once fully converted, all drivers will be loadable, including PCI and ISA. (Some other changes appear to have snuck in, including a port of Soren's ATA driver to the Alpha. Soren, back this out if you need to.) This is a checkpoint of work-in-progress, but is quite functional. The bulk of the work was done over the last few years by Doug Rabson and Garrett Wollman. Approved by: core
1999-04-16 21:22:55 +00:00
/*
* Remove routes and flush queues.
*/
Link ALTQ to the build and break with ABI for struct ifnet. Please recompile your (network) modules as well as any userland that might make sense of sizeof(struct ifnet). This does not change the queueing yet. These changes will follow in a seperate commit. Same with the driver changes, which need case by case evaluation. __FreeBSD_version bump will follow. Tested-by: (i386)LINT
2004-06-13 17:29:10 +00:00
#ifdef ALTQ
if (ALTQ_IS_ENABLED(&ifp->if_snd))
altq_disable(&ifp->if_snd);
if (ALTQ_IS_ATTACHED(&ifp->if_snd))
altq_detach(&ifp->if_snd);
#endif
Bring the 'new-bus' to the i386. This extensively changes the way the i386 platform boots, it is no longer ISA-centric, and is fully dynamic. Most old drivers compile and run without modification via 'compatability shims' to enable a smoother transition. eisa, isapnp and pccard* are not yet using the new resource manager. Once fully converted, all drivers will be loadable, including PCI and ISA. (Some other changes appear to have snuck in, including a port of Soren's ATA driver to the Alpha. Soren, back this out if you need to.) This is a checkpoint of work-in-progress, but is quite functional. The bulk of the work was done over the last few years by Doug Rabson and Garrett Wollman. Approved by: core
1999-04-16 21:22:55 +00:00
Separate out address-detaching part of if_detach into if_purgeaddrs, so if_tap doesn't need to rely on locally-rolled code to do same. The observable symptom of if_tap's bzero'ing the address details was a crash in "ifconfig tap0" after an if_tap device was closed. Reported By: Matti Saarinen (mjsaarin at cc dot helsinki dot fi)
2005-05-25 13:52:03 +00:00
if_purgeaddrs(ifp);
Bring the 'new-bus' to the i386. This extensively changes the way the i386 platform boots, it is no longer ISA-centric, and is fully dynamic. Most old drivers compile and run without modification via 'compatability shims' to enable a smoother transition. eisa, isapnp and pccard* are not yet using the new resource manager. Once fully converted, all drivers will be loadable, including PCI and ISA. (Some other changes appear to have snuck in, including a port of Soren's ATA driver to the Alpha. Soren, back this out if you need to.) This is a checkpoint of work-in-progress, but is quite functional. The bulk of the work was done over the last few years by Doug Rabson and Garrett Wollman. Approved by: core
1999-04-16 21:22:55 +00:00
Take a first cut at cleaning up ifnet removal and multicast socket panics, which occur when stale ifnet pointers are left in struct moptions hung off of inpcbs: - Add in_ifdetach(), which matches in6_ifdetach(), and allows the protocol to perform early tear-down on the interface early in if_detach(). - Annotate that if_detach() needs careful consideration. - Remove calls to in_pcbpurgeif0() in the handling of SIOCDIFADDR -- this is not the place to detect interface removal! This also removes what is basically a nasty (and now unnecessary) hack. - Invoke in_pcbpurgeif0() from in_ifdetach(), in both raw and UDP IPv4 sockets. It is now possible to run the msocket_ifnet_remove regression test using HEAD without panicking. MFC after: 3 days
2005-09-18 17:36:28 +00:00
#ifdef INET
in_ifdetach(ifp);
#endif
Sync with recent KAME. This work was based on kame-20010528-freebsd43-snap.tgz and some critical problem after the snap was out were fixed. There are many many changes since last KAME merge. TODO: - The definitions of SADB_* in sys/net/pfkeyv2.h are still different from RFC2407/IANA assignment because of binary compatibility issue. It should be fixed under 5-CURRENT. - ip6po_m member of struct ip6_pktopts is no longer used. But, it is still there because of binary compatibility issue. It should be removed under 5-CURRENT. Reviewed by: itojun Obtained from: KAME MFC after: 3 weeks
2001-06-11 12:39:29 +00:00
#ifdef INET6
/*
* Remove all IPv6 kernel structs related to ifp. This should be done
* before removing routing entries below, since IPv6 interface direct
* routes are expected to be removed by the IPv6-specific kernel API.
* Otherwise, the kernel will detect some inconsistency and bark it.
*/
in6_ifdetach(ifp);
#endif
Implement reference counting for ifmultiaddr, in_multi, and in6_multi structures. Detect when ifnet instances are detached from the network stack and perform appropriate cleanup to prevent memory leaks. This has been implemented in such a way as to be backwards ABI compatible. Kernel consumers are changed to use if_delmulti_ifma(); in_delmulti() is unable to detect interface removal by design, as it performs searches on structures which are removed with the interface. With this architectural change, the panics FreeBSD users have experienced with carp and pfsync should be resolved. Obtained from: p4 branch bms_netdev Reviewed by: andre Sponsored by: Garance A Drosehn Idea from: NetBSD MFC after: 1 month
2007-03-20 00:36:10 +00:00
if_purgemaddrs(ifp);
net: move interface removal notification up in if_detach_internal This is needed to prevent having interfaces with ifp->if_addr == NULL on bridge interfaces. Moving the notification event handlers up makes sure the interfaces are removed before doing any more cleanup. Sponsored by: Citrix Systems R&D Reviewed by: melifaro Differential Revision: https://reviews.freebsd.org/D598 net/if.c - Move interface removal notification up in if_detach_internal.
2014-08-16 10:47:24 +00:00
/* Announce that the interface is gone. */
rt_ifannouncemsg(ifp, IFAN_DEPARTURE);
EVENTHANDLER_INVOKE(ifnet_departure_event, ifp);
if (IS_DEFAULT_VNET(curvnet))
devctl_notify("IFNET", ifp->if_xname, "DETACH", NULL);
Introduce the if_vmove() function, which will be used in the future for reassigning ifnets from one vnet to another. if_vmove() works by calling a restricted subset of actions normally executed by if_detach() on an ifnet in the current vnet, and then switches to the target vnet and executes an appropriate subset of if_attach() actions there. if_attach() and if_detach() have become wrapper functions around if_attach_internal() and if_detach_internal(), where the later variants have an additional argument, a flag indicating whether a full attach or detach sequence is to be executed, or only a restricted subset suitable for moving an ifnet from one vnet to another. Hence, if_vmove() will not call if_detach() and if_attach() directly, but will call the if_detach_internal() and if_attach_internal() variants instead, with the vmove flag set. While here, staticize ifnet_setbyindex() since it is not referenced from outside of sys/net/if.c. Also rename ifccnt field in struct vimage to ifcnt, and do some minor whitespace garbage collection where appropriate. This change should have no functional impact on nooptions VIMAGE kernel builds. Reviewed by: bz, rwatson, brooks? Approved by: julian (mentor)
2009-05-22 22:09:00 +00:00
if (!vmove) {
/*
* Prevent further calls into the device driver via ifnet.
*/
if_dead(ifp);
Fix a recently introduced panic in if_detach() by delaying the invalidation of ifindex_table[] entry. Probably this code should be moved even further down, but for the time being let's do it this way.
2004-04-19 17:28:15 +00:00
Introduce the if_vmove() function, which will be used in the future for reassigning ifnets from one vnet to another. if_vmove() works by calling a restricted subset of actions normally executed by if_detach() on an ifnet in the current vnet, and then switches to the target vnet and executes an appropriate subset of if_attach() actions there. if_attach() and if_detach() have become wrapper functions around if_attach_internal() and if_detach_internal(), where the later variants have an additional argument, a flag indicating whether a full attach or detach sequence is to be executed, or only a restricted subset suitable for moving an ifnet from one vnet to another. Hence, if_vmove() will not call if_detach() and if_attach() directly, but will call the if_detach_internal() and if_attach_internal() variants instead, with the vmove flag set. While here, staticize ifnet_setbyindex() since it is not referenced from outside of sys/net/if.c. Also rename ifccnt field in struct vimage to ifcnt, and do some minor whitespace garbage collection where appropriate. This change should have no functional impact on nooptions VIMAGE kernel builds. Reviewed by: bz, rwatson, brooks? Approved by: julian (mentor)
2009-05-22 22:09:00 +00:00
/*
* Clean up all addresses.
*/
Similarly to r301505 protect the removal of the ifa from the if_addrhead by a lock (as well as the check that the list is not empty). Obtained from: projects/vnet MFC after: 2 weeks Sponsored by: The FreeBSD Foundation
2016-06-06 16:23:02 +00:00
IF_ADDR_WLOCK(ifp);
ifnet: Replace if_addr_lock rwlock with epoch + mutex Run on LLNW canaries and tested by pho@ gallatin: Using a 14-core, 28-HTT single socket E5-2697 v3 with a 40GbE MLX5 based ConnectX 4-LX NIC, I see an almost 12% improvement in received packet rate, and a larger improvement in bytes delivered all the way to userspace. When the host receiving 64 streams of netperf -H $DUT -t UDP_STREAM -- -m 1, I see, using nstat -I mce0 1 before the patch: InMpps OMpps InGbs OGbs err TCP Est %CPU syscalls csw irq GBfree 4.98 0.00 4.42 0.00 4235592 33 83.80 4720653 2149771 1235 247.32 4.73 0.00 4.20 0.00 4025260 33 82.99 4724900 2139833 1204 247.32 4.72 0.00 4.20 0.00 4035252 33 82.14 4719162 2132023 1264 247.32 4.71 0.00 4.21 0.00 4073206 33 83.68 4744973 2123317 1347 247.32 4.72 0.00 4.21 0.00 4061118 33 80.82 4713615 2188091 1490 247.32 4.72 0.00 4.21 0.00 4051675 33 85.29 4727399 2109011 1205 247.32 4.73 0.00 4.21 0.00 4039056 33 84.65 4724735 2102603 1053 247.32 After the patch InMpps OMpps InGbs OGbs err TCP Est %CPU syscalls csw irq GBfree 5.43 0.00 4.20 0.00 3313143 33 84.96 5434214 1900162 2656 245.51 5.43 0.00 4.20 0.00 3308527 33 85.24 5439695 1809382 2521 245.51 5.42 0.00 4.19 0.00 3316778 33 87.54 5416028 1805835 2256 245.51 5.42 0.00 4.19 0.00 3317673 33 90.44 5426044 1763056 2332 245.51 5.42 0.00 4.19 0.00 3314839 33 88.11 5435732 1792218 2499 245.52 5.44 0.00 4.19 0.00 3293228 33 91.84 5426301 1668597 2121 245.52 Similarly, netperf reports 230Mb/s before the patch, and 270Mb/s after the patch Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15366
2018-05-18 20:13:34 +00:00
if (!CK_STAILQ_EMPTY(&ifp->if_addrhead)) {
ifa = CK_STAILQ_FIRST(&ifp->if_addrhead);
CK_STAILQ_REMOVE(&ifp->if_addrhead, ifa, ifaddr, ifa_link);
Similarly to r301505 protect the removal of the ifa from the if_addrhead by a lock (as well as the check that the list is not empty). Obtained from: projects/vnet MFC after: 2 weeks Sponsored by: The FreeBSD Foundation
2016-06-06 16:23:02 +00:00
IF_ADDR_WUNLOCK(ifp);
Clean up common ifaddr management: - Unify reference count and lock initialization in a single function, ifa_init(). - Move tear-down from a macro (IFAFREE) to a function ifa_free(). - Move reference count bump from a macro (IFAREF) to a function ifa_ref(). - Instead of using a u_int protected by a mutex to refcount(9) for reference count management. The ifa_mtx is now used for exactly one ioctl, and possibly should be removed. MFC after: 3 weeks
2009-06-21 19:30:33 +00:00
ifa_free(ifa);
Similarly to r301505 protect the removal of the ifa from the if_addrhead by a lock (as well as the check that the list is not empty). Obtained from: projects/vnet MFC after: 2 weeks Sponsored by: The FreeBSD Foundation
2016-06-06 16:23:02 +00:00
} else
IF_ADDR_WUNLOCK(ifp);
Do not attempt to clean up data that has not been initialized yet. This fixes two kernel panics on boot when the xl driver fails to allocate bus/port/memory resources. Reviewed by: silence on -net
2004-08-06 09:08:33 +00:00
}
AF_LINK sockaddr has to be attached to ifp->if_addrlist until the end, as many of the code assumes that TAILQ_FIRST(ifp->if_addrlist) is non-null. Submitted by: itojun
2003-10-16 13:38:29 +00:00
MFP r274295: * Move interface route cleanup to route.c:rt_flushifroutes() * Convert most of "for (fibnum = 0; fibnum < rt_numfibs; fibnum++)" users to use new rt_foreach_fib() instead of hand-rolling cycles.
2015-08-08 18:14:59 +00:00
rt_flushifroutes(ifp);
Introduce an interface announcement message for the routing socket so that routing daemons and other interested parties know when an interface is attached/detached. PR: kern/33747 Obtained from: NetBSD MFC after: 2 weeks
2002-01-18 14:33:04 +00:00
After r302054 unloading an network interface driver on a kernel without VIMAGE support would dereference a NULL point unconditionally leading to a panic. Wrap the entire VIMAGE related code with #ifdefs rather than just the decision making part to save an extra bit of resources. Reported by: np Sponsored by: The FreeBSD Foundation MFC After: 13 days Approved by: re (marius)
2016-06-22 11:45:30 +00:00
#ifdef VIMAGE
Get closer to a VIMAGE network stack teardown from top to bottom rather than removing the network interfaces first. This change is rather larger and convoluted as the ordering requirements cannot be separated. Move the pfil(9) framework to SI_SUB_PROTO_PFIL, move Firewalls and related modules to their own SI_SUB_PROTO_FIREWALL. Move initialization of "physical" interfaces to SI_SUB_DRIVERS, move virtual (cloned) interfaces to SI_SUB_PSEUDO. Move Multicast to SI_SUB_PROTO_MC. Re-work parts of multicast initialisation and teardown, not taking the huge amount of memory into account if used as a module yet. For interface teardown we try to do as many of them as we can on SI_SUB_INIT_IF, but for some this makes no sense, e.g., when tunnelling over a higher layer protocol such as IP. In that case the interface has to go along (or before) the higher layer protocol is shutdown. Kernel hhooks need to go last on teardown as they may be used at various higher layers and we cannot remove them before we cleaned up the higher layers. For interface teardown there are multiple paths: (a) a cloned interface is destroyed (inside a VIMAGE or in the base system), (b) any interface is moved from a virtual network stack to a different network stack ("vmove"), or (c) a virtual network stack is being shut down. All code paths go through if_detach_internal() where we, depending on the vmove flag or the vnet state, make a decision on how much to shut down; in case we are destroying a VNET the individual protocol layers will cleanup their own parts thus we cannot do so again for each interface as we end up with, e.g., double-frees, destroying locks twice or acquiring already destroyed locks. When calling into protocol cleanups we equally have to tell them whether they need to detach upper layer protocols ("ulp") or not (e.g., in6_ifdetach()). Provide or enahnce helper functions to do proper cleanup at a protocol rather than at an interface level. Approved by: re (hrs) Obtained from: projects/vnet Reviewed by: gnn, jhb Sponsored by: The FreeBSD Foundation MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D6747
2016-06-21 13:48:49 +00:00
finish_vnet_shutdown:
After r302054 unloading an network interface driver on a kernel without VIMAGE support would dereference a NULL point unconditionally leading to a panic. Wrap the entire VIMAGE related code with #ifdefs rather than just the decision making part to save an extra bit of resources. Reported by: np Sponsored by: The FreeBSD Foundation MFC After: 13 days Approved by: re (marius)
2016-06-22 11:45:30 +00:00
#endif
In if_detach_internal() we cannot hold the af_data lock over the dom_ifdetach() calls as they might sleep for callout_drain(). Do as we do in if_attachdomain1() [r121470] and handle if_afdata_initialized earlier and call dom_ifdetach() unlocked. Discussed with: rwatson MFC after: 10 days
2010-04-11 11:51:44 +00:00
/*
* We cannot hold the lock over dom_ifdetach calls as they might
* sleep, for example trying to drain a callout, thus open up the
* theoretical race with re-attaching.
*/
Since dp->dom_ifattach calls malloc() with M_WAITOK, we cannot use mutex lock directly here. Protect ifp->if_afdata instead. Reported by: grehan
2003-10-24 16:57:59 +00:00
IF_AFDATA_LOCK(ifp);
In if_detach_internal() we cannot hold the af_data lock over the dom_ifdetach() calls as they might sleep for callout_drain(). Do as we do in if_attachdomain1() [r121470] and handle if_afdata_initialized earlier and call dom_ifdetach() unlocked. Discussed with: rwatson MFC after: 10 days
2010-04-11 11:51:44 +00:00
i = ifp->if_afdata_initialized;
ifp->if_afdata_initialized = 0;
IF_AFDATA_UNLOCK(ifp);
for (dp = domains; i > 0 && dp; dp = dp->dom_next) {
After tearing down the interface per-"domain" bits, set the data area to NULL to avoid it being mis-treated on a possible re-attach but also to get a clean NULL pointer derefence in case of errors due to unexpected race conditions elsewhere in the code, e.g., callouts. Obtained from: projects/vnet MFC after: 2 weeks Sponsored by: The FreeBSD Foundation
2016-06-06 22:59:58 +00:00
if (dp->dom_ifdetach && ifp->if_afdata[dp->dom_family]) {
- add dom_if{attach,detach} framework. - transition to use ifp->if_afdata. Obtained from: KAME
2003-10-17 15:46:31 +00:00
(*dp->dom_ifdetach)(ifp,
ifp->if_afdata[dp->dom_family]);
After tearing down the interface per-"domain" bits, set the data area to NULL to avoid it being mis-treated on a possible re-attach but also to get a clean NULL pointer derefence in case of errors due to unexpected race conditions elsewhere in the code, e.g., callouts. Obtained from: projects/vnet MFC after: 2 weeks Sponsored by: The FreeBSD Foundation
2016-06-06 22:59:58 +00:00
ifp->if_afdata[dp->dom_family] = NULL;
}
- add dom_if{attach,detach} framework. - transition to use ifp->if_afdata. Obtained from: KAME
2003-10-17 15:46:31 +00:00
}
If vnets are torn down while ifconfig runs an ioctl to say, destroy an epair(4), we may hit if_detach_internal() without holding a lock and by the time we aquire it the interface might be gone. We should not panic() in this case as it is our fault for not holding the lock all the way. It is not ideal to return silently without error to user space, but other callers will all ignore the return values so do not change the entire KPI for little benefit for now. The ifp will be dealt with one way or another still. Sponsored by: The FreeBSD Foundation MFC after: 2 weeks Reviewed by: gnn Differential Revision: https://reviews.freebsd.org/D4529
2015-12-22 15:03:45 +00:00
return (0);
Introduce the if_vmove() function, which will be used in the future for reassigning ifnets from one vnet to another. if_vmove() works by calling a restricted subset of actions normally executed by if_detach() on an ifnet in the current vnet, and then switches to the target vnet and executes an appropriate subset of if_attach() actions there. if_attach() and if_detach() have become wrapper functions around if_attach_internal() and if_detach_internal(), where the later variants have an additional argument, a flag indicating whether a full attach or detach sequence is to be executed, or only a restricted subset suitable for moving an ifnet from one vnet to another. Hence, if_vmove() will not call if_detach() and if_attach() directly, but will call the if_detach_internal() and if_attach_internal() variants instead, with the vmove flag set. While here, staticize ifnet_setbyindex() since it is not referenced from outside of sys/net/if.c. Also rename ifccnt field in struct vimage to ifcnt, and do some minor whitespace garbage collection where appropriate. This change should have no functional impact on nooptions VIMAGE kernel builds. Reviewed by: bz, rwatson, brooks? Approved by: julian (mentor)
2009-05-22 22:09:00 +00:00
}
Permit buiding kernels with options VIMAGE, restricted to only a single active network stack instance. Turning on options VIMAGE at compile time yields the following changes relative to default kernel build: 1) V_ accessor macros for virtualized variables resolve to structure fields via base pointers, instead of being resolved as fields in global structs or plain global variables. As an example, V_ifnet becomes: options VIMAGE: ((struct vnet_net *) vnet_net)->_ifnet default build: vnet_net_0._ifnet options VIMAGE_GLOBALS: ifnet 2) INIT_VNET_* macros will declare and set up base pointers to be used by V_ accessor macros, instead of resolving to whitespace: INIT_VNET_NET(ifp->if_vnet); becomes struct vnet_net *vnet_net = (ifp->if_vnet)->mod_data[VNET_MOD_NET]; 3) Memory for vnet modules registered via vnet_mod_register() is now allocated at run time in sys/kern/kern_vimage.c, instead of per vnet module structs being declared as globals. If required, vnet modules can now request the framework to provide them with allocated bzeroed memory by filling in the vmi_size field in their vmi_modinfo structures. 4) structs socket, ifnet, inpcbinfo, tcpcb and syncache_head are extended to hold a pointer to the parent vnet. options VIMAGE builds will fill in those fields as required. 5) curvnet is introduced as a new global variable in options VIMAGE builds, always pointing to the default and only struct vnet. 6) struct sysctl_oid has been extended with additional two fields to store major and minor virtualization module identifiers, oid_v_subs and oid_v_mod. SYSCTL_V_* family of macros will fill in those fields accordingly, and store the offset in the appropriate vnet container struct in oid_arg1. In sysctl handlers dealing with virtualized sysctls, the SYSCTL_RESOLVE_V_ARG1() macro will compute the address of the target variable and make it available in arg1 variable for further processing. Unused fields in structs vnet_inet, vnet_inet6 and vnet_ipfw have been deleted. Reviewed by: bz, rwatson Approved by: julian (mentor)
2009-04-30 13:36:26 +00:00
#ifdef VIMAGE
Introduce the if_vmove() function, which will be used in the future for reassigning ifnets from one vnet to another. if_vmove() works by calling a restricted subset of actions normally executed by if_detach() on an ifnet in the current vnet, and then switches to the target vnet and executes an appropriate subset of if_attach() actions there. if_attach() and if_detach() have become wrapper functions around if_attach_internal() and if_detach_internal(), where the later variants have an additional argument, a flag indicating whether a full attach or detach sequence is to be executed, or only a restricted subset suitable for moving an ifnet from one vnet to another. Hence, if_vmove() will not call if_detach() and if_attach() directly, but will call the if_detach_internal() and if_attach_internal() variants instead, with the vmove flag set. While here, staticize ifnet_setbyindex() since it is not referenced from outside of sys/net/if.c. Also rename ifccnt field in struct vimage to ifcnt, and do some minor whitespace garbage collection where appropriate. This change should have no functional impact on nooptions VIMAGE kernel builds. Reviewed by: bz, rwatson, brooks? Approved by: julian (mentor)
2009-05-22 22:09:00 +00:00
/*
* if_vmove() performs a limited version of if_detach() in current
* vnet and if_attach()es the ifnet to the vnet specified as 2nd arg.
* An attempt is made to shrink if_index in current vnet, find an
* unused if_index in target vnet and calls if_grow() if necessary,
* and finally find an unused if_xname for the target vnet.
*/
Get closer to a VIMAGE network stack teardown from top to bottom rather than removing the network interfaces first. This change is rather larger and convoluted as the ordering requirements cannot be separated. Move the pfil(9) framework to SI_SUB_PROTO_PFIL, move Firewalls and related modules to their own SI_SUB_PROTO_FIREWALL. Move initialization of "physical" interfaces to SI_SUB_DRIVERS, move virtual (cloned) interfaces to SI_SUB_PSEUDO. Move Multicast to SI_SUB_PROTO_MC. Re-work parts of multicast initialisation and teardown, not taking the huge amount of memory into account if used as a module yet. For interface teardown we try to do as many of them as we can on SI_SUB_INIT_IF, but for some this makes no sense, e.g., when tunnelling over a higher layer protocol such as IP. In that case the interface has to go along (or before) the higher layer protocol is shutdown. Kernel hhooks need to go last on teardown as they may be used at various higher layers and we cannot remove them before we cleaned up the higher layers. For interface teardown there are multiple paths: (a) a cloned interface is destroyed (inside a VIMAGE or in the base system), (b) any interface is moved from a virtual network stack to a different network stack ("vmove"), or (c) a virtual network stack is being shut down. All code paths go through if_detach_internal() where we, depending on the vmove flag or the vnet state, make a decision on how much to shut down; in case we are destroying a VNET the individual protocol layers will cleanup their own parts thus we cannot do so again for each interface as we end up with, e.g., double-frees, destroying locks twice or acquiring already destroyed locks. When calling into protocol cleanups we equally have to tell them whether they need to detach upper layer protocols ("ulp") or not (e.g., in6_ifdetach()). Provide or enahnce helper functions to do proper cleanup at a protocol rather than at an interface level. Approved by: re (hrs) Obtained from: projects/vnet Reviewed by: gnn, jhb Sponsored by: The FreeBSD Foundation MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D6747
2016-06-21 13:48:49 +00:00
static void
Introduce the if_vmove() function, which will be used in the future for reassigning ifnets from one vnet to another. if_vmove() works by calling a restricted subset of actions normally executed by if_detach() on an ifnet in the current vnet, and then switches to the target vnet and executes an appropriate subset of if_attach() actions there. if_attach() and if_detach() have become wrapper functions around if_attach_internal() and if_detach_internal(), where the later variants have an additional argument, a flag indicating whether a full attach or detach sequence is to be executed, or only a restricted subset suitable for moving an ifnet from one vnet to another. Hence, if_vmove() will not call if_detach() and if_attach() directly, but will call the if_detach_internal() and if_attach_internal() variants instead, with the vmove flag set. While here, staticize ifnet_setbyindex() since it is not referenced from outside of sys/net/if.c. Also rename ifccnt field in struct vimage to ifcnt, and do some minor whitespace garbage collection where appropriate. This change should have no functional impact on nooptions VIMAGE kernel builds. Reviewed by: bz, rwatson, brooks? Approved by: julian (mentor)
2009-05-22 22:09:00 +00:00
if_vmove(struct ifnet *ifp, struct vnet *new_vnet)
{
Fix group membership of cloned interfaces when one is moved by if_vmove(). In if_vmove(), if_detach_internal() and if_attach_internal() were called in series to detach and reattach the interface. When detaching, if_delgroup() was called and the interface leaves all of the group membership. And then upon attachment, if_addgroup(ifp, IFG_ALL) was called and it joined only "all" group again. This had a problem. Normally, a cloned interface automatically joins a group whose name is ifc_name of the cloner in addition to "all" upon creation. However, if_vmove() removed the membership and did not restore upon attachment. Differential Revision: https://reviews.freebsd.org/D1859
2015-03-02 20:00:03 +00:00
struct if_clone *ifc;
During if_vmove() we call if_detach_internal() which in turn calls the event handler notifying about interface departure and one of the consumers will detach if_bpf. There is no way for us to re-attach this easily as the DLT and hdrlen are only given on interface creation. Add a function to allow us to query the DLT and hdrlen from a current BPF attachment and after if_attach_internal() manually re-add the if_bpf attachment using these values. Found by panics triggered by nd6 packets running past BPF_MTAP() with no proper if_bpf pointer on the interface. Also add a basic DDB show function to investigate the if_bpf attachment of an interface. Reviewed by: gnn MFC after: 2 weeks Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D5896
2016-04-11 10:00:38 +00:00
u_int bif_dlt, bif_hdrlen;
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
void *old;
If vnets are torn down while ifconfig runs an ioctl to say, destroy an epair(4), we may hit if_detach_internal() without holding a lock and by the time we aquire it the interface might be gone. We should not panic() in this case as it is our fault for not holding the lock all the way. It is not ideal to return silently without error to user space, but other callers will all ignore the return values so do not change the entire KPI for little benefit for now. The ifp will be dealt with one way or another still. Sponsored by: The FreeBSD Foundation MFC after: 2 weeks Reviewed by: gnn Differential Revision: https://reviews.freebsd.org/D4529
2015-12-22 15:03:45 +00:00
int rc;
Introduce the if_vmove() function, which will be used in the future for reassigning ifnets from one vnet to another. if_vmove() works by calling a restricted subset of actions normally executed by if_detach() on an ifnet in the current vnet, and then switches to the target vnet and executes an appropriate subset of if_attach() actions there. if_attach() and if_detach() have become wrapper functions around if_attach_internal() and if_detach_internal(), where the later variants have an additional argument, a flag indicating whether a full attach or detach sequence is to be executed, or only a restricted subset suitable for moving an ifnet from one vnet to another. Hence, if_vmove() will not call if_detach() and if_attach() directly, but will call the if_detach_internal() and if_attach_internal() variants instead, with the vmove flag set. While here, staticize ifnet_setbyindex() since it is not referenced from outside of sys/net/if.c. Also rename ifccnt field in struct vimage to ifcnt, and do some minor whitespace garbage collection where appropriate. This change should have no functional impact on nooptions VIMAGE kernel builds. Reviewed by: bz, rwatson, brooks? Approved by: julian (mentor)
2009-05-22 22:09:00 +00:00
During if_vmove() we call if_detach_internal() which in turn calls the event handler notifying about interface departure and one of the consumers will detach if_bpf. There is no way for us to re-attach this easily as the DLT and hdrlen are only given on interface creation. Add a function to allow us to query the DLT and hdrlen from a current BPF attachment and after if_attach_internal() manually re-add the if_bpf attachment using these values. Found by panics triggered by nd6 packets running past BPF_MTAP() with no proper if_bpf pointer on the interface. Also add a basic DDB show function to investigate the if_bpf attachment of an interface. Reviewed by: gnn MFC after: 2 weeks Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D5896
2016-04-11 10:00:38 +00:00
/*
* if_detach_internal() will call the eventhandler to notify
* interface departure. That will detach if_bpf. We need to
* safe the dlt and hdrlen so we can re-attach it later.
*/
bpf_get_bp_params(ifp->if_bpf, &bif_dlt, &bif_hdrlen);
Introduce the if_vmove() function, which will be used in the future for reassigning ifnets from one vnet to another. if_vmove() works by calling a restricted subset of actions normally executed by if_detach() on an ifnet in the current vnet, and then switches to the target vnet and executes an appropriate subset of if_attach() actions there. if_attach() and if_detach() have become wrapper functions around if_attach_internal() and if_detach_internal(), where the later variants have an additional argument, a flag indicating whether a full attach or detach sequence is to be executed, or only a restricted subset suitable for moving an ifnet from one vnet to another. Hence, if_vmove() will not call if_detach() and if_attach() directly, but will call the if_detach_internal() and if_attach_internal() variants instead, with the vmove flag set. While here, staticize ifnet_setbyindex() since it is not referenced from outside of sys/net/if.c. Also rename ifccnt field in struct vimage to ifcnt, and do some minor whitespace garbage collection where appropriate. This change should have no functional impact on nooptions VIMAGE kernel builds. Reviewed by: bz, rwatson, brooks? Approved by: julian (mentor)
2009-05-22 22:09:00 +00:00
/*
* Detach from current vnet, but preserve LLADDR info, do not
* mark as dead etc. so that the ifnet can be reattached later.
If vnets are torn down while ifconfig runs an ioctl to say, destroy an epair(4), we may hit if_detach_internal() without holding a lock and by the time we aquire it the interface might be gone. We should not panic() in this case as it is our fault for not holding the lock all the way. It is not ideal to return silently without error to user space, but other callers will all ignore the return values so do not change the entire KPI for little benefit for now. The ifp will be dealt with one way or another still. Sponsored by: The FreeBSD Foundation MFC after: 2 weeks Reviewed by: gnn Differential Revision: https://reviews.freebsd.org/D4529
2015-12-22 15:03:45 +00:00
* If we cannot find it, we lost the race to someone else.
Introduce the if_vmove() function, which will be used in the future for reassigning ifnets from one vnet to another. if_vmove() works by calling a restricted subset of actions normally executed by if_detach() on an ifnet in the current vnet, and then switches to the target vnet and executes an appropriate subset of if_attach() actions there. if_attach() and if_detach() have become wrapper functions around if_attach_internal() and if_detach_internal(), where the later variants have an additional argument, a flag indicating whether a full attach or detach sequence is to be executed, or only a restricted subset suitable for moving an ifnet from one vnet to another. Hence, if_vmove() will not call if_detach() and if_attach() directly, but will call the if_detach_internal() and if_attach_internal() variants instead, with the vmove flag set. While here, staticize ifnet_setbyindex() since it is not referenced from outside of sys/net/if.c. Also rename ifccnt field in struct vimage to ifcnt, and do some minor whitespace garbage collection where appropriate. This change should have no functional impact on nooptions VIMAGE kernel builds. Reviewed by: bz, rwatson, brooks? Approved by: julian (mentor)
2009-05-22 22:09:00 +00:00
*/
If vnets are torn down while ifconfig runs an ioctl to say, destroy an epair(4), we may hit if_detach_internal() without holding a lock and by the time we aquire it the interface might be gone. We should not panic() in this case as it is our fault for not holding the lock all the way. It is not ideal to return silently without error to user space, but other callers will all ignore the return values so do not change the entire KPI for little benefit for now. The ifp will be dealt with one way or another still. Sponsored by: The FreeBSD Foundation MFC after: 2 weeks Reviewed by: gnn Differential Revision: https://reviews.freebsd.org/D4529
2015-12-22 15:03:45 +00:00
rc = if_detach_internal(ifp, 1, &ifc);
if (rc != 0)
return;
Introduce the if_vmove() function, which will be used in the future for reassigning ifnets from one vnet to another. if_vmove() works by calling a restricted subset of actions normally executed by if_detach() on an ifnet in the current vnet, and then switches to the target vnet and executes an appropriate subset of if_attach() actions there. if_attach() and if_detach() have become wrapper functions around if_attach_internal() and if_detach_internal(), where the later variants have an additional argument, a flag indicating whether a full attach or detach sequence is to be executed, or only a restricted subset suitable for moving an ifnet from one vnet to another. Hence, if_vmove() will not call if_detach() and if_attach() directly, but will call the if_detach_internal() and if_attach_internal() variants instead, with the vmove flag set. While here, staticize ifnet_setbyindex() since it is not referenced from outside of sys/net/if.c. Also rename ifccnt field in struct vimage to ifcnt, and do some minor whitespace garbage collection where appropriate. This change should have no functional impact on nooptions VIMAGE kernel builds. Reviewed by: bz, rwatson, brooks? Approved by: julian (mentor)
2009-05-22 22:09:00 +00:00
/*
Rework global locks for interface list and index management, correcting several critical bugs, including race conditions and lock order issues: Replace the single rwlock, ifnet_lock, with two locks, an rwlock and an sxlock. Either can be held to stablize the lists and indexes, but both are required to write. This allows the list to be held stable in both network interrupt contexts and sleepable user threads across sleeping memory allocations or device driver interactions. As before, writes to the interface list must occur from sleepable contexts. Reviewed by: bz, julian MFC after: 3 days
2009-08-23 20:40:19 +00:00
* Unlink the ifnet from ifindex_table[] in current vnet, and shrink
* the if_index for that vnet if possible.
*
* NOTE: IFNET_WLOCK/IFNET_WUNLOCK() are assumed to be unvirtualized,
* or we'd lock on one vnet and unlock on another.
Introduce the if_vmove() function, which will be used in the future for reassigning ifnets from one vnet to another. if_vmove() works by calling a restricted subset of actions normally executed by if_detach() on an ifnet in the current vnet, and then switches to the target vnet and executes an appropriate subset of if_attach() actions there. if_attach() and if_detach() have become wrapper functions around if_attach_internal() and if_detach_internal(), where the later variants have an additional argument, a flag indicating whether a full attach or detach sequence is to be executed, or only a restricted subset suitable for moving an ifnet from one vnet to another. Hence, if_vmove() will not call if_detach() and if_attach() directly, but will call the if_detach_internal() and if_attach_internal() variants instead, with the vmove flag set. While here, staticize ifnet_setbyindex() since it is not referenced from outside of sys/net/if.c. Also rename ifccnt field in struct vimage to ifcnt, and do some minor whitespace garbage collection where appropriate. This change should have no functional impact on nooptions VIMAGE kernel builds. Reviewed by: bz, rwatson, brooks? Approved by: julian (mentor)
2009-05-22 22:09:00 +00:00
*/
Build on Jeff Roberson's linker-set based dynamic per-CPU allocator (DPCPU), as suggested by Peter Wemm, and implement a new per-virtual network stack memory allocator. Modify vnet to use the allocator instead of monolithic global container structures (vinet, ...). This change solves many binary compatibility problems associated with VIMAGE, and restores ELF symbols for virtualized global variables. Each virtualized global variable exists as a "reference copy", and also once per virtual network stack. Virtualized global variables are tagged at compile-time, placing the in a special linker set, which is loaded into a contiguous region of kernel memory. Virtualized global variables in the base kernel are linked as normal, but those in modules are copied and relocated to a reserved portion of the kernel's vnet region with the help of a the kernel linker. Virtualized global variables exist in per-vnet memory set up when the network stack instance is created, and are initialized statically from the reference copy. Run-time access occurs via an accessor macro, which converts from the current vnet and requested symbol to a per-vnet address. When "options VIMAGE" is not compiled into the kernel, normal global ELF symbols will be used instead and indirection is avoided. This change restores static initialization for network stack global variables, restores support for non-global symbols and types, eliminates the need for many subsystem constructors, eliminates large per-subsystem structures that caused many binary compatibility issues both for monitoring applications (netstat) and kernel modules, removes the per-function INIT_VNET_*() macros throughout the stack, eliminates the need for vnet_symmap ksym(2) munging, and eliminates duplicate definitions of virtualized globals under VIMAGE_GLOBALS. Bump __FreeBSD_version and update UPDATING. Portions submitted by: bz Reviewed by: bz, zec Discussed with: gnn, jamie, jeff, jhb, julian, sam Suggested by: peter Approved by: re (kensmith)
2009-07-14 22:48:30 +00:00
IFNET_WLOCK();
Add IFNET_HOLD reserved pointer value for the ifindex ifnet array, which allows an index to be reserved for an ifnet without making the ifnet available for management operations. Use this in if_alloc() while the ifnet lock is released between initial index allocation and completion of ifnet initialization. Add ifindex_free() to centralize the implementation of releasing an ifindex value. Use in if_free() and if_vmove(), as well as when releasing a held index in if_alloc(). Reviewed by: bz MFC after: 3 days
2009-08-26 11:13:10 +00:00
ifindex_free_locked(ifp->if_index);
When moving an ethernet ifnet from one vnet to another, destroy the associated ng_ether netgraph node in the current vnet, and create a new one in the target vnet. Reviewed by: julian MFC after: 3 days
2010-08-13 18:17:32 +00:00
IFNET_WUNLOCK();
/*
* Perform interface-specific reassignment tasks, if provided by
* the driver.
*/
if (ifp->if_reassign != NULL)
ifp->if_reassign(ifp, new_vnet, NULL);
Introduce the if_vmove() function, which will be used in the future for reassigning ifnets from one vnet to another. if_vmove() works by calling a restricted subset of actions normally executed by if_detach() on an ifnet in the current vnet, and then switches to the target vnet and executes an appropriate subset of if_attach() actions there. if_attach() and if_detach() have become wrapper functions around if_attach_internal() and if_detach_internal(), where the later variants have an additional argument, a flag indicating whether a full attach or detach sequence is to be executed, or only a restricted subset suitable for moving an ifnet from one vnet to another. Hence, if_vmove() will not call if_detach() and if_attach() directly, but will call the if_detach_internal() and if_attach_internal() variants instead, with the vmove flag set. While here, staticize ifnet_setbyindex() since it is not referenced from outside of sys/net/if.c. Also rename ifccnt field in struct vimage to ifcnt, and do some minor whitespace garbage collection where appropriate. This change should have no functional impact on nooptions VIMAGE kernel builds. Reviewed by: bz, rwatson, brooks? Approved by: julian (mentor)
2009-05-22 22:09:00 +00:00
/*
* Switch to the context of the target vnet.
*/
CURVNET_SET_QUIET(new_vnet);
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
restart:
When moving an ethernet ifnet from one vnet to another, destroy the associated ng_ether netgraph node in the current vnet, and create a new one in the target vnet. Reviewed by: julian MFC after: 3 days
2010-08-13 18:17:32 +00:00
IFNET_WLOCK();
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
ifp->if_index = ifindex_alloc(&old);
if (__predict_false(ifp->if_index == USHRT_MAX)) {
IFNET_WUNLOCK();
epoch_wait_preempt(net_epoch_preempt);
free(old, M_IFNET);
goto restart;
}
ifnet_setbyindex(ifp->if_index, ifp);
Introduce the if_vmove() function, which will be used in the future for reassigning ifnets from one vnet to another. if_vmove() works by calling a restricted subset of actions normally executed by if_detach() on an ifnet in the current vnet, and then switches to the target vnet and executes an appropriate subset of if_attach() actions there. if_attach() and if_detach() have become wrapper functions around if_attach_internal() and if_detach_internal(), where the later variants have an additional argument, a flag indicating whether a full attach or detach sequence is to be executed, or only a restricted subset suitable for moving an ifnet from one vnet to another. Hence, if_vmove() will not call if_detach() and if_attach() directly, but will call the if_detach_internal() and if_attach_internal() variants instead, with the vmove flag set. While here, staticize ifnet_setbyindex() since it is not referenced from outside of sys/net/if.c. Also rename ifccnt field in struct vimage to ifcnt, and do some minor whitespace garbage collection where appropriate. This change should have no functional impact on nooptions VIMAGE kernel builds. Reviewed by: bz, rwatson, brooks? Approved by: julian (mentor)
2009-05-22 22:09:00 +00:00
IFNET_WUNLOCK();
Fix group membership of cloned interfaces when one is moved by if_vmove(). In if_vmove(), if_detach_internal() and if_attach_internal() were called in series to detach and reattach the interface. When detaching, if_delgroup() was called and the interface leaves all of the group membership. And then upon attachment, if_addgroup(ifp, IFG_ALL) was called and it joined only "all" group again. This had a problem. Normally, a cloned interface automatically joins a group whose name is ifc_name of the cloner in addition to "all" upon creation. However, if_vmove() removed the membership and did not restore upon attachment. Differential Revision: https://reviews.freebsd.org/D1859
2015-03-02 20:00:03 +00:00
if_attach_internal(ifp, 1, ifc);
Introduce the if_vmove() function, which will be used in the future for reassigning ifnets from one vnet to another. if_vmove() works by calling a restricted subset of actions normally executed by if_detach() on an ifnet in the current vnet, and then switches to the target vnet and executes an appropriate subset of if_attach() actions there. if_attach() and if_detach() have become wrapper functions around if_attach_internal() and if_detach_internal(), where the later variants have an additional argument, a flag indicating whether a full attach or detach sequence is to be executed, or only a restricted subset suitable for moving an ifnet from one vnet to another. Hence, if_vmove() will not call if_detach() and if_attach() directly, but will call the if_detach_internal() and if_attach_internal() variants instead, with the vmove flag set. While here, staticize ifnet_setbyindex() since it is not referenced from outside of sys/net/if.c. Also rename ifccnt field in struct vimage to ifcnt, and do some minor whitespace garbage collection where appropriate. This change should have no functional impact on nooptions VIMAGE kernel builds. Reviewed by: bz, rwatson, brooks? Approved by: julian (mentor)
2009-05-22 22:09:00 +00:00
During if_vmove() we call if_detach_internal() which in turn calls the event handler notifying about interface departure and one of the consumers will detach if_bpf. There is no way for us to re-attach this easily as the DLT and hdrlen are only given on interface creation. Add a function to allow us to query the DLT and hdrlen from a current BPF attachment and after if_attach_internal() manually re-add the if_bpf attachment using these values. Found by panics triggered by nd6 packets running past BPF_MTAP() with no proper if_bpf pointer on the interface. Also add a basic DDB show function to investigate the if_bpf attachment of an interface. Reviewed by: gnn MFC after: 2 weeks Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D5896
2016-04-11 10:00:38 +00:00
if (ifp->if_bpf == NULL)
bpfattach(ifp, bif_dlt, bif_hdrlen);
Introduce the if_vmove() function, which will be used in the future for reassigning ifnets from one vnet to another. if_vmove() works by calling a restricted subset of actions normally executed by if_detach() on an ifnet in the current vnet, and then switches to the target vnet and executes an appropriate subset of if_attach() actions there. if_attach() and if_detach() have become wrapper functions around if_attach_internal() and if_detach_internal(), where the later variants have an additional argument, a flag indicating whether a full attach or detach sequence is to be executed, or only a restricted subset suitable for moving an ifnet from one vnet to another. Hence, if_vmove() will not call if_detach() and if_attach() directly, but will call the if_detach_internal() and if_attach_internal() variants instead, with the vmove flag set. While here, staticize ifnet_setbyindex() since it is not referenced from outside of sys/net/if.c. Also rename ifccnt field in struct vimage to ifcnt, and do some minor whitespace garbage collection where appropriate. This change should have no functional impact on nooptions VIMAGE kernel builds. Reviewed by: bz, rwatson, brooks? Approved by: julian (mentor)
2009-05-22 22:09:00 +00:00
CURVNET_RESTORE();
Two more fixes to if_detach. These are generic to all interfaces and do not pollute the interface further. o Run if_detach at splnet(). o Creatively swipe the relevant parts of the netatm atm_nif_detach which will delete the relevant references to the interface going away.
1999-12-17 06:46:07 +00:00
}
Make the in-kernel logic for the SIOCSIFVNET, SIOCSIFRVNET ioctls (ifconfig ifN (-)vnet <jname|jid>) work correctly. Move vi_if_move to if.c and split it up into two functions(*), one for each ioctl. In the reclaim case, correctly set the vnet before calling if_vmove. Instead of silently allowing a move of an interface from the current vnet to the current vnet, return an error. (*) There is some duplicate interface name checking before actually moving the interface between network stacks without locking and thus race prone. Ideally if_vmove will correctly and automagically handle these in the future. Suggested by: rwatson (*) Approved by: re (kib)
2009-07-26 11:29:26 +00:00
/*
* Move an ifnet to or from another child prison/vnet, specified by the jail id.
*/
static int
if_vmove_loan(struct thread *td, struct ifnet *ifp, char *ifname, int jid)
{
struct prison *pr;
struct ifnet *difp;
/* Try to find the prison within our visibility. */
sx_slock(&allprison_lock);
pr = prison_find_child(td->td_ucred->cr_prison, jid);
sx_sunlock(&allprison_lock);
if (pr == NULL)
return (ENXIO);
prison_hold_locked(pr);
mtx_unlock(&pr->pr_mtx);
/* Do not try to move the iface from and to the same prison. */
if (pr->pr_vnet == ifp->if_vnet) {
prison_free(pr);
return (EEXIST);
}
/* Make sure the named iface does not exists in the dst. prison/vnet. */
/* XXX Lock interfaces to avoid races. */
Appease VNET_DEBUG - in if_vmove we temporarily switch i.e. recurse from one vnet to another which is OK, so no need to flood the console with warnings here. Approved by: re (rwatson), julian (mentor)
2009-08-14 22:46:45 +00:00
CURVNET_SET_QUIET(pr->pr_vnet);
Make the in-kernel logic for the SIOCSIFVNET, SIOCSIFRVNET ioctls (ifconfig ifN (-)vnet <jname|jid>) work correctly. Move vi_if_move to if.c and split it up into two functions(*), one for each ioctl. In the reclaim case, correctly set the vnet before calling if_vmove. Instead of silently allowing a move of an interface from the current vnet to the current vnet, return an error. (*) There is some duplicate interface name checking before actually moving the interface between network stacks without locking and thus race prone. Ideally if_vmove will correctly and automagically handle these in the future. Suggested by: rwatson (*) Approved by: re (kib)
2009-07-26 11:29:26 +00:00
difp = ifunit(ifname);
if (difp != NULL) {
Get closer to a VIMAGE network stack teardown from top to bottom rather than removing the network interfaces first. This change is rather larger and convoluted as the ordering requirements cannot be separated. Move the pfil(9) framework to SI_SUB_PROTO_PFIL, move Firewalls and related modules to their own SI_SUB_PROTO_FIREWALL. Move initialization of "physical" interfaces to SI_SUB_DRIVERS, move virtual (cloned) interfaces to SI_SUB_PSEUDO. Move Multicast to SI_SUB_PROTO_MC. Re-work parts of multicast initialisation and teardown, not taking the huge amount of memory into account if used as a module yet. For interface teardown we try to do as many of them as we can on SI_SUB_INIT_IF, but for some this makes no sense, e.g., when tunnelling over a higher layer protocol such as IP. In that case the interface has to go along (or before) the higher layer protocol is shutdown. Kernel hhooks need to go last on teardown as they may be used at various higher layers and we cannot remove them before we cleaned up the higher layers. For interface teardown there are multiple paths: (a) a cloned interface is destroyed (inside a VIMAGE or in the base system), (b) any interface is moved from a virtual network stack to a different network stack ("vmove"), or (c) a virtual network stack is being shut down. All code paths go through if_detach_internal() where we, depending on the vmove flag or the vnet state, make a decision on how much to shut down; in case we are destroying a VNET the individual protocol layers will cleanup their own parts thus we cannot do so again for each interface as we end up with, e.g., double-frees, destroying locks twice or acquiring already destroyed locks. When calling into protocol cleanups we equally have to tell them whether they need to detach upper layer protocols ("ulp") or not (e.g., in6_ifdetach()). Provide or enahnce helper functions to do proper cleanup at a protocol rather than at an interface level. Approved by: re (hrs) Obtained from: projects/vnet Reviewed by: gnn, jhb Sponsored by: The FreeBSD Foundation MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D6747
2016-06-21 13:48:49 +00:00
CURVNET_RESTORE();
Make the in-kernel logic for the SIOCSIFVNET, SIOCSIFRVNET ioctls (ifconfig ifN (-)vnet <jname|jid>) work correctly. Move vi_if_move to if.c and split it up into two functions(*), one for each ioctl. In the reclaim case, correctly set the vnet before calling if_vmove. Instead of silently allowing a move of an interface from the current vnet to the current vnet, return an error. (*) There is some duplicate interface name checking before actually moving the interface between network stacks without locking and thus race prone. Ideally if_vmove will correctly and automagically handle these in the future. Suggested by: rwatson (*) Approved by: re (kib)
2009-07-26 11:29:26 +00:00
prison_free(pr);
return (EEXIST);
}
Get closer to a VIMAGE network stack teardown from top to bottom rather than removing the network interfaces first. This change is rather larger and convoluted as the ordering requirements cannot be separated. Move the pfil(9) framework to SI_SUB_PROTO_PFIL, move Firewalls and related modules to their own SI_SUB_PROTO_FIREWALL. Move initialization of "physical" interfaces to SI_SUB_DRIVERS, move virtual (cloned) interfaces to SI_SUB_PSEUDO. Move Multicast to SI_SUB_PROTO_MC. Re-work parts of multicast initialisation and teardown, not taking the huge amount of memory into account if used as a module yet. For interface teardown we try to do as many of them as we can on SI_SUB_INIT_IF, but for some this makes no sense, e.g., when tunnelling over a higher layer protocol such as IP. In that case the interface has to go along (or before) the higher layer protocol is shutdown. Kernel hhooks need to go last on teardown as they may be used at various higher layers and we cannot remove them before we cleaned up the higher layers. For interface teardown there are multiple paths: (a) a cloned interface is destroyed (inside a VIMAGE or in the base system), (b) any interface is moved from a virtual network stack to a different network stack ("vmove"), or (c) a virtual network stack is being shut down. All code paths go through if_detach_internal() where we, depending on the vmove flag or the vnet state, make a decision on how much to shut down; in case we are destroying a VNET the individual protocol layers will cleanup their own parts thus we cannot do so again for each interface as we end up with, e.g., double-frees, destroying locks twice or acquiring already destroyed locks. When calling into protocol cleanups we equally have to tell them whether they need to detach upper layer protocols ("ulp") or not (e.g., in6_ifdetach()). Provide or enahnce helper functions to do proper cleanup at a protocol rather than at an interface level. Approved by: re (hrs) Obtained from: projects/vnet Reviewed by: gnn, jhb Sponsored by: The FreeBSD Foundation MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D6747
2016-06-21 13:48:49 +00:00
/* Make sure the VNET is stable. */
Factor out VNET shutdown check into an own vnet structure field. Remove the now obsolete vnet_state field. This greatly simplifies the detection of VNET shutdown and avoids code duplication. Discussed with: bz@ MFC after: 1 week Sponsored by: Mellanox Technologies
2019-10-07 14:15:41 +00:00
if (ifp->if_vnet->vnet_shutdown) {
Get closer to a VIMAGE network stack teardown from top to bottom rather than removing the network interfaces first. This change is rather larger and convoluted as the ordering requirements cannot be separated. Move the pfil(9) framework to SI_SUB_PROTO_PFIL, move Firewalls and related modules to their own SI_SUB_PROTO_FIREWALL. Move initialization of "physical" interfaces to SI_SUB_DRIVERS, move virtual (cloned) interfaces to SI_SUB_PSEUDO. Move Multicast to SI_SUB_PROTO_MC. Re-work parts of multicast initialisation and teardown, not taking the huge amount of memory into account if used as a module yet. For interface teardown we try to do as many of them as we can on SI_SUB_INIT_IF, but for some this makes no sense, e.g., when tunnelling over a higher layer protocol such as IP. In that case the interface has to go along (or before) the higher layer protocol is shutdown. Kernel hhooks need to go last on teardown as they may be used at various higher layers and we cannot remove them before we cleaned up the higher layers. For interface teardown there are multiple paths: (a) a cloned interface is destroyed (inside a VIMAGE or in the base system), (b) any interface is moved from a virtual network stack to a different network stack ("vmove"), or (c) a virtual network stack is being shut down. All code paths go through if_detach_internal() where we, depending on the vmove flag or the vnet state, make a decision on how much to shut down; in case we are destroying a VNET the individual protocol layers will cleanup their own parts thus we cannot do so again for each interface as we end up with, e.g., double-frees, destroying locks twice or acquiring already destroyed locks. When calling into protocol cleanups we equally have to tell them whether they need to detach upper layer protocols ("ulp") or not (e.g., in6_ifdetach()). Provide or enahnce helper functions to do proper cleanup at a protocol rather than at an interface level. Approved by: re (hrs) Obtained from: projects/vnet Reviewed by: gnn, jhb Sponsored by: The FreeBSD Foundation MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D6747
2016-06-21 13:48:49 +00:00
CURVNET_RESTORE();
prison_free(pr);
return (EBUSY);
}
CURVNET_RESTORE();
Make the in-kernel logic for the SIOCSIFVNET, SIOCSIFRVNET ioctls (ifconfig ifN (-)vnet <jname|jid>) work correctly. Move vi_if_move to if.c and split it up into two functions(*), one for each ioctl. In the reclaim case, correctly set the vnet before calling if_vmove. Instead of silently allowing a move of an interface from the current vnet to the current vnet, return an error. (*) There is some duplicate interface name checking before actually moving the interface between network stacks without locking and thus race prone. Ideally if_vmove will correctly and automagically handle these in the future. Suggested by: rwatson (*) Approved by: re (kib)
2009-07-26 11:29:26 +00:00
/* Move the interface into the child jail/vnet. */
if_vmove(ifp, pr->pr_vnet);
/* Report the new if_xname back to the userland. */
sprintf(ifname, "%s", ifp->if_xname);
prison_free(pr);
return (0);
}
static int
if_vmove_reclaim(struct thread *td, char *ifname, int jid)
{
struct prison *pr;
struct vnet *vnet_dst;
struct ifnet *ifp;
/* Try to find the prison within our visibility. */
sx_slock(&allprison_lock);
pr = prison_find_child(td->td_ucred->cr_prison, jid);
sx_sunlock(&allprison_lock);
if (pr == NULL)
return (ENXIO);
prison_hold_locked(pr);
mtx_unlock(&pr->pr_mtx);
/* Make sure the named iface exists in the source prison/vnet. */
CURVNET_SET(pr->pr_vnet);
ifp = ifunit(ifname); /* XXX Lock to avoid races. */
if (ifp == NULL) {
CURVNET_RESTORE();
prison_free(pr);
return (ENXIO);
}
/* Do not try to move the iface from and to the same prison. */
vnet_dst = TD_TO_VNET(td);
if (vnet_dst == ifp->if_vnet) {
CURVNET_RESTORE();
prison_free(pr);
return (EEXIST);
}
Get closer to a VIMAGE network stack teardown from top to bottom rather than removing the network interfaces first. This change is rather larger and convoluted as the ordering requirements cannot be separated. Move the pfil(9) framework to SI_SUB_PROTO_PFIL, move Firewalls and related modules to their own SI_SUB_PROTO_FIREWALL. Move initialization of "physical" interfaces to SI_SUB_DRIVERS, move virtual (cloned) interfaces to SI_SUB_PSEUDO. Move Multicast to SI_SUB_PROTO_MC. Re-work parts of multicast initialisation and teardown, not taking the huge amount of memory into account if used as a module yet. For interface teardown we try to do as many of them as we can on SI_SUB_INIT_IF, but for some this makes no sense, e.g., when tunnelling over a higher layer protocol such as IP. In that case the interface has to go along (or before) the higher layer protocol is shutdown. Kernel hhooks need to go last on teardown as they may be used at various higher layers and we cannot remove them before we cleaned up the higher layers. For interface teardown there are multiple paths: (a) a cloned interface is destroyed (inside a VIMAGE or in the base system), (b) any interface is moved from a virtual network stack to a different network stack ("vmove"), or (c) a virtual network stack is being shut down. All code paths go through if_detach_internal() where we, depending on the vmove flag or the vnet state, make a decision on how much to shut down; in case we are destroying a VNET the individual protocol layers will cleanup their own parts thus we cannot do so again for each interface as we end up with, e.g., double-frees, destroying locks twice or acquiring already destroyed locks. When calling into protocol cleanups we equally have to tell them whether they need to detach upper layer protocols ("ulp") or not (e.g., in6_ifdetach()). Provide or enahnce helper functions to do proper cleanup at a protocol rather than at an interface level. Approved by: re (hrs) Obtained from: projects/vnet Reviewed by: gnn, jhb Sponsored by: The FreeBSD Foundation MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D6747
2016-06-21 13:48:49 +00:00
/* Make sure the VNET is stable. */
Factor out VNET shutdown check into an own vnet structure field. Remove the now obsolete vnet_state field. This greatly simplifies the detection of VNET shutdown and avoids code duplication. Discussed with: bz@ MFC after: 1 week Sponsored by: Mellanox Technologies
2019-10-07 14:15:41 +00:00
if (ifp->if_vnet->vnet_shutdown) {
Get closer to a VIMAGE network stack teardown from top to bottom rather than removing the network interfaces first. This change is rather larger and convoluted as the ordering requirements cannot be separated. Move the pfil(9) framework to SI_SUB_PROTO_PFIL, move Firewalls and related modules to their own SI_SUB_PROTO_FIREWALL. Move initialization of "physical" interfaces to SI_SUB_DRIVERS, move virtual (cloned) interfaces to SI_SUB_PSEUDO. Move Multicast to SI_SUB_PROTO_MC. Re-work parts of multicast initialisation and teardown, not taking the huge amount of memory into account if used as a module yet. For interface teardown we try to do as many of them as we can on SI_SUB_INIT_IF, but for some this makes no sense, e.g., when tunnelling over a higher layer protocol such as IP. In that case the interface has to go along (or before) the higher layer protocol is shutdown. Kernel hhooks need to go last on teardown as they may be used at various higher layers and we cannot remove them before we cleaned up the higher layers. For interface teardown there are multiple paths: (a) a cloned interface is destroyed (inside a VIMAGE or in the base system), (b) any interface is moved from a virtual network stack to a different network stack ("vmove"), or (c) a virtual network stack is being shut down. All code paths go through if_detach_internal() where we, depending on the vmove flag or the vnet state, make a decision on how much to shut down; in case we are destroying a VNET the individual protocol layers will cleanup their own parts thus we cannot do so again for each interface as we end up with, e.g., double-frees, destroying locks twice or acquiring already destroyed locks. When calling into protocol cleanups we equally have to tell them whether they need to detach upper layer protocols ("ulp") or not (e.g., in6_ifdetach()). Provide or enahnce helper functions to do proper cleanup at a protocol rather than at an interface level. Approved by: re (hrs) Obtained from: projects/vnet Reviewed by: gnn, jhb Sponsored by: The FreeBSD Foundation MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D6747
2016-06-21 13:48:49 +00:00
CURVNET_RESTORE();
prison_free(pr);
return (EBUSY);
}
Make the in-kernel logic for the SIOCSIFVNET, SIOCSIFRVNET ioctls (ifconfig ifN (-)vnet <jname|jid>) work correctly. Move vi_if_move to if.c and split it up into two functions(*), one for each ioctl. In the reclaim case, correctly set the vnet before calling if_vmove. Instead of silently allowing a move of an interface from the current vnet to the current vnet, return an error. (*) There is some duplicate interface name checking before actually moving the interface between network stacks without locking and thus race prone. Ideally if_vmove will correctly and automagically handle these in the future. Suggested by: rwatson (*) Approved by: re (kib)
2009-07-26 11:29:26 +00:00
/* Get interface back from child jail/vnet. */
if_vmove(ifp, vnet_dst);
CURVNET_RESTORE();
/* Report the new if_xname back to the userland. */
sprintf(ifname, "%s", ifp->if_xname);
prison_free(pr);
return (0);
}
Introduce the if_vmove() function, which will be used in the future for reassigning ifnets from one vnet to another. if_vmove() works by calling a restricted subset of actions normally executed by if_detach() on an ifnet in the current vnet, and then switches to the target vnet and executes an appropriate subset of if_attach() actions there. if_attach() and if_detach() have become wrapper functions around if_attach_internal() and if_detach_internal(), where the later variants have an additional argument, a flag indicating whether a full attach or detach sequence is to be executed, or only a restricted subset suitable for moving an ifnet from one vnet to another. Hence, if_vmove() will not call if_detach() and if_attach() directly, but will call the if_detach_internal() and if_attach_internal() variants instead, with the vmove flag set. While here, staticize ifnet_setbyindex() since it is not referenced from outside of sys/net/if.c. Also rename ifccnt field in struct vimage to ifcnt, and do some minor whitespace garbage collection where appropriate. This change should have no functional impact on nooptions VIMAGE kernel builds. Reviewed by: bz, rwatson, brooks? Approved by: julian (mentor)
2009-05-22 22:09:00 +00:00
#endif /* VIMAGE */
Two more fixes to if_detach. These are generic to all interfaces and do not pollute the interface further. o Run if_detach at splnet(). o Creatively swipe the relevant parts of the netatm atm_nif_detach which will delete the relevant references to the interface going away.
1999-12-17 06:46:07 +00:00
Import interface groups from OpenBSD. This allows to group interfaces in order to - for example - apply firewall rules to a whole group of interfaces. This is required for importing pf from OpenBSD 3.9 Obtained from: OpenBSD (with changes) Discussed on: -net (back in April)
2006-06-19 22:20:45 +00:00
/*
* Add a group to an interface
*/
int
if_addgroup(struct ifnet *ifp, const char *groupname)
{
struct ifg_list *ifgl;
struct ifg_group *ifg = NULL;
struct ifg_member *ifgm;
Merge the projects/pf/head branch, that was worked on for last six months, into head. The most significant achievements in the new code: o Fine grained locking, thus much better performance. o Fixes to many problems in pf, that were specific to FreeBSD port. New code doesn't have that many ifdefs and much less OpenBSDisms, thus is more attractive to our developers. Those interested in details, can browse through SVN log of the projects/pf/head branch. And for reference, here is exact list of revisions merged: r232043, r232044, r232062, r232148, r232149, r232150, r232298, r232330, r232332, r232340, r232386, r232390, r232391, r232605, r232655, r232656, r232661, r232662, r232663, r232664, r232673, r232691, r233309, r233782, r233829, r233830, r233834, r233835, r233836, r233865, r233866, r233868, r233873, r234056, r234096, r234100, r234108, r234175, r234187, r234223, r234271, r234272, r234282, r234307, r234309, r234382, r234384, r234456, r234486, r234606, r234640, r234641, r234642, r234644, r234651, r235505, r235506, r235535, r235605, r235606, r235826, r235991, r235993, r236168, r236173, r236179, r236180, r236181, r236186, r236223, r236227, r236230, r236252, r236254, r236298, r236299, r236300, r236301, r236397, r236398, r236399, r236499, r236512, r236513, r236525, r236526, r236545, r236548, r236553, r236554, r236556, r236557, r236561, r236570, r236630, r236672, r236673, r236679, r236706, r236710, r236718, r237154, r237155, r237169, r237314, r237363, r237364, r237368, r237369, r237376, r237440, r237442, r237751, r237783, r237784, r237785, r237788, r237791, r238421, r238522, r238523, r238524, r238525, r239173, r239186, r239644, r239652, r239661, r239773, r240125, r240130, r240131, r240136, r240186, r240196, r240212. I'd like to thank people who participated in early testing: Tested by: Florian Smeets <flo freebsd.org> Tested by: Chekaluk Vitaly <artemrts ukr.net> Tested by: Ben Wilber <ben desync.com> Tested by: Ian FREISLICH <ianf cloudseed.co.za>
2012-09-08 06:41:54 +00:00
int new = 0;
Import interface groups from OpenBSD. This allows to group interfaces in order to - for example - apply firewall rules to a whole group of interfaces. This is required for importing pf from OpenBSD 3.9 Obtained from: OpenBSD (with changes) Discussed on: -net (back in April)
2006-06-19 22:20:45 +00:00
if (groupname[0] && groupname[strlen(groupname) - 1] >= '0' &&
groupname[strlen(groupname) - 1] <= '9')
return (EINVAL);
IFNET_WLOCK();
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
CK_STAILQ_FOREACH(ifgl, &ifp->if_groups, ifgl_next)
Import interface groups from OpenBSD. This allows to group interfaces in order to - for example - apply firewall rules to a whole group of interfaces. This is required for importing pf from OpenBSD 3.9 Obtained from: OpenBSD (with changes) Discussed on: -net (back in April)
2006-06-19 22:20:45 +00:00
if (!strcmp(ifgl->ifgl_group->ifg_group, groupname)) {
IFNET_WUNLOCK();
return (EEXIST);
}
if ((ifgl = (struct ifg_list *)malloc(sizeof(struct ifg_list), M_TEMP,
M_NOWAIT)) == NULL) {
IFNET_WUNLOCK();
return (ENOMEM);
}
if ((ifgm = (struct ifg_member *)malloc(sizeof(struct ifg_member),
M_TEMP, M_NOWAIT)) == NULL) {
free(ifgl, M_TEMP);
IFNET_WUNLOCK();
return (ENOMEM);
}
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
CK_STAILQ_FOREACH(ifg, &V_ifg_head, ifg_next)
Import interface groups from OpenBSD. This allows to group interfaces in order to - for example - apply firewall rules to a whole group of interfaces. This is required for importing pf from OpenBSD 3.9 Obtained from: OpenBSD (with changes) Discussed on: -net (back in April)
2006-06-19 22:20:45 +00:00
if (!strcmp(ifg->ifg_group, groupname))
break;
if (ifg == NULL) {
if ((ifg = (struct ifg_group *)malloc(sizeof(struct ifg_group),
M_TEMP, M_NOWAIT)) == NULL) {
free(ifgl, M_TEMP);
free(ifgm, M_TEMP);
IFNET_WUNLOCK();
return (ENOMEM);
}
strlcpy(ifg->ifg_group, groupname, sizeof(ifg->ifg_group));
ifg->ifg_refcnt = 0;
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
CK_STAILQ_INIT(&ifg->ifg_members);
CK_STAILQ_INSERT_TAIL(&V_ifg_head, ifg, ifg_next);
Merge the projects/pf/head branch, that was worked on for last six months, into head. The most significant achievements in the new code: o Fine grained locking, thus much better performance. o Fixes to many problems in pf, that were specific to FreeBSD port. New code doesn't have that many ifdefs and much less OpenBSDisms, thus is more attractive to our developers. Those interested in details, can browse through SVN log of the projects/pf/head branch. And for reference, here is exact list of revisions merged: r232043, r232044, r232062, r232148, r232149, r232150, r232298, r232330, r232332, r232340, r232386, r232390, r232391, r232605, r232655, r232656, r232661, r232662, r232663, r232664, r232673, r232691, r233309, r233782, r233829, r233830, r233834, r233835, r233836, r233865, r233866, r233868, r233873, r234056, r234096, r234100, r234108, r234175, r234187, r234223, r234271, r234272, r234282, r234307, r234309, r234382, r234384, r234456, r234486, r234606, r234640, r234641, r234642, r234644, r234651, r235505, r235506, r235535, r235605, r235606, r235826, r235991, r235993, r236168, r236173, r236179, r236180, r236181, r236186, r236223, r236227, r236230, r236252, r236254, r236298, r236299, r236300, r236301, r236397, r236398, r236399, r236499, r236512, r236513, r236525, r236526, r236545, r236548, r236553, r236554, r236556, r236557, r236561, r236570, r236630, r236672, r236673, r236679, r236706, r236710, r236718, r237154, r237155, r237169, r237314, r237363, r237364, r237368, r237369, r237376, r237440, r237442, r237751, r237783, r237784, r237785, r237788, r237791, r238421, r238522, r238523, r238524, r238525, r239173, r239186, r239644, r239652, r239661, r239773, r240125, r240130, r240131, r240136, r240186, r240196, r240212. I'd like to thank people who participated in early testing: Tested by: Florian Smeets <flo freebsd.org> Tested by: Chekaluk Vitaly <artemrts ukr.net> Tested by: Ben Wilber <ben desync.com> Tested by: Ian FREISLICH <ianf cloudseed.co.za>
2012-09-08 06:41:54 +00:00
new = 1;
Import interface groups from OpenBSD. This allows to group interfaces in order to - for example - apply firewall rules to a whole group of interfaces. This is required for importing pf from OpenBSD 3.9 Obtained from: OpenBSD (with changes) Discussed on: -net (back in April)
2006-06-19 22:20:45 +00:00
}
ifg->ifg_refcnt++;
ifgl->ifgl_group = ifg;
ifgm->ifgm_ifp = ifp;
Convert all users of IF_ADDR_LOCK to use new locking macros that specify either a read lock or write lock. Reviewed by: bz MFC after: 2 weeks
2012-01-05 19:00:36 +00:00
IF_ADDR_WLOCK(ifp);
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
CK_STAILQ_INSERT_TAIL(&ifg->ifg_members, ifgm, ifgm_next);
CK_STAILQ_INSERT_TAIL(&ifp->if_groups, ifgl, ifgl_next);
Convert all users of IF_ADDR_LOCK to use new locking macros that specify either a read lock or write lock. Reviewed by: bz MFC after: 2 weeks
2012-01-05 19:00:36 +00:00
IF_ADDR_WUNLOCK(ifp);
Import interface groups from OpenBSD. This allows to group interfaces in order to - for example - apply firewall rules to a whole group of interfaces. This is required for importing pf from OpenBSD 3.9 Obtained from: OpenBSD (with changes) Discussed on: -net (back in April)
2006-06-19 22:20:45 +00:00
IFNET_WUNLOCK();
Merge the projects/pf/head branch, that was worked on for last six months, into head. The most significant achievements in the new code: o Fine grained locking, thus much better performance. o Fixes to many problems in pf, that were specific to FreeBSD port. New code doesn't have that many ifdefs and much less OpenBSDisms, thus is more attractive to our developers. Those interested in details, can browse through SVN log of the projects/pf/head branch. And for reference, here is exact list of revisions merged: r232043, r232044, r232062, r232148, r232149, r232150, r232298, r232330, r232332, r232340, r232386, r232390, r232391, r232605, r232655, r232656, r232661, r232662, r232663, r232664, r232673, r232691, r233309, r233782, r233829, r233830, r233834, r233835, r233836, r233865, r233866, r233868, r233873, r234056, r234096, r234100, r234108, r234175, r234187, r234223, r234271, r234272, r234282, r234307, r234309, r234382, r234384, r234456, r234486, r234606, r234640, r234641, r234642, r234644, r234651, r235505, r235506, r235535, r235605, r235606, r235826, r235991, r235993, r236168, r236173, r236179, r236180, r236181, r236186, r236223, r236227, r236230, r236252, r236254, r236298, r236299, r236300, r236301, r236397, r236398, r236399, r236499, r236512, r236513, r236525, r236526, r236545, r236548, r236553, r236554, r236556, r236557, r236561, r236570, r236630, r236672, r236673, r236679, r236706, r236710, r236718, r237154, r237155, r237169, r237314, r237363, r237364, r237368, r237369, r237376, r237440, r237442, r237751, r237783, r237784, r237785, r237788, r237791, r238421, r238522, r238523, r238524, r238525, r239173, r239186, r239644, r239652, r239661, r239773, r240125, r240130, r240131, r240136, r240186, r240196, r240212. I'd like to thank people who participated in early testing: Tested by: Florian Smeets <flo freebsd.org> Tested by: Chekaluk Vitaly <artemrts ukr.net> Tested by: Ben Wilber <ben desync.com> Tested by: Ian FREISLICH <ianf cloudseed.co.za>
2012-09-08 06:41:54 +00:00
if (new)
EVENTHANDLER_INVOKE(group_attach_event, ifg);
Import interface groups from OpenBSD. This allows to group interfaces in order to - for example - apply firewall rules to a whole group of interfaces. This is required for importing pf from OpenBSD 3.9 Obtained from: OpenBSD (with changes) Discussed on: -net (back in April)
2006-06-19 22:20:45 +00:00
EVENTHANDLER_INVOKE(group_change_event, groupname);
return (0);
}
/*
* Remove a group from an interface
*/
int
if_delgroup(struct ifnet *ifp, const char *groupname)
{
struct ifg_list *ifgl;
struct ifg_member *ifgm;
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
int freeifgl;
Import interface groups from OpenBSD. This allows to group interfaces in order to - for example - apply firewall rules to a whole group of interfaces. This is required for importing pf from OpenBSD 3.9 Obtained from: OpenBSD (with changes) Discussed on: -net (back in April)
2006-06-19 22:20:45 +00:00
IFNET_WLOCK();
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
CK_STAILQ_FOREACH(ifgl, &ifp->if_groups, ifgl_next)
Import interface groups from OpenBSD. This allows to group interfaces in order to - for example - apply firewall rules to a whole group of interfaces. This is required for importing pf from OpenBSD 3.9 Obtained from: OpenBSD (with changes) Discussed on: -net (back in April)
2006-06-19 22:20:45 +00:00
if (!strcmp(ifgl->ifgl_group->ifg_group, groupname))
break;
if (ifgl == NULL) {
IFNET_WUNLOCK();
return (ENOENT);
}
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
freeifgl = 0;
Convert all users of IF_ADDR_LOCK to use new locking macros that specify either a read lock or write lock. Reviewed by: bz MFC after: 2 weeks
2012-01-05 19:00:36 +00:00
IF_ADDR_WLOCK(ifp);
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
CK_STAILQ_REMOVE(&ifp->if_groups, ifgl, ifg_list, ifgl_next);
Convert all users of IF_ADDR_LOCK to use new locking macros that specify either a read lock or write lock. Reviewed by: bz MFC after: 2 weeks
2012-01-05 19:00:36 +00:00
IF_ADDR_WUNLOCK(ifp);
Import interface groups from OpenBSD. This allows to group interfaces in order to - for example - apply firewall rules to a whole group of interfaces. This is required for importing pf from OpenBSD 3.9 Obtained from: OpenBSD (with changes) Discussed on: -net (back in April)
2006-06-19 22:20:45 +00:00
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
CK_STAILQ_FOREACH(ifgm, &ifgl->ifgl_group->ifg_members, ifgm_next)
Import interface groups from OpenBSD. This allows to group interfaces in order to - for example - apply firewall rules to a whole group of interfaces. This is required for importing pf from OpenBSD 3.9 Obtained from: OpenBSD (with changes) Discussed on: -net (back in April)
2006-06-19 22:20:45 +00:00
if (ifgm->ifgm_ifp == ifp)
break;
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
if (ifgm != NULL)
CK_STAILQ_REMOVE(&ifgl->ifgl_group->ifg_members, ifgm, ifg_member, ifgm_next);
Import interface groups from OpenBSD. This allows to group interfaces in order to - for example - apply firewall rules to a whole group of interfaces. This is required for importing pf from OpenBSD 3.9 Obtained from: OpenBSD (with changes) Discussed on: -net (back in April)
2006-06-19 22:20:45 +00:00
if (--ifgl->ifgl_group->ifg_refcnt == 0) {
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
CK_STAILQ_REMOVE(&V_ifg_head, ifgl->ifgl_group, ifg_group, ifg_next);
freeifgl = 1;
}
IFNET_WUNLOCK();
epoch_wait_preempt(net_epoch_preempt);
if (freeifgl) {
Import interface groups from OpenBSD. This allows to group interfaces in order to - for example - apply firewall rules to a whole group of interfaces. This is required for importing pf from OpenBSD 3.9 Obtained from: OpenBSD (with changes) Discussed on: -net (back in April)
2006-06-19 22:20:45 +00:00
EVENTHANDLER_INVOKE(group_detach_event, ifgl->ifgl_group);
free(ifgl->ifgl_group, M_TEMP);
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
}
free(ifgm, M_TEMP);
Import interface groups from OpenBSD. This allows to group interfaces in order to - for example - apply firewall rules to a whole group of interfaces. This is required for importing pf from OpenBSD 3.9 Obtained from: OpenBSD (with changes) Discussed on: -net (back in April)
2006-06-19 22:20:45 +00:00
free(ifgl, M_TEMP);
EVENTHANDLER_INVOKE(group_change_event, groupname);
return (0);
}
Follow up for r190895 It's not only the "all" group that is affected, but all groups on the given interface. PR: kern/130977, kern/131310 MFC after: 3 days (%vnet)
2009-04-10 19:16:14 +00:00
/*
* Remove an interface from all groups
*/
static void
if_delgroups(struct ifnet *ifp)
{
struct ifg_list *ifgl;
struct ifg_member *ifgm;
char groupname[IFNAMSIZ];
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
int ifglfree;
Follow up for r190895 It's not only the "all" group that is affected, but all groups on the given interface. PR: kern/130977, kern/131310 MFC after: 3 days (%vnet)
2009-04-10 19:16:14 +00:00
IFNET_WLOCK();
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
while (!CK_STAILQ_EMPTY(&ifp->if_groups)) {
ifgl = CK_STAILQ_FIRST(&ifp->if_groups);
Follow up for r190895 It's not only the "all" group that is affected, but all groups on the given interface. PR: kern/130977, kern/131310 MFC after: 3 days (%vnet)
2009-04-10 19:16:14 +00:00
strlcpy(groupname, ifgl->ifgl_group->ifg_group, IFNAMSIZ);
Convert all users of IF_ADDR_LOCK to use new locking macros that specify either a read lock or write lock. Reviewed by: bz MFC after: 2 weeks
2012-01-05 19:00:36 +00:00
IF_ADDR_WLOCK(ifp);
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
CK_STAILQ_REMOVE(&ifp->if_groups, ifgl, ifg_list, ifgl_next);
Convert all users of IF_ADDR_LOCK to use new locking macros that specify either a read lock or write lock. Reviewed by: bz MFC after: 2 weeks
2012-01-05 19:00:36 +00:00
IF_ADDR_WUNLOCK(ifp);
Follow up for r190895 It's not only the "all" group that is affected, but all groups on the given interface. PR: kern/130977, kern/131310 MFC after: 3 days (%vnet)
2009-04-10 19:16:14 +00:00
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
CK_STAILQ_FOREACH(ifgm, &ifgl->ifgl_group->ifg_members, ifgm_next)
Follow up for r190895 It's not only the "all" group that is affected, but all groups on the given interface. PR: kern/130977, kern/131310 MFC after: 3 days (%vnet)
2009-04-10 19:16:14 +00:00
if (ifgm->ifgm_ifp == ifp)
break;
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
if (ifgm != NULL)
CK_STAILQ_REMOVE(&ifgl->ifgl_group->ifg_members, ifgm, ifg_member,
Follow up for r190895 It's not only the "all" group that is affected, but all groups on the given interface. PR: kern/130977, kern/131310 MFC after: 3 days (%vnet)
2009-04-10 19:16:14 +00:00
ifgm_next);
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
ifglfree = 0;
if (--ifgl->ifgl_group->ifg_refcnt == 0) {
CK_STAILQ_REMOVE(&V_ifg_head, ifgl->ifgl_group, ifg_group, ifg_next);
ifglfree = 1;
Follow up for r190895 It's not only the "all" group that is affected, but all groups on the given interface. PR: kern/130977, kern/131310 MFC after: 3 days (%vnet)
2009-04-10 19:16:14 +00:00
}
if_delgroups: add missed unlock introduced by r334118
2018-05-24 17:54:08 +00:00
IFNET_WUNLOCK();
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
epoch_wait_preempt(net_epoch_preempt);
free(ifgm, M_TEMP);
if (ifglfree) {
Follow up for r190895 It's not only the "all" group that is affected, but all groups on the given interface. PR: kern/130977, kern/131310 MFC after: 3 days (%vnet)
2009-04-10 19:16:14 +00:00
EVENTHANDLER_INVOKE(group_detach_event,
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
ifgl->ifgl_group);
Follow up for r190895 It's not only the "all" group that is affected, but all groups on the given interface. PR: kern/130977, kern/131310 MFC after: 3 days (%vnet)
2009-04-10 19:16:14 +00:00
free(ifgl->ifgl_group, M_TEMP);
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
}
Follow up for r190895 It's not only the "all" group that is affected, but all groups on the given interface. PR: kern/130977, kern/131310 MFC after: 3 days (%vnet)
2009-04-10 19:16:14 +00:00
EVENTHANDLER_INVOKE(group_change_event, groupname);
IFNET_WLOCK();
}
IFNET_WUNLOCK();
}
Add 32-bit compat for ioctls that take struct ifgroupreq. Use an accessor to access ifgr_group and ifgr_groups. Use an macro CASE_IOC_IFGROUPREQ(cmd) in place of case statements such as "case SIOCAIFGROUP:". This avoids poluting the switch statements with large numbers of #ifdefs. Reviewed by: kib Obtained from: CheriBSD MFC after: 1 week Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D14960
2018-04-05 22:14:55 +00:00
static char *
ifgr_group_get(void *ifgrp)
{
union ifgroupreq_union *ifgrup;
ifgrup = ifgrp;
#ifdef COMPAT_FREEBSD32
if (SV_CURPROC_FLAG(SV_ILP32))
return (&ifgrup->ifgr32.ifgr_ifgru.ifgru_group[0]);
#endif
return (&ifgrup->ifgr.ifgr_ifgru.ifgru_group[0]);
}
static struct ifg_req *
ifgr_groups_get(void *ifgrp)
{
union ifgroupreq_union *ifgrup;
ifgrup = ifgrp;
#ifdef COMPAT_FREEBSD32
if (SV_CURPROC_FLAG(SV_ILP32))
return ((struct ifg_req *)(uintptr_t)
ifgrup->ifgr32.ifgr_ifgru.ifgru_groups);
#endif
return (ifgrup->ifgr.ifgr_ifgru.ifgru_groups);
}
Import interface groups from OpenBSD. This allows to group interfaces in order to - for example - apply firewall rules to a whole group of interfaces. This is required for importing pf from OpenBSD 3.9 Obtained from: OpenBSD (with changes) Discussed on: -net (back in April)
2006-06-19 22:20:45 +00:00
/*
Add 32-bit compat for ioctls that take struct ifgroupreq. Use an accessor to access ifgr_group and ifgr_groups. Use an macro CASE_IOC_IFGROUPREQ(cmd) in place of case statements such as "case SIOCAIFGROUP:". This avoids poluting the switch statements with large numbers of #ifdefs. Reviewed by: kib Obtained from: CheriBSD MFC after: 1 week Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D14960
2018-04-05 22:14:55 +00:00
* Stores all groups from an interface in memory pointed to by ifgr.
Import interface groups from OpenBSD. This allows to group interfaces in order to - for example - apply firewall rules to a whole group of interfaces. This is required for importing pf from OpenBSD 3.9 Obtained from: OpenBSD (with changes) Discussed on: -net (back in April)
2006-06-19 22:20:45 +00:00
*/
static int
Add 32-bit compat for ioctls that take struct ifgroupreq. Use an accessor to access ifgr_group and ifgr_groups. Use an macro CASE_IOC_IFGROUPREQ(cmd) in place of case statements such as "case SIOCAIFGROUP:". This avoids poluting the switch statements with large numbers of #ifdefs. Reviewed by: kib Obtained from: CheriBSD MFC after: 1 week Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D14960
2018-04-05 22:14:55 +00:00
if_getgroup(struct ifgroupreq *ifgr, struct ifnet *ifp)
Import interface groups from OpenBSD. This allows to group interfaces in order to - for example - apply firewall rules to a whole group of interfaces. This is required for importing pf from OpenBSD 3.9 Obtained from: OpenBSD (with changes) Discussed on: -net (back in April)
2006-06-19 22:20:45 +00:00
{
int len, error;
struct ifg_list *ifgl;
struct ifg_req ifgrq, *ifgp;
Widen NET_EPOCH coverage. When epoch(9) was introduced to network stack, it was basically dropped in place of existing locking, which was mutexes and rwlocks. For the sake of performance mutex covered areas were as small as possible, so became epoch covered areas. However, epoch doesn't introduce any contention, it just delays memory reclaim. So, there is no point to minimise epoch covered areas in sense of performance. Meanwhile entering/exiting epoch also has non-zero CPU usage, so doing this less often is a win. Not the least is also code maintainability. In the new paradigm we can assume that at any stage of processing a packet, we are inside network epoch. This makes coding both input and output path way easier. On output path we already enter epoch quite early - in the ip_output(), in the ip6_output(). This patch does the same for the input path. All ISR processing, network related callouts, other ways of packet injection to the network stack shall be performed in net_epoch. Any leaf function that walks network configuration now asserts epoch. Tricky part is configuration code paths - ioctls, sysctls. They also call into leaf functions, so some need to be changed. This patch would introduce more epoch recursions (see EPOCH_TRACE) than we had before. They will be cleaned up separately, as several of them aren't trivial. Note, that unlike a lock recursion the epoch recursion is safe and just wastes a bit of resources. Reviewed by: gallatin, hselasky, cy, adrian, kristof Differential Revision: https://reviews.freebsd.org/D19111
2019-10-07 22:40:05 +00:00
NET_EPOCH_ASSERT();
Import interface groups from OpenBSD. This allows to group interfaces in order to - for example - apply firewall rules to a whole group of interfaces. This is required for importing pf from OpenBSD 3.9 Obtained from: OpenBSD (with changes) Discussed on: -net (back in April)
2006-06-19 22:20:45 +00:00
if (ifgr->ifgr_len == 0) {
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
CK_STAILQ_FOREACH(ifgl, &ifp->if_groups, ifgl_next)
Import interface groups from OpenBSD. This allows to group interfaces in order to - for example - apply firewall rules to a whole group of interfaces. This is required for importing pf from OpenBSD 3.9 Obtained from: OpenBSD (with changes) Discussed on: -net (back in April)
2006-06-19 22:20:45 +00:00
ifgr->ifgr_len += sizeof(struct ifg_req);
return (0);
}
len = ifgr->ifgr_len;
Add 32-bit compat for ioctls that take struct ifgroupreq. Use an accessor to access ifgr_group and ifgr_groups. Use an macro CASE_IOC_IFGROUPREQ(cmd) in place of case statements such as "case SIOCAIFGROUP:". This avoids poluting the switch statements with large numbers of #ifdefs. Reviewed by: kib Obtained from: CheriBSD MFC after: 1 week Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D14960
2018-04-05 22:14:55 +00:00
ifgp = ifgr_groups_get(ifgr);
Import interface groups from OpenBSD. This allows to group interfaces in order to - for example - apply firewall rules to a whole group of interfaces. This is required for importing pf from OpenBSD 3.9 Obtained from: OpenBSD (with changes) Discussed on: -net (back in April)
2006-06-19 22:20:45 +00:00
/* XXX: wire */
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
CK_STAILQ_FOREACH(ifgl, &ifp->if_groups, ifgl_next) {
Widen NET_EPOCH coverage. When epoch(9) was introduced to network stack, it was basically dropped in place of existing locking, which was mutexes and rwlocks. For the sake of performance mutex covered areas were as small as possible, so became epoch covered areas. However, epoch doesn't introduce any contention, it just delays memory reclaim. So, there is no point to minimise epoch covered areas in sense of performance. Meanwhile entering/exiting epoch also has non-zero CPU usage, so doing this less often is a win. Not the least is also code maintainability. In the new paradigm we can assume that at any stage of processing a packet, we are inside network epoch. This makes coding both input and output path way easier. On output path we already enter epoch quite early - in the ip_output(), in the ip6_output(). This patch does the same for the input path. All ISR processing, network related callouts, other ways of packet injection to the network stack shall be performed in net_epoch. Any leaf function that walks network configuration now asserts epoch. Tricky part is configuration code paths - ioctls, sysctls. They also call into leaf functions, so some need to be changed. This patch would introduce more epoch recursions (see EPOCH_TRACE) than we had before. They will be cleaned up separately, as several of them aren't trivial. Note, that unlike a lock recursion the epoch recursion is safe and just wastes a bit of resources. Reviewed by: gallatin, hselasky, cy, adrian, kristof Differential Revision: https://reviews.freebsd.org/D19111
2019-10-07 22:40:05 +00:00
if (len < sizeof(ifgrq))
Import interface groups from OpenBSD. This allows to group interfaces in order to - for example - apply firewall rules to a whole group of interfaces. This is required for importing pf from OpenBSD 3.9 Obtained from: OpenBSD (with changes) Discussed on: -net (back in April)
2006-06-19 22:20:45 +00:00
return (EINVAL);
bzero(&ifgrq, sizeof ifgrq);
strlcpy(ifgrq.ifgrq_group, ifgl->ifgl_group->ifg_group,
sizeof(ifgrq.ifgrq_group));
Widen NET_EPOCH coverage. When epoch(9) was introduced to network stack, it was basically dropped in place of existing locking, which was mutexes and rwlocks. For the sake of performance mutex covered areas were as small as possible, so became epoch covered areas. However, epoch doesn't introduce any contention, it just delays memory reclaim. So, there is no point to minimise epoch covered areas in sense of performance. Meanwhile entering/exiting epoch also has non-zero CPU usage, so doing this less often is a win. Not the least is also code maintainability. In the new paradigm we can assume that at any stage of processing a packet, we are inside network epoch. This makes coding both input and output path way easier. On output path we already enter epoch quite early - in the ip_output(), in the ip6_output(). This patch does the same for the input path. All ISR processing, network related callouts, other ways of packet injection to the network stack shall be performed in net_epoch. Any leaf function that walks network configuration now asserts epoch. Tricky part is configuration code paths - ioctls, sysctls. They also call into leaf functions, so some need to be changed. This patch would introduce more epoch recursions (see EPOCH_TRACE) than we had before. They will be cleaned up separately, as several of them aren't trivial. Note, that unlike a lock recursion the epoch recursion is safe and just wastes a bit of resources. Reviewed by: gallatin, hselasky, cy, adrian, kristof Differential Revision: https://reviews.freebsd.org/D19111
2019-10-07 22:40:05 +00:00
if ((error = copyout(&ifgrq, ifgp, sizeof(struct ifg_req))))
Import interface groups from OpenBSD. This allows to group interfaces in order to - for example - apply firewall rules to a whole group of interfaces. This is required for importing pf from OpenBSD 3.9 Obtained from: OpenBSD (with changes) Discussed on: -net (back in April)
2006-06-19 22:20:45 +00:00
return (error);
len -= sizeof(ifgrq);
ifgp++;
}
return (0);
}
/*
Add 32-bit compat for ioctls that take struct ifgroupreq. Use an accessor to access ifgr_group and ifgr_groups. Use an macro CASE_IOC_IFGROUPREQ(cmd) in place of case statements such as "case SIOCAIFGROUP:". This avoids poluting the switch statements with large numbers of #ifdefs. Reviewed by: kib Obtained from: CheriBSD MFC after: 1 week Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D14960
2018-04-05 22:14:55 +00:00
* Stores all members of a group in memory pointed to by igfr
Import interface groups from OpenBSD. This allows to group interfaces in order to - for example - apply firewall rules to a whole group of interfaces. This is required for importing pf from OpenBSD 3.9 Obtained from: OpenBSD (with changes) Discussed on: -net (back in April)
2006-06-19 22:20:45 +00:00
*/
static int
Add 32-bit compat for ioctls that take struct ifgroupreq. Use an accessor to access ifgr_group and ifgr_groups. Use an macro CASE_IOC_IFGROUPREQ(cmd) in place of case statements such as "case SIOCAIFGROUP:". This avoids poluting the switch statements with large numbers of #ifdefs. Reviewed by: kib Obtained from: CheriBSD MFC after: 1 week Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D14960
2018-04-05 22:14:55 +00:00
if_getgroupmembers(struct ifgroupreq *ifgr)
Import interface groups from OpenBSD. This allows to group interfaces in order to - for example - apply firewall rules to a whole group of interfaces. This is required for importing pf from OpenBSD 3.9 Obtained from: OpenBSD (with changes) Discussed on: -net (back in April)
2006-06-19 22:20:45 +00:00
{
struct ifg_group *ifg;
struct ifg_member *ifgm;
struct ifg_req ifgrq, *ifgp;
int len, error;
IFNET_RLOCK();
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
CK_STAILQ_FOREACH(ifg, &V_ifg_head, ifg_next)
Import interface groups from OpenBSD. This allows to group interfaces in order to - for example - apply firewall rules to a whole group of interfaces. This is required for importing pf from OpenBSD 3.9 Obtained from: OpenBSD (with changes) Discussed on: -net (back in April)
2006-06-19 22:20:45 +00:00
if (!strcmp(ifg->ifg_group, ifgr->ifgr_name))
break;
if (ifg == NULL) {
IFNET_RUNLOCK();
return (ENOENT);
}
if (ifgr->ifgr_len == 0) {
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
CK_STAILQ_FOREACH(ifgm, &ifg->ifg_members, ifgm_next)
Import interface groups from OpenBSD. This allows to group interfaces in order to - for example - apply firewall rules to a whole group of interfaces. This is required for importing pf from OpenBSD 3.9 Obtained from: OpenBSD (with changes) Discussed on: -net (back in April)
2006-06-19 22:20:45 +00:00
ifgr->ifgr_len += sizeof(ifgrq);
IFNET_RUNLOCK();
return (0);
}
len = ifgr->ifgr_len;
Add 32-bit compat for ioctls that take struct ifgroupreq. Use an accessor to access ifgr_group and ifgr_groups. Use an macro CASE_IOC_IFGROUPREQ(cmd) in place of case statements such as "case SIOCAIFGROUP:". This avoids poluting the switch statements with large numbers of #ifdefs. Reviewed by: kib Obtained from: CheriBSD MFC after: 1 week Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D14960
2018-04-05 22:14:55 +00:00
ifgp = ifgr_groups_get(ifgr);
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
CK_STAILQ_FOREACH(ifgm, &ifg->ifg_members, ifgm_next) {
Import interface groups from OpenBSD. This allows to group interfaces in order to - for example - apply firewall rules to a whole group of interfaces. This is required for importing pf from OpenBSD 3.9 Obtained from: OpenBSD (with changes) Discussed on: -net (back in April)
2006-06-19 22:20:45 +00:00
if (len < sizeof(ifgrq)) {
IFNET_RUNLOCK();
return (EINVAL);
}
bzero(&ifgrq, sizeof ifgrq);
strlcpy(ifgrq.ifgrq_member, ifgm->ifgm_ifp->if_xname,
sizeof(ifgrq.ifgrq_member));
if ((error = copyout(&ifgrq, ifgp, sizeof(struct ifg_req)))) {
IFNET_RUNLOCK();
return (error);
}
len -= sizeof(ifgrq);
ifgp++;
}
IFNET_RUNLOCK();
return (0);
}
o Remove struct if_data from struct ifnet. Now it is merely API structure for route(4) socket and ifmib(4) sysctl. o Move fields from if_data to ifnet, but keep all statistic counters separate, since they should disappear later. o Provide function if_data_copy() to fill if_data, utilize it in routing socket and ifmib handler. o Provide overridable ifnet(9) method to fetch counters. If no provided, if_get_counters_compat() would be used, that returns old counters. Sponsored by: Netflix Sponsored by: Nginx, Inc.
2014-08-31 06:46:21 +00:00
/*
Finally, convert counters in struct ifnet to counter(9). Sponsored by: Netflix Sponsored by: Nginx, Inc.
2014-09-28 08:57:07 +00:00
* Return counter values from counter(9)s stored in ifnet.
o Remove struct if_data from struct ifnet. Now it is merely API structure for route(4) socket and ifmib(4) sysctl. o Move fields from if_data to ifnet, but keep all statistic counters separate, since they should disappear later. o Provide function if_data_copy() to fill if_data, utilize it in routing socket and ifmib handler. o Provide overridable ifnet(9) method to fetch counters. If no provided, if_get_counters_compat() would be used, that returns old counters. Sponsored by: Netflix Sponsored by: Nginx, Inc.
2014-08-31 06:46:21 +00:00
*/
uint64_t
While not too late rename 'ifnet_counter' to 'ift_counter'. One of the imporant moments that we discussed with Marcel and Anuranjan was that a converted driver should return false for 'grep ifnet if_driver.c' :) Sponsored by: Netflix Sponsored by: Nginx, Inc.
2014-09-18 14:47:13 +00:00
if_get_counter_default(struct ifnet *ifp, ift_counter cnt)
o Remove struct if_data from struct ifnet. Now it is merely API structure for route(4) socket and ifmib(4) sysctl. o Move fields from if_data to ifnet, but keep all statistic counters separate, since they should disappear later. o Provide function if_data_copy() to fill if_data, utilize it in routing socket and ifmib handler. o Provide overridable ifnet(9) method to fetch counters. If no provided, if_get_counters_compat() would be used, that returns old counters. Sponsored by: Netflix Sponsored by: Nginx, Inc.
2014-08-31 06:46:21 +00:00
{
Finally, convert counters in struct ifnet to counter(9). Sponsored by: Netflix Sponsored by: Nginx, Inc.
2014-09-28 08:57:07 +00:00
KASSERT(cnt < IFCOUNTERS, ("%s: invalid cnt %d", __func__, cnt));
return (counter_u64_fetch(ifp->if_counters[cnt]));
o Remove struct if_data from struct ifnet. Now it is merely API structure for route(4) socket and ifmib(4) sysctl. o Move fields from if_data to ifnet, but keep all statistic counters separate, since they should disappear later. o Provide function if_data_copy() to fill if_data, utilize it in routing socket and ifmib handler. o Provide overridable ifnet(9) method to fetch counters. If no provided, if_get_counters_compat() would be used, that returns old counters. Sponsored by: Netflix Sponsored by: Nginx, Inc.
2014-08-31 06:46:21 +00:00
}
Add if_inc_counter(), a generic method to update ifnet(9) counter w/o dereferencing the struct. Sponsored by: Netflix Sponsored by: Nginx, Inc.
2014-09-18 09:54:57 +00:00
/*
* Increase an ifnet counter. Usually used for counters shared
* between the stack and a driver, but function supports them all.
*/
void
While not too late rename 'ifnet_counter' to 'ift_counter'. One of the imporant moments that we discussed with Marcel and Anuranjan was that a converted driver should return false for 'grep ifnet if_driver.c' :) Sponsored by: Netflix Sponsored by: Nginx, Inc.
2014-09-18 14:47:13 +00:00
if_inc_counter(struct ifnet *ifp, ift_counter cnt, int64_t inc)
Add if_inc_counter(), a generic method to update ifnet(9) counter w/o dereferencing the struct. Sponsored by: Netflix Sponsored by: Nginx, Inc.
2014-09-18 09:54:57 +00:00
{
Finally, convert counters in struct ifnet to counter(9). Sponsored by: Netflix Sponsored by: Nginx, Inc.
2014-09-28 08:57:07 +00:00
KASSERT(cnt < IFCOUNTERS, ("%s: invalid cnt %d", __func__, cnt));
counter_u64_add(ifp->if_counters[cnt], inc);
Add if_inc_counter(), a generic method to update ifnet(9) counter w/o dereferencing the struct. Sponsored by: Netflix Sponsored by: Nginx, Inc.
2014-09-18 09:54:57 +00:00
}
o Remove struct if_data from struct ifnet. Now it is merely API structure for route(4) socket and ifmib(4) sysctl. o Move fields from if_data to ifnet, but keep all statistic counters separate, since they should disappear later. o Provide function if_data_copy() to fill if_data, utilize it in routing socket and ifmib handler. o Provide overridable ifnet(9) method to fetch counters. If no provided, if_get_counters_compat() would be used, that returns old counters. Sponsored by: Netflix Sponsored by: Nginx, Inc.
2014-08-31 06:46:21 +00:00
/*
* Copy data from ifnet to userland API structure if_data.
*/
void
if_data_copy(struct ifnet *ifp, struct if_data *ifd)
{
ifd->ifi_type = ifp->if_type;
ifd->ifi_physical = 0;
ifd->ifi_addrlen = ifp->if_addrlen;
ifd->ifi_hdrlen = ifp->if_hdrlen;
ifd->ifi_link_state = ifp->if_link_state;
ifd->ifi_vhid = 0;
ifd->ifi_datalen = sizeof(struct if_data);
ifd->ifi_mtu = ifp->if_mtu;
ifd->ifi_metric = ifp->if_metric;
ifd->ifi_baudrate = ifp->if_baudrate;
ifd->ifi_hwassist = ifp->if_hwassist;
ifd->ifi_epoch = ifp->if_epoch;
ifd->ifi_lastchange = ifp->if_lastchange;
ifd->ifi_ipackets = ifp->if_get_counter(ifp, IFCOUNTER_IPACKETS);
ifd->ifi_ierrors = ifp->if_get_counter(ifp, IFCOUNTER_IERRORS);
ifd->ifi_opackets = ifp->if_get_counter(ifp, IFCOUNTER_OPACKETS);
ifd->ifi_oerrors = ifp->if_get_counter(ifp, IFCOUNTER_OERRORS);
ifd->ifi_collisions = ifp->if_get_counter(ifp, IFCOUNTER_COLLISIONS);
ifd->ifi_ibytes = ifp->if_get_counter(ifp, IFCOUNTER_IBYTES);
ifd->ifi_obytes = ifp->if_get_counter(ifp, IFCOUNTER_OBYTES);
ifd->ifi_imcasts = ifp->if_get_counter(ifp, IFCOUNTER_IMCASTS);
ifd->ifi_omcasts = ifp->if_get_counter(ifp, IFCOUNTER_OMCASTS);
ifd->ifi_iqdrops = ifp->if_get_counter(ifp, IFCOUNTER_IQDROPS);
ifd->ifi_oqdrops = ifp->if_get_counter(ifp, IFCOUNTER_OQDROPS);
ifd->ifi_noproto = ifp->if_get_counter(ifp, IFCOUNTER_NOPROTO);
}
Define four wrapper functions for interface address locking, if_addr_rlock() and if_addr_runlock() for regular address lists, and if_maddr_rlock() and if_maddr_runlock() for multicast address lists. We will use these in various kernel modules to avoid encoding specific type and locking strategy information into modules that currently use IF_ADDR_LOCK() and IF_ADDR_UNLOCK() directly. MFC after: 6 weeks
2009-06-26 00:36:47 +00:00
/*
* Wrapper functions for struct ifnet address list locking macros. These are
* used by kernel modules to avoid encoding programming interface or binary
* interface assumptions that may be violated when kernel-internal locking
* approaches change.
*/
void
if_addr_rlock(struct ifnet *ifp)
{
For compatibility KPI functions like if_addr_rlock() that used to have mutexes but now are converted to epoch(9) use thread-private epoch_tracker. Embedding tracker into ifnet(9) or ifnet derived structures creates a non reentrable function, that will fail miserably if called simultaneously from two different contexts. A thread private tracker will provide a single tracker that would allow to call these functions safely. It doesn't allow nested call, but this is not expected from compatibility KPIs. Reviewed by: markj
2018-11-13 22:58:38 +00:00
epoch_enter_preempt(net_epoch_preempt, curthread->td_et);
Define four wrapper functions for interface address locking, if_addr_rlock() and if_addr_runlock() for regular address lists, and if_maddr_rlock() and if_maddr_runlock() for multicast address lists. We will use these in various kernel modules to avoid encoding specific type and locking strategy information into modules that currently use IF_ADDR_LOCK() and IF_ADDR_UNLOCK() directly. MFC after: 6 weeks
2009-06-26 00:36:47 +00:00
}
void
if_addr_runlock(struct ifnet *ifp)
{
For compatibility KPI functions like if_addr_rlock() that used to have mutexes but now are converted to epoch(9) use thread-private epoch_tracker. Embedding tracker into ifnet(9) or ifnet derived structures creates a non reentrable function, that will fail miserably if called simultaneously from two different contexts. A thread private tracker will provide a single tracker that would allow to call these functions safely. It doesn't allow nested call, but this is not expected from compatibility KPIs. Reviewed by: markj
2018-11-13 22:58:38 +00:00
epoch_exit_preempt(net_epoch_preempt, curthread->td_et);
Define four wrapper functions for interface address locking, if_addr_rlock() and if_addr_runlock() for regular address lists, and if_maddr_rlock() and if_maddr_runlock() for multicast address lists. We will use these in various kernel modules to avoid encoding specific type and locking strategy information into modules that currently use IF_ADDR_LOCK() and IF_ADDR_UNLOCK() directly. MFC after: 6 weeks
2009-06-26 00:36:47 +00:00
}
void
Introduce a procedural interface to the ifnet structure. The new interface allows the ifnet structure to be defined as an opaque type in NIC drivers. This then allows the ifnet structure to be changed without a need to change or recompile NIC drivers. Put differently, NIC drivers can be written and compiled once and be used with different network stack implementations, provided of course that those network stack implementations have an API and ABI compatible interface. This commit introduces the 'if_t' type to replace 'struct ifnet *' as the type of a network interface. The 'if_t' type is defined as 'void *' to enable the compiler to perform type conversion to 'struct ifnet *' and vice versa where needed and without warnings. The functions that implement the API are the only functions that need to have an explicit cast. The MII code has been converted to use the driver API to avoid unnecessary code churn. Code churn comes from having to work with both converted and unconverted drivers in correlation with having callback functions that take an interface. By converting the MII code first, the callback functions can be defined so that the compiler will perform the typecasts automatically. As soon as all drivers have been converted, the if_t type can be redefined as needed and the API functions can be fix to not need an explicit cast. The immediate benefactors of this change are: 1. Juniper Networks - The network stack implementation in Junos is entirely different from FreeBSD's one and this change allows Juniper to build "stock" NIC drivers that can be used in combination with both the FreeBSD and Junos stacks. 2. FreeBSD - This change opens the door towards changing ifnet and implementing new features and optimizations in the network stack without it requiring a change in the many NIC drivers FreeBSD has. Submitted by: Anuranjan Shukla <anshukla@juniper.net> Reviewed by: glebius@ Obtained from: Juniper Networks, Inc.
2014-06-02 17:54:39 +00:00
if_maddr_rlock(if_t ifp)
Define four wrapper functions for interface address locking, if_addr_rlock() and if_addr_runlock() for regular address lists, and if_maddr_rlock() and if_maddr_runlock() for multicast address lists. We will use these in various kernel modules to avoid encoding specific type and locking strategy information into modules that currently use IF_ADDR_LOCK() and IF_ADDR_UNLOCK() directly. MFC after: 6 weeks
2009-06-26 00:36:47 +00:00
{
For compatibility KPI functions like if_addr_rlock() that used to have mutexes but now are converted to epoch(9) use thread-private epoch_tracker. Embedding tracker into ifnet(9) or ifnet derived structures creates a non reentrable function, that will fail miserably if called simultaneously from two different contexts. A thread private tracker will provide a single tracker that would allow to call these functions safely. It doesn't allow nested call, but this is not expected from compatibility KPIs. Reviewed by: markj
2018-11-13 22:58:38 +00:00
epoch_enter_preempt(net_epoch_preempt, curthread->td_et);
Define four wrapper functions for interface address locking, if_addr_rlock() and if_addr_runlock() for regular address lists, and if_maddr_rlock() and if_maddr_runlock() for multicast address lists. We will use these in various kernel modules to avoid encoding specific type and locking strategy information into modules that currently use IF_ADDR_LOCK() and IF_ADDR_UNLOCK() directly. MFC after: 6 weeks
2009-06-26 00:36:47 +00:00
}
void
Introduce a procedural interface to the ifnet structure. The new interface allows the ifnet structure to be defined as an opaque type in NIC drivers. This then allows the ifnet structure to be changed without a need to change or recompile NIC drivers. Put differently, NIC drivers can be written and compiled once and be used with different network stack implementations, provided of course that those network stack implementations have an API and ABI compatible interface. This commit introduces the 'if_t' type to replace 'struct ifnet *' as the type of a network interface. The 'if_t' type is defined as 'void *' to enable the compiler to perform type conversion to 'struct ifnet *' and vice versa where needed and without warnings. The functions that implement the API are the only functions that need to have an explicit cast. The MII code has been converted to use the driver API to avoid unnecessary code churn. Code churn comes from having to work with both converted and unconverted drivers in correlation with having callback functions that take an interface. By converting the MII code first, the callback functions can be defined so that the compiler will perform the typecasts automatically. As soon as all drivers have been converted, the if_t type can be redefined as needed and the API functions can be fix to not need an explicit cast. The immediate benefactors of this change are: 1. Juniper Networks - The network stack implementation in Junos is entirely different from FreeBSD's one and this change allows Juniper to build "stock" NIC drivers that can be used in combination with both the FreeBSD and Junos stacks. 2. FreeBSD - This change opens the door towards changing ifnet and implementing new features and optimizations in the network stack without it requiring a change in the many NIC drivers FreeBSD has. Submitted by: Anuranjan Shukla <anshukla@juniper.net> Reviewed by: glebius@ Obtained from: Juniper Networks, Inc.
2014-06-02 17:54:39 +00:00
if_maddr_runlock(if_t ifp)
Define four wrapper functions for interface address locking, if_addr_rlock() and if_addr_runlock() for regular address lists, and if_maddr_rlock() and if_maddr_runlock() for multicast address lists. We will use these in various kernel modules to avoid encoding specific type and locking strategy information into modules that currently use IF_ADDR_LOCK() and IF_ADDR_UNLOCK() directly. MFC after: 6 weeks
2009-06-26 00:36:47 +00:00
{
For compatibility KPI functions like if_addr_rlock() that used to have mutexes but now are converted to epoch(9) use thread-private epoch_tracker. Embedding tracker into ifnet(9) or ifnet derived structures creates a non reentrable function, that will fail miserably if called simultaneously from two different contexts. A thread private tracker will provide a single tracker that would allow to call these functions safely. It doesn't allow nested call, but this is not expected from compatibility KPIs. Reviewed by: markj
2018-11-13 22:58:38 +00:00
epoch_exit_preempt(net_epoch_preempt, curthread->td_et);
Define four wrapper functions for interface address locking, if_addr_rlock() and if_addr_runlock() for regular address lists, and if_maddr_rlock() and if_maddr_runlock() for multicast address lists. We will use these in various kernel modules to avoid encoding specific type and locking strategy information into modules that currently use IF_ADDR_LOCK() and IF_ADDR_UNLOCK() directly. MFC after: 6 weeks
2009-06-26 00:36:47 +00:00
}
Clean up common ifaddr management: - Unify reference count and lock initialization in a single function, ifa_init(). - Move tear-down from a macro (IFAFREE) to a function ifa_free(). - Move reference count bump from a macro (IFAREF) to a function ifa_ref(). - Instead of using a u_int protected by a mutex to refcount(9) for reference count management. The ifa_mtx is now used for exactly one ioctl, and possibly should be removed. MFC after: 3 weeks
2009-06-21 19:30:33 +00:00
/*
Fix typo in r231010. Submitted by: linimon
2012-02-05 12:52:28 +00:00
* Initialization, destruction and refcounting functions for ifaddrs.
Clean up common ifaddr management: - Unify reference count and lock initialization in a single function, ifa_init(). - Move tear-down from a macro (IFAFREE) to a function ifa_free(). - Move reference count bump from a macro (IFAREF) to a function ifa_ref(). - Instead of using a u_int protected by a mutex to refcount(9) for reference count management. The ifa_mtx is now used for exactly one ioctl, and possibly should be removed. MFC after: 3 weeks
2009-06-21 19:30:33 +00:00
*/
Remove ifa_init() and provide ifa_alloc() that will allocate and setup struct ifaddr internally. Sponsored by: Netflix Sponsored by: Nginx, Inc.
2013-10-15 10:31:42 +00:00
struct ifaddr *
ifa_alloc(size_t size, int flags)
Clean up common ifaddr management: - Unify reference count and lock initialization in a single function, ifa_init(). - Move tear-down from a macro (IFAFREE) to a function ifa_free(). - Move reference count bump from a macro (IFAREF) to a function ifa_ref(). - Instead of using a u_int protected by a mutex to refcount(9) for reference count management. The ifa_mtx is now used for exactly one ioctl, and possibly should be removed. MFC after: 3 weeks
2009-06-21 19:30:33 +00:00
{
Remove ifa_init() and provide ifa_alloc() that will allocate and setup struct ifaddr internally. Sponsored by: Netflix Sponsored by: Nginx, Inc.
2013-10-15 10:31:42 +00:00
struct ifaddr *ifa;
KASSERT(size >= sizeof(struct ifaddr),
("%s: invalid size %zu", __func__, size));
ifa = malloc(size, M_IFADDR, M_ZERO | flags);
if (ifa == NULL)
return (NULL);
Clean up common ifaddr management: - Unify reference count and lock initialization in a single function, ifa_init(). - Move tear-down from a macro (IFAFREE) to a function ifa_free(). - Move reference count bump from a macro (IFAREF) to a function ifa_ref(). - Instead of using a u_int protected by a mutex to refcount(9) for reference count management. The ifa_mtx is now used for exactly one ioctl, and possibly should be removed. MFC after: 3 weeks
2009-06-21 19:30:33 +00:00
- Utilize counter(9) to accumulate statistics on interface addresses. Add four counters to struct ifaddr. This kills '+=' on a variables shared between processors for every packet. - Nuke struct if_data from struct ifaddr. - In ip_input() do not put a reference on ifaddr, instead update statistics right now in place and do IN_IFADDR_RUNLOCK(). These removes atomic(9) for every packet. [1] - To properly support NET_RT_IFLISTL sysctl used by getifaddrs(3), in rtsock.c fill if_data fields using counter_u64_fetch(). - Accidentially fix bug in COMPAT_32 version of NET_RT_IFLISTL, which took if_data not from the ifaddr, but from ifaddr's ifnet. [2] Submitted by: melifaro [1], pluknet[2] Sponsored by: Netflix Sponsored by: Nginx, Inc.
2013-10-15 11:37:57 +00:00
if ((ifa->ifa_opackets = counter_u64_alloc(flags)) == NULL)
goto fail;
if ((ifa->ifa_ipackets = counter_u64_alloc(flags)) == NULL)
goto fail;
if ((ifa->ifa_obytes = counter_u64_alloc(flags)) == NULL)
goto fail;
if ((ifa->ifa_ibytes = counter_u64_alloc(flags)) == NULL)
goto fail;
Clean up common ifaddr management: - Unify reference count and lock initialization in a single function, ifa_init(). - Move tear-down from a macro (IFAFREE) to a function ifa_free(). - Move reference count bump from a macro (IFAREF) to a function ifa_ref(). - Instead of using a u_int protected by a mutex to refcount(9) for reference count management. The ifa_mtx is now used for exactly one ioctl, and possibly should be removed. MFC after: 3 weeks
2009-06-21 19:30:33 +00:00
refcount_init(&ifa->ifa_refcnt, 1);
Remove ifa_init() and provide ifa_alloc() that will allocate and setup struct ifaddr internally. Sponsored by: Netflix Sponsored by: Nginx, Inc.
2013-10-15 10:31:42 +00:00
return (ifa);
- Utilize counter(9) to accumulate statistics on interface addresses. Add four counters to struct ifaddr. This kills '+=' on a variables shared between processors for every packet. - Nuke struct if_data from struct ifaddr. - In ip_input() do not put a reference on ifaddr, instead update statistics right now in place and do IN_IFADDR_RUNLOCK(). These removes atomic(9) for every packet. [1] - To properly support NET_RT_IFLISTL sysctl used by getifaddrs(3), in rtsock.c fill if_data fields using counter_u64_fetch(). - Accidentially fix bug in COMPAT_32 version of NET_RT_IFLISTL, which took if_data not from the ifaddr, but from ifaddr's ifnet. [2] Submitted by: melifaro [1], pluknet[2] Sponsored by: Netflix Sponsored by: Nginx, Inc.
2013-10-15 11:37:57 +00:00
fail:
/* free(NULL) is okay */
counter_u64_free(ifa->ifa_opackets);
counter_u64_free(ifa->ifa_ipackets);
counter_u64_free(ifa->ifa_obytes);
counter_u64_free(ifa->ifa_ibytes);
free(ifa, M_IFADDR);
return (NULL);
Clean up common ifaddr management: - Unify reference count and lock initialization in a single function, ifa_init(). - Move tear-down from a macro (IFAFREE) to a function ifa_free(). - Move reference count bump from a macro (IFAREF) to a function ifa_ref(). - Instead of using a u_int protected by a mutex to refcount(9) for reference count management. The ifa_mtx is now used for exactly one ioctl, and possibly should be removed. MFC after: 3 weeks
2009-06-21 19:30:33 +00:00
}
void
ifa_ref(struct ifaddr *ifa)
{
refcount_acquire(&ifa->ifa_refcnt);
}
ifnet: Replace if_addr_lock rwlock with epoch + mutex Run on LLNW canaries and tested by pho@ gallatin: Using a 14-core, 28-HTT single socket E5-2697 v3 with a 40GbE MLX5 based ConnectX 4-LX NIC, I see an almost 12% improvement in received packet rate, and a larger improvement in bytes delivered all the way to userspace. When the host receiving 64 streams of netperf -H $DUT -t UDP_STREAM -- -m 1, I see, using nstat -I mce0 1 before the patch: InMpps OMpps InGbs OGbs err TCP Est %CPU syscalls csw irq GBfree 4.98 0.00 4.42 0.00 4235592 33 83.80 4720653 2149771 1235 247.32 4.73 0.00 4.20 0.00 4025260 33 82.99 4724900 2139833 1204 247.32 4.72 0.00 4.20 0.00 4035252 33 82.14 4719162 2132023 1264 247.32 4.71 0.00 4.21 0.00 4073206 33 83.68 4744973 2123317 1347 247.32 4.72 0.00 4.21 0.00 4061118 33 80.82 4713615 2188091 1490 247.32 4.72 0.00 4.21 0.00 4051675 33 85.29 4727399 2109011 1205 247.32 4.73 0.00 4.21 0.00 4039056 33 84.65 4724735 2102603 1053 247.32 After the patch InMpps OMpps InGbs OGbs err TCP Est %CPU syscalls csw irq GBfree 5.43 0.00 4.20 0.00 3313143 33 84.96 5434214 1900162 2656 245.51 5.43 0.00 4.20 0.00 3308527 33 85.24 5439695 1809382 2521 245.51 5.42 0.00 4.19 0.00 3316778 33 87.54 5416028 1805835 2256 245.51 5.42 0.00 4.19 0.00 3317673 33 90.44 5426044 1763056 2332 245.51 5.42 0.00 4.19 0.00 3314839 33 88.11 5435732 1792218 2499 245.52 5.44 0.00 4.19 0.00 3293228 33 91.84 5426301 1668597 2121 245.52 Similarly, netperf reports 230Mb/s before the patch, and 270Mb/s after the patch Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15366
2018-05-18 20:13:34 +00:00
static void
ifa_destroy(epoch_context_t ctx)
{
struct ifaddr *ifa;
ifa = __containerof(ctx, struct ifaddr, ifa_epoch_ctx);
counter_u64_free(ifa->ifa_opackets);
counter_u64_free(ifa->ifa_ipackets);
counter_u64_free(ifa->ifa_obytes);
counter_u64_free(ifa->ifa_ibytes);
free(ifa, M_IFADDR);
}
Clean up common ifaddr management: - Unify reference count and lock initialization in a single function, ifa_init(). - Move tear-down from a macro (IFAFREE) to a function ifa_free(). - Move reference count bump from a macro (IFAREF) to a function ifa_ref(). - Instead of using a u_int protected by a mutex to refcount(9) for reference count management. The ifa_mtx is now used for exactly one ioctl, and possibly should be removed. MFC after: 3 weeks
2009-06-21 19:30:33 +00:00
void
ifa_free(struct ifaddr *ifa)
{
ifnet: Replace if_addr_lock rwlock with epoch + mutex Run on LLNW canaries and tested by pho@ gallatin: Using a 14-core, 28-HTT single socket E5-2697 v3 with a 40GbE MLX5 based ConnectX 4-LX NIC, I see an almost 12% improvement in received packet rate, and a larger improvement in bytes delivered all the way to userspace. When the host receiving 64 streams of netperf -H $DUT -t UDP_STREAM -- -m 1, I see, using nstat -I mce0 1 before the patch: InMpps OMpps InGbs OGbs err TCP Est %CPU syscalls csw irq GBfree 4.98 0.00 4.42 0.00 4235592 33 83.80 4720653 2149771 1235 247.32 4.73 0.00 4.20 0.00 4025260 33 82.99 4724900 2139833 1204 247.32 4.72 0.00 4.20 0.00 4035252 33 82.14 4719162 2132023 1264 247.32 4.71 0.00 4.21 0.00 4073206 33 83.68 4744973 2123317 1347 247.32 4.72 0.00 4.21 0.00 4061118 33 80.82 4713615 2188091 1490 247.32 4.72 0.00 4.21 0.00 4051675 33 85.29 4727399 2109011 1205 247.32 4.73 0.00 4.21 0.00 4039056 33 84.65 4724735 2102603 1053 247.32 After the patch InMpps OMpps InGbs OGbs err TCP Est %CPU syscalls csw irq GBfree 5.43 0.00 4.20 0.00 3313143 33 84.96 5434214 1900162 2656 245.51 5.43 0.00 4.20 0.00 3308527 33 85.24 5439695 1809382 2521 245.51 5.42 0.00 4.19 0.00 3316778 33 87.54 5416028 1805835 2256 245.51 5.42 0.00 4.19 0.00 3317673 33 90.44 5426044 1763056 2332 245.51 5.42 0.00 4.19 0.00 3314839 33 88.11 5435732 1792218 2499 245.52 5.44 0.00 4.19 0.00 3293228 33 91.84 5426301 1668597 2121 245.52 Similarly, netperf reports 230Mb/s before the patch, and 270Mb/s after the patch Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15366
2018-05-18 20:13:34 +00:00
if (refcount_release(&ifa->ifa_refcnt))
epoch_call(net_epoch_preempt, &ifa->ifa_epoch_ctx, ifa_destroy);
Clean up common ifaddr management: - Unify reference count and lock initialization in a single function, ifa_init(). - Move tear-down from a macro (IFAFREE) to a function ifa_free(). - Move reference count bump from a macro (IFAREF) to a function ifa_ref(). - Instead of using a u_int protected by a mutex to refcount(9) for reference count management. The ifa_mtx is now used for exactly one ioctl, and possibly should be removed. MFC after: 3 weeks
2009-06-21 19:30:33 +00:00
}
ifnet: Replace if_addr_lock rwlock with epoch + mutex Run on LLNW canaries and tested by pho@ gallatin: Using a 14-core, 28-HTT single socket E5-2697 v3 with a 40GbE MLX5 based ConnectX 4-LX NIC, I see an almost 12% improvement in received packet rate, and a larger improvement in bytes delivered all the way to userspace. When the host receiving 64 streams of netperf -H $DUT -t UDP_STREAM -- -m 1, I see, using nstat -I mce0 1 before the patch: InMpps OMpps InGbs OGbs err TCP Est %CPU syscalls csw irq GBfree 4.98 0.00 4.42 0.00 4235592 33 83.80 4720653 2149771 1235 247.32 4.73 0.00 4.20 0.00 4025260 33 82.99 4724900 2139833 1204 247.32 4.72 0.00 4.20 0.00 4035252 33 82.14 4719162 2132023 1264 247.32 4.71 0.00 4.21 0.00 4073206 33 83.68 4744973 2123317 1347 247.32 4.72 0.00 4.21 0.00 4061118 33 80.82 4713615 2188091 1490 247.32 4.72 0.00 4.21 0.00 4051675 33 85.29 4727399 2109011 1205 247.32 4.73 0.00 4.21 0.00 4039056 33 84.65 4724735 2102603 1053 247.32 After the patch InMpps OMpps InGbs OGbs err TCP Est %CPU syscalls csw irq GBfree 5.43 0.00 4.20 0.00 3313143 33 84.96 5434214 1900162 2656 245.51 5.43 0.00 4.20 0.00 3308527 33 85.24 5439695 1809382 2521 245.51 5.42 0.00 4.19 0.00 3316778 33 87.54 5416028 1805835 2256 245.51 5.42 0.00 4.19 0.00 3317673 33 90.44 5426044 1763056 2332 245.51 5.42 0.00 4.19 0.00 3314839 33 88.11 5435732 1792218 2499 245.52 5.44 0.00 4.19 0.00 3293228 33 91.84 5426301 1668597 2121 245.52 Similarly, netperf reports 230Mb/s before the patch, and 270Mb/s after the patch Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15366
2018-05-18 20:13:34 +00:00
Unify loopback route switching: * prepare gateway before insertion * use RTM_CHANGE instead of explicit find/change route * Remove fib argument from ifa_switch_loopback_route added in r264887: if old ifp fib differes from new one, that the caller is doing something wrong * Make ifa_*_loopback_route call single ifa_maintain_loopback_route().
2015-09-16 06:23:15 +00:00
static int
ifa_maintain_loopback_route(int cmd, const char *otype, struct ifaddr *ifa,
struct sockaddr *ia)
Self pointing routes are installed for configured interface addresses and address aliases. After an interface is brought down and brought back up again, those self pointing routes disappeared. This patch ensures after an interface is brought back up, the loopback routes are reinstalled properly. Reviewed by: bz MFC after: immediately
2009-09-15 19:18:34 +00:00
{
Fix rt_ifa selection during loopback route insertion process. Currently such routes are added with a link-level IFA, which is plain wrong. Only after the insertion they get fixed by the special link_rtrequest() ifa handler. This behaviour complicates routing code and makes ifa selection more complex. Streamline this process by explicitly moving link_rtrequest() logic to the pre-insertion rt_getifa_fib() ifa selector. Avoid calling all this logic in the loopback route case by explicitly specifying proper rt_ifa inside the ifa_maintain_loopback_route().§ MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D20076
2019-05-19 21:49:56 +00:00
struct epoch_tracker et;
Unify loopback route switching: * prepare gateway before insertion * use RTM_CHANGE instead of explicit find/change route * Remove fib argument from ifa_switch_loopback_route added in r264887: if old ifp fib differes from new one, that the caller is doing something wrong * Make ifa_*_loopback_route call single ifa_maintain_loopback_route().
2015-09-16 06:23:15 +00:00
int error;
Self pointing routes are installed for configured interface addresses and address aliases. After an interface is brought down and brought back up again, those self pointing routes disappeared. This patch ensures after an interface is brought back up, the loopback routes are reinstalled properly. Reviewed by: bz MFC after: immediately
2009-09-15 19:18:34 +00:00
struct rt_addrinfo info;
Unify loopback route switching: * prepare gateway before insertion * use RTM_CHANGE instead of explicit find/change route * Remove fib argument from ifa_switch_loopback_route added in r264887: if old ifp fib differes from new one, that the caller is doing something wrong * Make ifa_*_loopback_route call single ifa_maintain_loopback_route().
2015-09-16 06:23:15 +00:00
struct sockaddr_dl null_sdl;
struct ifnet *ifp;
ifp = ifa->ifa_ifp;
Self pointing routes are installed for configured interface addresses and address aliases. After an interface is brought down and brought back up again, those self pointing routes disappeared. This patch ensures after an interface is brought back up, the loopback routes are reinstalled properly. Reviewed by: bz MFC after: immediately
2009-09-15 19:18:34 +00:00
bzero(&info, sizeof(info));
Unify loopback route switching: * prepare gateway before insertion * use RTM_CHANGE instead of explicit find/change route * Remove fib argument from ifa_switch_loopback_route added in r264887: if old ifp fib differes from new one, that the caller is doing something wrong * Make ifa_*_loopback_route call single ifa_maintain_loopback_route().
2015-09-16 06:23:15 +00:00
if (cmd != RTM_DELETE)
info.rti_ifp = V_loif;
Fix rt_ifa selection during loopback route insertion process. Currently such routes are added with a link-level IFA, which is plain wrong. Only after the insertion they get fixed by the special link_rtrequest() ifa handler. This behaviour complicates routing code and makes ifa selection more complex. Streamline this process by explicitly moving link_rtrequest() logic to the pre-insertion rt_getifa_fib() ifa selector. Avoid calling all this logic in the loopback route case by explicitly specifying proper rt_ifa inside the ifa_maintain_loopback_route().§ MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D20076
2019-05-19 21:49:56 +00:00
if (cmd == RTM_ADD) {
/* explicitly specify (loopback) ifa */
if (info.rti_ifp != NULL) {
NET_EPOCH_ENTER(et);
info.rti_ifa = ifaof_ifpforaddr(ifa->ifa_addr, info.rti_ifp);
if (info.rti_ifa != NULL)
ifa_ref(info.rti_ifa);
NET_EPOCH_EXIT(et);
}
}
Allow a process to assign an IP address to local ppp interface even if kernel routing table already has a route to the address in question installed by some routing daemon (PR 223129). Also, allow loopback route deletion when stopping a VIMAGE jail (PR 222647). PR: 222647, 223129 Reviewed by: gnn Approved by: avg (mentor), mav (mentor) MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D12747
2017-11-05 14:41:48 +00:00
info.rti_flags = ifa->ifa_flags | RTF_HOST | RTF_STATIC | RTF_PINNED;
Self pointing routes are installed for configured interface addresses and address aliases. After an interface is brought down and brought back up again, those self pointing routes disappeared. This patch ensures after an interface is brought back up, the loopback routes are reinstalled properly. Reviewed by: bz MFC after: immediately
2009-09-15 19:18:34 +00:00
info.rti_info[RTAX_DST] = ia;
info.rti_info[RTAX_GATEWAY] = (struct sockaddr *)&null_sdl;
Unify loopback route switching: * prepare gateway before insertion * use RTM_CHANGE instead of explicit find/change route * Remove fib argument from ifa_switch_loopback_route added in r264887: if old ifp fib differes from new one, that the caller is doing something wrong * Make ifa_*_loopback_route call single ifa_maintain_loopback_route().
2015-09-16 06:23:15 +00:00
link_init_sdl(ifp, (struct sockaddr *)&null_sdl, ifp->if_type);
error = rtrequest1_fib(cmd, &info, NULL, ifp->if_fib);
Slight cleanup of interface event logging. Make if_printf() use vlog() instead of vprintf(). This means it can no longer return the number of characters printed, as it used to, but every single call to if_printf() in the entire kernel ignores the return value anyway; just return 0 so we don't have to change the prototype. Consistently use if_printf() throughout sys/net/if.c, instead of a mixture of if_printf() and log(). In ifa_maintain_loopback_route(), don't needlessly log an error if we either failed to add a route because it already existed or failed to remove one because it did not. We still return an error code, though. MFC after: 1 week
2018-05-11 00:19:49 +00:00
if (error != 0 &&
!(cmd == RTM_ADD && error == EEXIST) &&
!(cmd == RTM_DELETE && error == ENOENT))
if_printf(ifp, "%s failed: %d\n", otype, error);
Self pointing routes are installed for configured interface addresses and address aliases. After an interface is brought down and brought back up again, those self pointing routes disappeared. This patch ensures after an interface is brought back up, the loopback routes are reinstalled properly. Reviewed by: bz MFC after: immediately
2009-09-15 19:18:34 +00:00
return (error);
}
int
Unify loopback route switching: * prepare gateway before insertion * use RTM_CHANGE instead of explicit find/change route * Remove fib argument from ifa_switch_loopback_route added in r264887: if old ifp fib differes from new one, that the caller is doing something wrong * Make ifa_*_loopback_route call single ifa_maintain_loopback_route().
2015-09-16 06:23:15 +00:00
ifa_add_loopback_route(struct ifaddr *ifa, struct sockaddr *ia)
Self pointing routes are installed for configured interface addresses and address aliases. After an interface is brought down and brought back up again, those self pointing routes disappeared. This patch ensures after an interface is brought back up, the loopback routes are reinstalled properly. Reviewed by: bz MFC after: immediately
2009-09-15 19:18:34 +00:00
{
Unify loopback route switching: * prepare gateway before insertion * use RTM_CHANGE instead of explicit find/change route * Remove fib argument from ifa_switch_loopback_route added in r264887: if old ifp fib differes from new one, that the caller is doing something wrong * Make ifa_*_loopback_route call single ifa_maintain_loopback_route().
2015-09-16 06:23:15 +00:00
return (ifa_maintain_loopback_route(RTM_ADD, "insertion", ifa, ia));
}
Self pointing routes are installed for configured interface addresses and address aliases. After an interface is brought down and brought back up again, those self pointing routes disappeared. This patch ensures after an interface is brought back up, the loopback routes are reinstalled properly. Reviewed by: bz MFC after: immediately
2009-09-15 19:18:34 +00:00
Unify loopback route switching: * prepare gateway before insertion * use RTM_CHANGE instead of explicit find/change route * Remove fib argument from ifa_switch_loopback_route added in r264887: if old ifp fib differes from new one, that the caller is doing something wrong * Make ifa_*_loopback_route call single ifa_maintain_loopback_route().
2015-09-16 06:23:15 +00:00
int
ifa_del_loopback_route(struct ifaddr *ifa, struct sockaddr *ia)
{
Self pointing routes are installed for configured interface addresses and address aliases. After an interface is brought down and brought back up again, those self pointing routes disappeared. This patch ensures after an interface is brought back up, the loopback routes are reinstalled properly. Reviewed by: bz MFC after: immediately
2009-09-15 19:18:34 +00:00
Unify loopback route switching: * prepare gateway before insertion * use RTM_CHANGE instead of explicit find/change route * Remove fib argument from ifa_switch_loopback_route added in r264887: if old ifp fib differes from new one, that the caller is doing something wrong * Make ifa_*_loopback_route call single ifa_maintain_loopback_route().
2015-09-16 06:23:15 +00:00
return (ifa_maintain_loopback_route(RTM_DELETE, "deletion", ifa, ia));
Self pointing routes are installed for configured interface addresses and address aliases. After an interface is brought down and brought back up again, those self pointing routes disappeared. This patch ensures after an interface is brought back up, the loopback routes are reinstalled properly. Reviewed by: bz MFC after: immediately
2009-09-15 19:18:34 +00:00
}
In complemence to ifa_add_loopback_route() and ifa_del_loopback_route() provide function ifa_switch_loopback_route() that will be used in case when an interface address used for a loopback route goes away, but we have another interface address with same address value and want to preserve loopback route. Sponsored by: Netflix Sponsored by: Nginx, Inc.
2013-11-05 07:36:17 +00:00
int
Unify loopback route switching: * prepare gateway before insertion * use RTM_CHANGE instead of explicit find/change route * Remove fib argument from ifa_switch_loopback_route added in r264887: if old ifp fib differes from new one, that the caller is doing something wrong * Make ifa_*_loopback_route call single ifa_maintain_loopback_route().
2015-09-16 06:23:15 +00:00
ifa_switch_loopback_route(struct ifaddr *ifa, struct sockaddr *ia)
In complemence to ifa_add_loopback_route() and ifa_del_loopback_route() provide function ifa_switch_loopback_route() that will be used in case when an interface address used for a loopback route goes away, but we have another interface address with same address value and want to preserve loopback route. Sponsored by: Netflix Sponsored by: Nginx, Inc.
2013-11-05 07:36:17 +00:00
{
Unify loopback route switching: * prepare gateway before insertion * use RTM_CHANGE instead of explicit find/change route * Remove fib argument from ifa_switch_loopback_route added in r264887: if old ifp fib differes from new one, that the caller is doing something wrong * Make ifa_*_loopback_route call single ifa_maintain_loopback_route().
2015-09-16 06:23:15 +00:00
return (ifa_maintain_loopback_route(RTM_CHANGE, "switch", ifa, ia));
In complemence to ifa_add_loopback_route() and ifa_del_loopback_route() provide function ifa_switch_loopback_route() that will be used in case when an interface address used for a loopback route goes away, but we have another interface address with same address value and want to preserve loopback route. Sponsored by: Netflix Sponsored by: Nginx, Inc.
2013-11-05 07:36:17 +00:00
}
Fix a bug in if_findmulti(), whereby it would not find (and thus delete) a link-layer multicast group membership. Such memberships are needed in order to support protocols such as IS-IS without putting the interface into PROMISC or ALLMULTI modes. sa_equal() is not OK for comparing sockaddr_dl as it has deeper structure than a simple byte array, so add sa_dl_equal() and use that instead. Reviewed by: rwatson Verified with: /usr/sbin/mtest Bug found by: Jouke Witteveen MFC after: 2 weeks
2007-02-22 00:14:02 +00:00
/*
* XXX: Because sockaddr_dl has deeper structure than the sockaddr
* structs used to represent other address families, it is necessary
* to perform a different comparison.
*/
#define sa_dl_equal(a1, a2) \
Constantify lookup key in ifa_ifwith* functions. Some places in our network stack already have const arguments (like if_output() routines and LLE functions). Code using ifa_ifwith (and similar functins) along with LLE/_output functions is currently bound to use tricks like __DECONST(). Provide a cleaner way by making sockaddr lookup key really constant. MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D3464
2015-09-05 05:33:20 +00:00
((((const struct sockaddr_dl *)(a1))->sdl_len == \
((const struct sockaddr_dl *)(a2))->sdl_len) && \
(bcmp(CLLADDR((const struct sockaddr_dl *)(a1)), \
CLLADDR((const struct sockaddr_dl *)(a2)), \
((const struct sockaddr_dl *)(a1))->sdl_alen) == 0))
Lock up ifaddr reference counts.
2002-12-18 11:46:59 +00:00
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
/*
* Locate an interface based on a complete address.
*/
/*ARGSUSED*/
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
struct ifaddr *
ifa_ifwithaddr(const struct sockaddr *addr)
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
{
Cosmetic cleanups and rearrangement for code to come. There should be no functional change in this commit.
2001-09-06 00:44:45 +00:00
struct ifnet *ifp;
struct ifaddr *ifa;
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
Widen NET_EPOCH coverage. When epoch(9) was introduced to network stack, it was basically dropped in place of existing locking, which was mutexes and rwlocks. For the sake of performance mutex covered areas were as small as possible, so became epoch covered areas. However, epoch doesn't introduce any contention, it just delays memory reclaim. So, there is no point to minimise epoch covered areas in sense of performance. Meanwhile entering/exiting epoch also has non-zero CPU usage, so doing this less often is a win. Not the least is also code maintainability. In the new paradigm we can assume that at any stage of processing a packet, we are inside network epoch. This makes coding both input and output path way easier. On output path we already enter epoch quite early - in the ip_output(), in the ip6_output(). This patch does the same for the input path. All ISR processing, network related callouts, other ways of packet injection to the network stack shall be performed in net_epoch. Any leaf function that walks network configuration now asserts epoch. Tricky part is configuration code paths - ioctls, sysctls. They also call into leaf functions, so some need to be changed. This patch would introduce more epoch recursions (see EPOCH_TRACE) than we had before. They will be cleaned up separately, as several of them aren't trivial. Note, that unlike a lock recursion the epoch recursion is safe and just wastes a bit of resources. Reviewed by: gallatin, hselasky, cy, adrian, kristof Differential Revision: https://reviews.freebsd.org/D19111
2019-10-07 22:40:05 +00:00
NET_EPOCH_ASSERT();
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
CK_STAILQ_FOREACH(ifp, &V_ifnet, if_link) {
ifnet: Replace if_addr_lock rwlock with epoch + mutex Run on LLNW canaries and tested by pho@ gallatin: Using a 14-core, 28-HTT single socket E5-2697 v3 with a 40GbE MLX5 based ConnectX 4-LX NIC, I see an almost 12% improvement in received packet rate, and a larger improvement in bytes delivered all the way to userspace. When the host receiving 64 streams of netperf -H $DUT -t UDP_STREAM -- -m 1, I see, using nstat -I mce0 1 before the patch: InMpps OMpps InGbs OGbs err TCP Est %CPU syscalls csw irq GBfree 4.98 0.00 4.42 0.00 4235592 33 83.80 4720653 2149771 1235 247.32 4.73 0.00 4.20 0.00 4025260 33 82.99 4724900 2139833 1204 247.32 4.72 0.00 4.20 0.00 4035252 33 82.14 4719162 2132023 1264 247.32 4.71 0.00 4.21 0.00 4073206 33 83.68 4744973 2123317 1347 247.32 4.72 0.00 4.21 0.00 4061118 33 80.82 4713615 2188091 1490 247.32 4.72 0.00 4.21 0.00 4051675 33 85.29 4727399 2109011 1205 247.32 4.73 0.00 4.21 0.00 4039056 33 84.65 4724735 2102603 1053 247.32 After the patch InMpps OMpps InGbs OGbs err TCP Est %CPU syscalls csw irq GBfree 5.43 0.00 4.20 0.00 3313143 33 84.96 5434214 1900162 2656 245.51 5.43 0.00 4.20 0.00 3308527 33 85.24 5439695 1809382 2521 245.51 5.42 0.00 4.19 0.00 3316778 33 87.54 5416028 1805835 2256 245.51 5.42 0.00 4.19 0.00 3317673 33 90.44 5426044 1763056 2332 245.51 5.42 0.00 4.19 0.00 3314839 33 88.11 5435732 1792218 2499 245.52 5.44 0.00 4.19 0.00 3293228 33 91.84 5426301 1668597 2121 245.52 Similarly, netperf reports 230Mb/s before the patch, and 270Mb/s after the patch Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15366
2018-05-18 20:13:34 +00:00
CK_STAILQ_FOREACH(ifa, &ifp->if_addrhead, ifa_link) {
Cosmetic cleanups and rearrangement for code to come. There should be no functional change in this commit.
2001-09-06 00:44:45 +00:00
if (ifa->ifa_addr->sa_family != addr->sa_family)
continue;
Acquire the interface address list lock over some iterations over if_addrhead. This closes some reader-writer races associated with the address list. MFC after: 2 weeks
2009-04-21 19:06:47 +00:00
if (sa_equal(addr, ifa->ifa_addr)) {
Cosmetic cleanups and rearrangement for code to come. There should be no functional change in this commit.
2001-09-06 00:44:45 +00:00
goto done;
Acquire the interface address list lock over some iterations over if_addrhead. This closes some reader-writer races associated with the address list. MFC after: 2 weeks
2009-04-21 19:06:47 +00:00
}
Cosmetic cleanups and rearrangement for code to come. There should be no functional change in this commit.
2001-09-06 00:44:45 +00:00
/* IP6 doesn't have broadcast */
if ((ifp->if_flags & IFF_BROADCAST) &&
ifa->ifa_broadaddr &&
ifa->ifa_broadaddr->sa_len != 0 &&
Acquire the interface address list lock over some iterations over if_addrhead. This closes some reader-writer races associated with the address list. MFC after: 2 weeks
2009-04-21 19:06:47 +00:00
sa_equal(ifa->ifa_broadaddr, addr)) {
Cosmetic cleanups and rearrangement for code to come. There should be no functional change in this commit.
2001-09-06 00:44:45 +00:00
goto done;
Acquire the interface address list lock over some iterations over if_addrhead. This closes some reader-writer races associated with the address list. MFC after: 2 weeks
2009-04-21 19:06:47 +00:00
}
Cosmetic cleanups and rearrangement for code to come. There should be no functional change in this commit.
2001-09-06 00:44:45 +00:00
}
Acquire the interface address list lock over some iterations over if_addrhead. This closes some reader-writer races associated with the address list. MFC after: 2 weeks
2009-04-21 19:06:47 +00:00
}
Cosmetic cleanups and rearrangement for code to come. There should be no functional change in this commit.
2001-09-06 00:44:45 +00:00
ifa = NULL;
done:
return (ifa);
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
}
Cosmetic cleanups and rearrangement for code to come. There should be no functional change in this commit.
2001-09-06 00:44:45 +00:00
Add a new function, ifa_ifwithaddr_check(), which rather than returning a pointer to an ifaddr matching the passed socket address, returns a boolean indicating whether one was present. In the (near) future, ifa_ifwithaddr() will return a referenced ifaddr rather than a raw ifaddr pointer, and the new wrapper will allow callers that care only about the boolean condition to avoid having to free that reference. MFC after: 3 weeks
2009-06-22 10:59:34 +00:00
int
Constantify lookup key in ifa_ifwith* functions. Some places in our network stack already have const arguments (like if_output() routines and LLE functions). Code using ifa_ifwith (and similar functins) along with LLE/_output functions is currently bound to use tricks like __DECONST(). Provide a cleaner way by making sockaddr lookup key really constant. MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D3464
2015-09-05 05:33:20 +00:00
ifa_ifwithaddr_check(const struct sockaddr *addr)
Add a new function, ifa_ifwithaddr_check(), which rather than returning a pointer to an ifaddr matching the passed socket address, returns a boolean indicating whether one was present. In the (near) future, ifa_ifwithaddr() will return a referenced ifaddr rather than a raw ifaddr pointer, and the new wrapper will allow callers that care only about the boolean condition to avoid having to free that reference. MFC after: 3 weeks
2009-06-22 10:59:34 +00:00
{
Mechanical cleanup of epoch(9) usage in network stack. - Remove macros that covertly create epoch_tracker on thread stack. Such macros a quite unsafe, e.g. will produce a buggy code if same macro is used in embedded scopes. Explicitly declare epoch_tracker always. - Unmask interface list IFNET_RLOCK_NOSLEEP(), interface address list IF_ADDR_RLOCK() and interface AF specific data IF_AFDATA_RLOCK() read locking macros to what they actually are - the net_epoch. Keeping them as is is very misleading. They all are named FOO_RLOCK(), while they no longer have lock semantics. Now they allow recursion and what's more important they now no longer guarantee protection against their companion WLOCK macros. Note: INP_HASH_RLOCK() has same problems, but not touched by this commit. This is non functional mechanical change. The only functionally changed functions are ni6_addrs() and ni6_store_addrs(), where we no longer enter epoch recursively. Discussed with: jtl, gallatin
2019-01-09 01:11:19 +00:00
struct epoch_tracker et;
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
int rc;
Add a new function, ifa_ifwithaddr_check(), which rather than returning a pointer to an ifaddr matching the passed socket address, returns a boolean indicating whether one was present. In the (near) future, ifa_ifwithaddr() will return a referenced ifaddr rather than a raw ifaddr pointer, and the new wrapper will allow callers that care only about the boolean condition to avoid having to free that reference. MFC after: 3 weeks
2009-06-22 10:59:34 +00:00
Mechanical cleanup of epoch(9) usage in network stack. - Remove macros that covertly create epoch_tracker on thread stack. Such macros a quite unsafe, e.g. will produce a buggy code if same macro is used in embedded scopes. Explicitly declare epoch_tracker always. - Unmask interface list IFNET_RLOCK_NOSLEEP(), interface address list IF_ADDR_RLOCK() and interface AF specific data IF_AFDATA_RLOCK() read locking macros to what they actually are - the net_epoch. Keeping them as is is very misleading. They all are named FOO_RLOCK(), while they no longer have lock semantics. Now they allow recursion and what's more important they now no longer guarantee protection against their companion WLOCK macros. Note: INP_HASH_RLOCK() has same problems, but not touched by this commit. This is non functional mechanical change. The only functionally changed functions are ni6_addrs() and ni6_store_addrs(), where we no longer enter epoch recursively. Discussed with: jtl, gallatin
2019-01-09 01:11:19 +00:00
NET_EPOCH_ENTER(et);
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
rc = (ifa_ifwithaddr(addr) != NULL);
Mechanical cleanup of epoch(9) usage in network stack. - Remove macros that covertly create epoch_tracker on thread stack. Such macros a quite unsafe, e.g. will produce a buggy code if same macro is used in embedded scopes. Explicitly declare epoch_tracker always. - Unmask interface list IFNET_RLOCK_NOSLEEP(), interface address list IF_ADDR_RLOCK() and interface AF specific data IF_AFDATA_RLOCK() read locking macros to what they actually are - the net_epoch. Keeping them as is is very misleading. They all are named FOO_RLOCK(), while they no longer have lock semantics. Now they allow recursion and what's more important they now no longer guarantee protection against their companion WLOCK macros. Note: INP_HASH_RLOCK() has same problems, but not touched by this commit. This is non functional mechanical change. The only functionally changed functions are ni6_addrs() and ni6_store_addrs(), where we no longer enter epoch recursively. Discussed with: jtl, gallatin
2019-01-09 01:11:19 +00:00
NET_EPOCH_EXIT(et);
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
return (rc);
Add a new function, ifa_ifwithaddr_check(), which rather than returning a pointer to an ifaddr matching the passed socket address, returns a boolean indicating whether one was present. In the (near) future, ifa_ifwithaddr() will return a referenced ifaddr rather than a raw ifaddr pointer, and the new wrapper will allow callers that care only about the boolean condition to avoid having to free that reference. MFC after: 3 weeks
2009-06-22 10:59:34 +00:00
}
Fix the socket option IP_ONESBCAST by giving it its own case in ip_output() and skip over the normal IP processing. Add a supporting function ifa_ifwithbroadaddr() to verify and validate the supplied subnet broadcast address. PR: kern/99558 Tested by: Andrey V. Elsukov <bu7cher-at-yandex.ru> Sponsored by: TCP/IP Optimization Fundraise 2005 MFC after: 3 days
2006-09-06 17:12:10 +00:00
/*
* Locate an interface based on the broadcast address.
*/
/* ARGSUSED */
struct ifaddr *
Constantify lookup key in ifa_ifwith* functions. Some places in our network stack already have const arguments (like if_output() routines and LLE functions). Code using ifa_ifwith (and similar functins) along with LLE/_output functions is currently bound to use tricks like __DECONST(). Provide a cleaner way by making sockaddr lookup key really constant. MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D3464
2015-09-05 05:33:20 +00:00
ifa_ifwithbroadaddr(const struct sockaddr *addr, int fibnum)
Fix the socket option IP_ONESBCAST by giving it its own case in ip_output() and skip over the normal IP processing. Add a supporting function ifa_ifwithbroadaddr() to verify and validate the supplied subnet broadcast address. PR: kern/99558 Tested by: Andrey V. Elsukov <bu7cher-at-yandex.ru> Sponsored by: TCP/IP Optimization Fundraise 2005 MFC after: 3 days
2006-09-06 17:12:10 +00:00
{
struct ifnet *ifp;
struct ifaddr *ifa;
epoch(9): allow preemptible epochs to compose - Add tracker argument to preemptible epochs - Inline epoch read path in kernel and tied modules - Change in_epoch to take an epoch as argument - Simplify tfb_tcp_do_segment to not take a ti_locked argument, there's no longer any benefit to dropping the pcbinfo lock and trying to do so just adds an error prone branchfest to these functions - Remove cases of same function recursion on the epoch as recursing is no longer free. - Remove the the TAILQ_ENTRY and epoch_section from struct thread as the tracker field is now stack or heap allocated as appropriate. Tested by: pho and Limelight Networks Reviewed by: kbowling at llnw dot com Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D16066
2018-07-04 02:47:16 +00:00
MPASS(in_epoch(net_epoch_preempt));
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
CK_STAILQ_FOREACH(ifp, &V_ifnet, if_link) {
Revisions 264905 and 266860 added a "int fib" argument to ifa_ifwithnet and ifa_ifwithdstaddr. For the sake of backwards compatibility, the new arguments were added to new functions named ifa_ifwithnet_fib and ifa_ifwithdstaddr_fib, while the old functions became wrappers around the new ones that passed RT_ALL_FIBS for the fib argument. However, the backwards compatibility is not desired for FreeBSD 11, because there are numerous other incompatible changes to the ifnet(9) API. We therefore decided to remove it from head but leave it in place for stable/9 and stable/10. In addition, this commit adds the fib argument to ifa_ifwithbroadaddr for consistency's sake. sys/sys/param.h Increment __FreeBSD_version sys/net/if.c sys/net/if_var.h sys/net/route.c Add fibnum argument to ifa_ifwithbroadaddr, and remove the _fib versions of ifa_ifwithdstaddr, ifa_ifwithnet, and ifa_ifwithroute. sys/net/route.c sys/net/rtsock.c sys/netinet/in_pcb.c sys/netinet/ip_options.c sys/netinet/ip_output.c sys/netinet6/nd6.c Fixup calls of modified functions. share/man/man9/ifnet.9 Document changed API. CR: https://reviews.freebsd.org/D458 MFC after: Never Sponsored by: Spectra Logic
2014-09-11 20:21:03 +00:00
if ((fibnum != RT_ALL_FIBS) && (ifp->if_fib != fibnum))
continue;
ifnet: Replace if_addr_lock rwlock with epoch + mutex Run on LLNW canaries and tested by pho@ gallatin: Using a 14-core, 28-HTT single socket E5-2697 v3 with a 40GbE MLX5 based ConnectX 4-LX NIC, I see an almost 12% improvement in received packet rate, and a larger improvement in bytes delivered all the way to userspace. When the host receiving 64 streams of netperf -H $DUT -t UDP_STREAM -- -m 1, I see, using nstat -I mce0 1 before the patch: InMpps OMpps InGbs OGbs err TCP Est %CPU syscalls csw irq GBfree 4.98 0.00 4.42 0.00 4235592 33 83.80 4720653 2149771 1235 247.32 4.73 0.00 4.20 0.00 4025260 33 82.99 4724900 2139833 1204 247.32 4.72 0.00 4.20 0.00 4035252 33 82.14 4719162 2132023 1264 247.32 4.71 0.00 4.21 0.00 4073206 33 83.68 4744973 2123317 1347 247.32 4.72 0.00 4.21 0.00 4061118 33 80.82 4713615 2188091 1490 247.32 4.72 0.00 4.21 0.00 4051675 33 85.29 4727399 2109011 1205 247.32 4.73 0.00 4.21 0.00 4039056 33 84.65 4724735 2102603 1053 247.32 After the patch InMpps OMpps InGbs OGbs err TCP Est %CPU syscalls csw irq GBfree 5.43 0.00 4.20 0.00 3313143 33 84.96 5434214 1900162 2656 245.51 5.43 0.00 4.20 0.00 3308527 33 85.24 5439695 1809382 2521 245.51 5.42 0.00 4.19 0.00 3316778 33 87.54 5416028 1805835 2256 245.51 5.42 0.00 4.19 0.00 3317673 33 90.44 5426044 1763056 2332 245.51 5.42 0.00 4.19 0.00 3314839 33 88.11 5435732 1792218 2499 245.52 5.44 0.00 4.19 0.00 3293228 33 91.84 5426301 1668597 2121 245.52 Similarly, netperf reports 230Mb/s before the patch, and 270Mb/s after the patch Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15366
2018-05-18 20:13:34 +00:00
CK_STAILQ_FOREACH(ifa, &ifp->if_addrhead, ifa_link) {
Fix the socket option IP_ONESBCAST by giving it its own case in ip_output() and skip over the normal IP processing. Add a supporting function ifa_ifwithbroadaddr() to verify and validate the supplied subnet broadcast address. PR: kern/99558 Tested by: Andrey V. Elsukov <bu7cher-at-yandex.ru> Sponsored by: TCP/IP Optimization Fundraise 2005 MFC after: 3 days
2006-09-06 17:12:10 +00:00
if (ifa->ifa_addr->sa_family != addr->sa_family)
continue;
if ((ifp->if_flags & IFF_BROADCAST) &&
ifa->ifa_broadaddr &&
ifa->ifa_broadaddr->sa_len != 0 &&
Acquire the interface address list lock over some iterations over if_addrhead. This closes some reader-writer races associated with the address list. MFC after: 2 weeks
2009-04-21 19:06:47 +00:00
sa_equal(ifa->ifa_broadaddr, addr)) {
Fix the socket option IP_ONESBCAST by giving it its own case in ip_output() and skip over the normal IP processing. Add a supporting function ifa_ifwithbroadaddr() to verify and validate the supplied subnet broadcast address. PR: kern/99558 Tested by: Andrey V. Elsukov <bu7cher-at-yandex.ru> Sponsored by: TCP/IP Optimization Fundraise 2005 MFC after: 3 days
2006-09-06 17:12:10 +00:00
goto done;
Acquire the interface address list lock over some iterations over if_addrhead. This closes some reader-writer races associated with the address list. MFC after: 2 weeks
2009-04-21 19:06:47 +00:00
}
Fix the socket option IP_ONESBCAST by giving it its own case in ip_output() and skip over the normal IP processing. Add a supporting function ifa_ifwithbroadaddr() to verify and validate the supplied subnet broadcast address. PR: kern/99558 Tested by: Andrey V. Elsukov <bu7cher-at-yandex.ru> Sponsored by: TCP/IP Optimization Fundraise 2005 MFC after: 3 days
2006-09-06 17:12:10 +00:00
}
Acquire the interface address list lock over some iterations over if_addrhead. This closes some reader-writer races associated with the address list. MFC after: 2 weeks
2009-04-21 19:06:47 +00:00
}
Fix the socket option IP_ONESBCAST by giving it its own case in ip_output() and skip over the normal IP processing. Add a supporting function ifa_ifwithbroadaddr() to verify and validate the supplied subnet broadcast address. PR: kern/99558 Tested by: Andrey V. Elsukov <bu7cher-at-yandex.ru> Sponsored by: TCP/IP Optimization Fundraise 2005 MFC after: 3 days
2006-09-06 17:12:10 +00:00
ifa = NULL;
done:
return (ifa);
}
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
/*
* Locate the point to point interface with a given destination address.
*/
/*ARGSUSED*/
struct ifaddr *
Constantify lookup key in ifa_ifwith* functions. Some places in our network stack already have const arguments (like if_output() routines and LLE functions). Code using ifa_ifwith (and similar functins) along with LLE/_output functions is currently bound to use tricks like __DECONST(). Provide a cleaner way by making sockaddr lookup key really constant. MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D3464
2015-09-05 05:33:20 +00:00
ifa_ifwithdstaddr(const struct sockaddr *addr, int fibnum)
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
{
Cosmetic cleanups and rearrangement for code to come. There should be no functional change in this commit.
2001-09-06 00:44:45 +00:00
struct ifnet *ifp;
struct ifaddr *ifa;
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
epoch(9): allow preemptible epochs to compose - Add tracker argument to preemptible epochs - Inline epoch read path in kernel and tied modules - Change in_epoch to take an epoch as argument - Simplify tfb_tcp_do_segment to not take a ti_locked argument, there's no longer any benefit to dropping the pcbinfo lock and trying to do so just adds an error prone branchfest to these functions - Remove cases of same function recursion on the epoch as recursing is no longer free. - Remove the the TAILQ_ENTRY and epoch_section from struct thread as the tracker field is now stack or heap allocated as appropriate. Tested by: pho and Limelight Networks Reviewed by: kbowling at llnw dot com Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D16066
2018-07-04 02:47:16 +00:00
MPASS(in_epoch(net_epoch_preempt));
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
CK_STAILQ_FOREACH(ifp, &V_ifnet, if_link) {
Cosmetic cleanups and rearrangement for code to come. There should be no functional change in this commit.
2001-09-06 00:44:45 +00:00
if ((ifp->if_flags & IFF_POINTOPOINT) == 0)
continue;
Fix unintended KBI change from r264905. Add _fib versions of ifa_ifwithnet() and ifa_ifwithdstaddr() The legacy functions will call the _fib() versions with RT_ALL_FIBS, preserving legacy behavior. sys/net/if_var.h sys/net/if.c Add legacy-compatible functions as described above. Ensure legacy behavior when RT_ALL_FIBS is passed as fibnum. sys/netinet/in_pcb.c sys/netinet/ip_output.c sys/netinet/ip_options.c sys/net/route.c sys/net/rtsock.c sys/netinet6/nd6.c Call with _fib() functions if we must use a specific fib, or the legacy functions otherwise. tests/sys/netinet/fibs_test.sh tests/sys/netinet/udp_dontroute.c Improve the udp_dontroute test. The bug that this test exercises is that ifa_ifwithnet() will return the wrong address, if multiple interfaces have addresses on the same subnet but with different fibs. The previous version of the test only considered one possible failure mode: that ifa_ifwithnet_fib() might fail to find any suitable address at all. The new version also checks whether ifa_ifwithnet_fib() finds the correct address by checking where the ARP request goes. Reported by: bz, hrs Reviewed by: hrs MFC after: 1 week X-MFC-with: 264905 Sponsored by: Spectra Logic
2014-05-29 21:03:49 +00:00
if ((fibnum != RT_ALL_FIBS) && (ifp->if_fib != fibnum))
Fix subnet and default routes on different FIBs on the same subnet. These two bugs are closely related. The root cause is that ifa_ifwithnet does not consider FIBs when searching for an interface address. sys/net/if_var.h sys/net/if.c Add a fib argument to ifa_ifwithnet and ifa_ifwithdstadddr. Those functions will only return an address whose interface fib equals the argument. sys/net/route.c Update calls to ifa_ifwithnet and ifa_ifwithdstaddr with fib arguments. sys/netinet/in.c Update in_addprefix to consider the interface fib when adding prefixes. This will prevent it from not adding a subnet route when one already exists on a different fib. sys/net/rtsock.c sys/netinet/in_pcb.c sys/netinet/ip_output.c sys/netinet/ip_options.c sys/netinet6/nd6.c Add RT_DEFAULT_FIB arguments to ifa_ifwithdstaddr and ifa_ifwithnet. In some cases it there wasn't a clear specific fib number to use. In others, I was unable to test those functions so I chose RT_DEFAULT_FIB to minimize divergence from current behavior. I will fix some of the latter changes along with PR kern/187553. tests/sys/netinet/fibs_test.sh tests/sys/netinet/udp_dontroute.c tests/sys/netinet/Makefile Revert r263738. The udp_dontroute test was right all along. However, bugs kern/187550 and kern/187553 cancelled each other out when it came to this test. Because of kern/187553, ifa_ifwithnet searched the default fib instead of the requested one, but because of kern/187550, there was an applicable subnet route on the default fib. The new test added in r263738 doesn't work right, however. I can verify with dtrace that ifa_ifwithnet returned the wrong address before I applied this commit, but route(8) miraculously found the correct interface to use anyway. I don't know how. Clear expected failure messages for kern/187550 and kern/187552. PR: kern/187550 PR: kern/187552 Reviewed by: melifaro MFC after: 3 weeks Sponsored by: Spectra Logic
2014-04-24 23:56:56 +00:00
continue;
ifnet: Replace if_addr_lock rwlock with epoch + mutex Run on LLNW canaries and tested by pho@ gallatin: Using a 14-core, 28-HTT single socket E5-2697 v3 with a 40GbE MLX5 based ConnectX 4-LX NIC, I see an almost 12% improvement in received packet rate, and a larger improvement in bytes delivered all the way to userspace. When the host receiving 64 streams of netperf -H $DUT -t UDP_STREAM -- -m 1, I see, using nstat -I mce0 1 before the patch: InMpps OMpps InGbs OGbs err TCP Est %CPU syscalls csw irq GBfree 4.98 0.00 4.42 0.00 4235592 33 83.80 4720653 2149771 1235 247.32 4.73 0.00 4.20 0.00 4025260 33 82.99 4724900 2139833 1204 247.32 4.72 0.00 4.20 0.00 4035252 33 82.14 4719162 2132023 1264 247.32 4.71 0.00 4.21 0.00 4073206 33 83.68 4744973 2123317 1347 247.32 4.72 0.00 4.21 0.00 4061118 33 80.82 4713615 2188091 1490 247.32 4.72 0.00 4.21 0.00 4051675 33 85.29 4727399 2109011 1205 247.32 4.73 0.00 4.21 0.00 4039056 33 84.65 4724735 2102603 1053 247.32 After the patch InMpps OMpps InGbs OGbs err TCP Est %CPU syscalls csw irq GBfree 5.43 0.00 4.20 0.00 3313143 33 84.96 5434214 1900162 2656 245.51 5.43 0.00 4.20 0.00 3308527 33 85.24 5439695 1809382 2521 245.51 5.42 0.00 4.19 0.00 3316778 33 87.54 5416028 1805835 2256 245.51 5.42 0.00 4.19 0.00 3317673 33 90.44 5426044 1763056 2332 245.51 5.42 0.00 4.19 0.00 3314839 33 88.11 5435732 1792218 2499 245.52 5.44 0.00 4.19 0.00 3293228 33 91.84 5426301 1668597 2121 245.52 Similarly, netperf reports 230Mb/s before the patch, and 270Mb/s after the patch Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15366
2018-05-18 20:13:34 +00:00
CK_STAILQ_FOREACH(ifa, &ifp->if_addrhead, ifa_link) {
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
if (ifa->ifa_addr->sa_family != addr->sa_family)
continue;
Make the checks for ptp interfaces in ifa_ifwithdstaddr() and ifa_ifwithnet() look more similar by comparing the pointer to NULL in both cases. MFC after: 3 months
2008-08-24 11:03:43 +00:00
if (ifa->ifa_dstaddr != NULL &&
Acquire the interface address list lock over some iterations over if_addrhead. This closes some reader-writer races associated with the address list. MFC after: 2 weeks
2009-04-21 19:06:47 +00:00
sa_equal(addr, ifa->ifa_dstaddr)) {
Cosmetic cleanups and rearrangement for code to come. There should be no functional change in this commit.
2001-09-06 00:44:45 +00:00
goto done;
Acquire the interface address list lock over some iterations over if_addrhead. This closes some reader-writer races associated with the address list. MFC after: 2 weeks
2009-04-21 19:06:47 +00:00
}
Cosmetic cleanups and rearrangement for code to come. There should be no functional change in this commit.
2001-09-06 00:44:45 +00:00
}
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
}
Cosmetic cleanups and rearrangement for code to come. There should be no functional change in this commit.
2001-09-06 00:44:45 +00:00
ifa = NULL;
done:
return (ifa);
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
}
/*
* Find an interface on a specific network. If many, choice
* is most specific found.
*/
struct ifaddr *
Constantify lookup key in ifa_ifwith* functions. Some places in our network stack already have const arguments (like if_output() routines and LLE functions). Code using ifa_ifwith (and similar functins) along with LLE/_output functions is currently bound to use tricks like __DECONST(). Provide a cleaner way by making sockaddr lookup key really constant. MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D3464
2015-09-05 05:33:20 +00:00
ifa_ifwithnet(const struct sockaddr *addr, int ignore_ptp, int fibnum)
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
{
Clean up whitespace, remove "register" keyword, ANSIfy. No functional changes.
2003-10-23 13:49:10 +00:00
struct ifnet *ifp;
struct ifaddr *ifa;
Modify most routines returning 'struct ifaddr *' to return references rather than pointers, requiring callers to properly dispose of those references. The following routines now return references: ifaddr_byindex ifa_ifwithaddr ifa_ifwithbroadaddr ifa_ifwithdstaddr ifa_ifwithnet ifaof_ifpforaddr ifa_ifwithroute ifa_ifwithroute_fib rt_getifa rt_getifa_fib IFP_TO_IA ip_rtaddr in6_ifawithifp in6ifa_ifpforlinklocal in6ifa_ifpwithaddr in6_ifadd carp_iamatch6 ip6_getdstifaddr Remove unused macro which didn't have required referencing: IFP_TO_IA6 This closes many small races in which changes to interface or address lists while an ifaddr was in use could lead to use of freed memory (etc). In a few cases, add missing if_addr_list locking required to safely acquire references. Because of a lack of deep copying support, we accept a race in which an in6_ifaddr pointed to by mbuf tags and extracted with ip6_getdstifaddr() doesn't hold a reference while in transmit. Once we have mbuf tag deep copy support, this can be fixed. Reviewed by: bz Obtained from: Apple, Inc. (portions) MFC after: 6 weeks (portions)
2009-06-23 20:19:09 +00:00
struct ifaddr *ifa_maybe = NULL;
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
u_int af = addr->sa_family;
Constantify lookup key in ifa_ifwith* functions. Some places in our network stack already have const arguments (like if_output() routines and LLE functions). Code using ifa_ifwith (and similar functins) along with LLE/_output functions is currently bound to use tricks like __DECONST(). Provide a cleaner way by making sockaddr lookup key really constant. MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D3464
2015-09-05 05:33:20 +00:00
const char *addr_data = addr->sa_data, *cplim;
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
epoch(9): allow preemptible epochs to compose - Add tracker argument to preemptible epochs - Inline epoch read path in kernel and tied modules - Change in_epoch to take an epoch as argument - Simplify tfb_tcp_do_segment to not take a ti_locked argument, there's no longer any benefit to dropping the pcbinfo lock and trying to do so just adds an error prone branchfest to these functions - Remove cases of same function recursion on the epoch as recursing is no longer free. - Remove the the TAILQ_ENTRY and epoch_section from struct thread as the tracker field is now stack or heap allocated as appropriate. Tested by: pho and Limelight Networks Reviewed by: kbowling at llnw dot com Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D16066
2018-07-04 02:47:16 +00:00
MPASS(in_epoch(net_epoch_preempt));
add some comments while trying to understand why appletalk gets some things wrong. (part of my continuing "comment it as you understand it" effort :)
1997-08-22 22:47:27 +00:00
/*
* AF_LINK addresses can be looked up directly by their index number,
* so do that if we can.
*/
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
if (af == AF_LINK) {
Constantify lookup key in ifa_ifwith* functions. Some places in our network stack already have const arguments (like if_output() routines and LLE functions). Code using ifa_ifwith (and similar functins) along with LLE/_output functions is currently bound to use tricks like __DECONST(). Provide a cleaner way by making sockaddr lookup key really constant. MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D3464
2015-09-05 05:33:20 +00:00
const struct sockaddr_dl *sdl = (const struct sockaddr_dl *)addr;
Commit step 1 of the vimage project, (network stack) virtualization work done by Marko Zec (zec@). This is the first in a series of commits over the course of the next few weeks. Mark all uses of global variables to be virtualized with a V_ prefix. Use macros to map them back to their global names for now, so this is a NOP change only. We hope to have caught at least 85-90% of what is needed so we do not invalidate a lot of outstanding patches again. Obtained from: //depot/projects/vimage-commit2/... Reviewed by: brooks, des, ed, mav, julian, jamie, kris, rwatson, zec, ... (various people I forgot, different versions) md5 (with a bit of help) Sponsored by: NLnet Foundation, The FreeBSD Foundation X-MFC after: never V_Commit_Message_Reviewed_By: more people than the patch
2008-08-17 23:27:27 +00:00
if (sdl->sdl_index && sdl->sdl_index <= V_if_index)
Wrap array accesses in macros, which also happen to be lvalues: ifnet_addrs[i - 1] -> ifaddr_byindex(i) ifindex2ifnet[i] -> ifnet_byindex(i) This is intended to ease the conversion to SMPng.
2001-09-06 02:40:43 +00:00
return (ifaddr_byindex(sdl->sdl_index));
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
}
add some comments while trying to understand why appletalk gets some things wrong. (part of my continuing "comment it as you understand it" effort :)
1997-08-22 22:47:27 +00:00
KAME netinet6 basic part(no IPsec,no V6 Multicast Forwarding, no UDP/TCP for IPv6 yet) With this patch, you can assigne IPv6 addr automatically, and can reply to IPv6 ping. Reviewed by: freebsd-arch, cvs-committers Obtained from: KAME project
1999-11-22 02:45:11 +00:00
/*
Modify most routines returning 'struct ifaddr *' to return references rather than pointers, requiring callers to properly dispose of those references. The following routines now return references: ifaddr_byindex ifa_ifwithaddr ifa_ifwithbroadaddr ifa_ifwithdstaddr ifa_ifwithnet ifaof_ifpforaddr ifa_ifwithroute ifa_ifwithroute_fib rt_getifa rt_getifa_fib IFP_TO_IA ip_rtaddr in6_ifawithifp in6ifa_ifpforlinklocal in6ifa_ifpwithaddr in6_ifadd carp_iamatch6 ip6_getdstifaddr Remove unused macro which didn't have required referencing: IFP_TO_IA6 This closes many small races in which changes to interface or address lists while an ifaddr was in use could lead to use of freed memory (etc). In a few cases, add missing if_addr_list locking required to safely acquire references. Because of a lack of deep copying support, we accept a race in which an in6_ifaddr pointed to by mbuf tags and extracted with ip6_getdstifaddr() doesn't hold a reference while in transmit. Once we have mbuf tag deep copy support, this can be fixed. Reviewed by: bz Obtained from: Apple, Inc. (portions) MFC after: 6 weeks (portions)
2009-06-23 20:19:09 +00:00
* Scan though each interface, looking for ones that have addresses
Remove part of comment that doesn't match reality.
2019-01-09 00:38:16 +00:00
* in this address family and the requested fib.
add some comments while trying to understand why appletalk gets some things wrong. (part of my continuing "comment it as you understand it" effort :)
1997-08-22 22:47:27 +00:00
*/
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
CK_STAILQ_FOREACH(ifp, &V_ifnet, if_link) {
Fix unintended KBI change from r264905. Add _fib versions of ifa_ifwithnet() and ifa_ifwithdstaddr() The legacy functions will call the _fib() versions with RT_ALL_FIBS, preserving legacy behavior. sys/net/if_var.h sys/net/if.c Add legacy-compatible functions as described above. Ensure legacy behavior when RT_ALL_FIBS is passed as fibnum. sys/netinet/in_pcb.c sys/netinet/ip_output.c sys/netinet/ip_options.c sys/net/route.c sys/net/rtsock.c sys/netinet6/nd6.c Call with _fib() functions if we must use a specific fib, or the legacy functions otherwise. tests/sys/netinet/fibs_test.sh tests/sys/netinet/udp_dontroute.c Improve the udp_dontroute test. The bug that this test exercises is that ifa_ifwithnet() will return the wrong address, if multiple interfaces have addresses on the same subnet but with different fibs. The previous version of the test only considered one possible failure mode: that ifa_ifwithnet_fib() might fail to find any suitable address at all. The new version also checks whether ifa_ifwithnet_fib() finds the correct address by checking where the ARP request goes. Reported by: bz, hrs Reviewed by: hrs MFC after: 1 week X-MFC-with: 264905 Sponsored by: Spectra Logic
2014-05-29 21:03:49 +00:00
if ((fibnum != RT_ALL_FIBS) && (ifp->if_fib != fibnum))
Fix subnet and default routes on different FIBs on the same subnet. These two bugs are closely related. The root cause is that ifa_ifwithnet does not consider FIBs when searching for an interface address. sys/net/if_var.h sys/net/if.c Add a fib argument to ifa_ifwithnet and ifa_ifwithdstadddr. Those functions will only return an address whose interface fib equals the argument. sys/net/route.c Update calls to ifa_ifwithnet and ifa_ifwithdstaddr with fib arguments. sys/netinet/in.c Update in_addprefix to consider the interface fib when adding prefixes. This will prevent it from not adding a subnet route when one already exists on a different fib. sys/net/rtsock.c sys/netinet/in_pcb.c sys/netinet/ip_output.c sys/netinet/ip_options.c sys/netinet6/nd6.c Add RT_DEFAULT_FIB arguments to ifa_ifwithdstaddr and ifa_ifwithnet. In some cases it there wasn't a clear specific fib number to use. In others, I was unable to test those functions so I chose RT_DEFAULT_FIB to minimize divergence from current behavior. I will fix some of the latter changes along with PR kern/187553. tests/sys/netinet/fibs_test.sh tests/sys/netinet/udp_dontroute.c tests/sys/netinet/Makefile Revert r263738. The udp_dontroute test was right all along. However, bugs kern/187550 and kern/187553 cancelled each other out when it came to this test. Because of kern/187553, ifa_ifwithnet searched the default fib instead of the requested one, but because of kern/187550, there was an applicable subnet route on the default fib. The new test added in r263738 doesn't work right, however. I can verify with dtrace that ifa_ifwithnet returned the wrong address before I applied this commit, but route(8) miraculously found the correct interface to use anyway. I don't know how. Clear expected failure messages for kern/187550 and kern/187552. PR: kern/187550 PR: kern/187552 Reviewed by: melifaro MFC after: 3 weeks Sponsored by: Spectra Logic
2014-04-24 23:56:56 +00:00
continue;
ifnet: Replace if_addr_lock rwlock with epoch + mutex Run on LLNW canaries and tested by pho@ gallatin: Using a 14-core, 28-HTT single socket E5-2697 v3 with a 40GbE MLX5 based ConnectX 4-LX NIC, I see an almost 12% improvement in received packet rate, and a larger improvement in bytes delivered all the way to userspace. When the host receiving 64 streams of netperf -H $DUT -t UDP_STREAM -- -m 1, I see, using nstat -I mce0 1 before the patch: InMpps OMpps InGbs OGbs err TCP Est %CPU syscalls csw irq GBfree 4.98 0.00 4.42 0.00 4235592 33 83.80 4720653 2149771 1235 247.32 4.73 0.00 4.20 0.00 4025260 33 82.99 4724900 2139833 1204 247.32 4.72 0.00 4.20 0.00 4035252 33 82.14 4719162 2132023 1264 247.32 4.71 0.00 4.21 0.00 4073206 33 83.68 4744973 2123317 1347 247.32 4.72 0.00 4.21 0.00 4061118 33 80.82 4713615 2188091 1490 247.32 4.72 0.00 4.21 0.00 4051675 33 85.29 4727399 2109011 1205 247.32 4.73 0.00 4.21 0.00 4039056 33 84.65 4724735 2102603 1053 247.32 After the patch InMpps OMpps InGbs OGbs err TCP Est %CPU syscalls csw irq GBfree 5.43 0.00 4.20 0.00 3313143 33 84.96 5434214 1900162 2656 245.51 5.43 0.00 4.20 0.00 3308527 33 85.24 5439695 1809382 2521 245.51 5.42 0.00 4.19 0.00 3316778 33 87.54 5416028 1805835 2256 245.51 5.42 0.00 4.19 0.00 3317673 33 90.44 5426044 1763056 2332 245.51 5.42 0.00 4.19 0.00 3314839 33 88.11 5435732 1792218 2499 245.52 5.44 0.00 4.19 0.00 3293228 33 91.84 5426301 1668597 2121 245.52 Similarly, netperf reports 230Mb/s before the patch, and 270Mb/s after the patch Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15366
2018-05-18 20:13:34 +00:00
CK_STAILQ_FOREACH(ifa, &ifp->if_addrhead, ifa_link) {
Constantify lookup key in ifa_ifwith* functions. Some places in our network stack already have const arguments (like if_output() routines and LLE functions). Code using ifa_ifwith (and similar functins) along with LLE/_output functions is currently bound to use tricks like __DECONST(). Provide a cleaner way by making sockaddr lookup key really constant. MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D3464
2015-09-05 05:33:20 +00:00
const char *cp, *cp2, *cp3;
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
Don't skip point-to-point interfaces if the netmask==0 (the netmask should be completely ignored for point-to-point interfaces). For point-to-point interfaces, route based on the destination address, not the local address. Submitted by: Peter Wemm
1995-06-28 05:31:03 +00:00
if (ifa->ifa_addr->sa_family != af)
add some comments while trying to understand why appletalk gets some things wrong. (part of my continuing "comment it as you understand it" effort :)
1997-08-22 22:47:27 +00:00
next: continue;
This patch fixes the problem where proxy ARP entries cannot be added over the if_ng interface. MFC after: 3 days
2010-05-25 20:42:35 +00:00
if (af == AF_INET &&
ifp->if_flags & IFF_POINTOPOINT && !ignore_ptp) {
add some comments while trying to understand why appletalk gets some things wrong. (part of my continuing "comment it as you understand it" effort :)
1997-08-22 22:47:27 +00:00
/*
KAME netinet6 basic part(no IPsec,no V6 Multicast Forwarding, no UDP/TCP for IPv6 yet) With this patch, you can assigne IPv6 addr automatically, and can reply to IPv6 ping. Reviewed by: freebsd-arch, cvs-committers Obtained from: KAME project
1999-11-22 02:45:11 +00:00
* This is a bit broken as it doesn't
* take into account that the remote end may
add some comments while trying to understand why appletalk gets some things wrong. (part of my continuing "comment it as you understand it" effort :)
1997-08-22 22:47:27 +00:00
* be a single node in the network we are
* looking for.
KAME netinet6 basic part(no IPsec,no V6 Multicast Forwarding, no UDP/TCP for IPv6 yet) With this patch, you can assigne IPv6 addr automatically, and can reply to IPv6 ping. Reviewed by: freebsd-arch, cvs-committers Obtained from: KAME project
1999-11-22 02:45:11 +00:00
* The trouble is that we don't know the
add some comments while trying to understand why appletalk gets some things wrong. (part of my continuing "comment it as you understand it" effort :)
1997-08-22 22:47:27 +00:00
* netmask for the remote end.
*/
Make the checks for ptp interfaces in ifa_ifwithdstaddr() and ifa_ifwithnet() look more similar by comparing the pointer to NULL in both cases. MFC after: 3 months
2008-08-24 11:03:43 +00:00
if (ifa->ifa_dstaddr != NULL &&
Acquire the interface address list lock over some iterations over if_addrhead. This closes some reader-writer races associated with the address list. MFC after: 2 weeks
2009-04-21 19:06:47 +00:00
sa_equal(addr, ifa->ifa_dstaddr)) {
Cosmetic cleanups and rearrangement for code to come. There should be no functional change in this commit.
2001-09-06 00:44:45 +00:00
goto done;
Acquire the interface address list lock over some iterations over if_addrhead. This closes some reader-writer races associated with the address list. MFC after: 2 weeks
2009-04-21 19:06:47 +00:00
}
Took out P2P_LOCALADDR_SHARE option and made it standard.
1995-06-15 00:19:56 +00:00
} else {
add some comments while trying to understand why appletalk gets some things wrong. (part of my continuing "comment it as you understand it" effort :)
1997-08-22 22:47:27 +00:00
/*
* Scan all the bits in the ifa's address.
* If a bit dissagrees with what we are
* looking for, mask it with the netmask
* to see if it really matters.
* (A byte at a time)
*/
Don't skip point-to-point interfaces if the netmask==0 (the netmask should be completely ignored for point-to-point interfaces). For point-to-point interfaces, route based on the destination address, not the local address. Submitted by: Peter Wemm
1995-06-28 05:31:03 +00:00
if (ifa->ifa_netmask == 0)
continue;
Added a fix for a bug which caused the wrong interface to be selected for broadcasts if point-to-point links shared the same IP address as the ethernet. The fix must be enabled with P2P_LOCALADDR_SHARE option in the kernel config file. This will someday likely be standard, but there isn't sufficient time before release to determine if there are any interoperability problems with routed and/or gated. Reviewed by: Garrett Wollman, and me Submitted by: Peter Wemm
1995-05-27 04:37:24 +00:00
cp = addr_data;
cp2 = ifa->ifa_addr->sa_data;
cp3 = ifa->ifa_netmask->sa_data;
add some comments while trying to understand why appletalk gets some things wrong. (part of my continuing "comment it as you understand it" effort :)
1997-08-22 22:47:27 +00:00
cplim = ifa->ifa_netmask->sa_len
+ (char *)ifa->ifa_netmask;
Added a fix for a bug which caused the wrong interface to be selected for broadcasts if point-to-point links shared the same IP address as the ethernet. The fix must be enabled with P2P_LOCALADDR_SHARE option in the kernel config file. This will someday likely be standard, but there isn't sufficient time before release to determine if there are any interoperability problems with routed and/or gated. Reviewed by: Garrett Wollman, and me Submitted by: Peter Wemm
1995-05-27 04:37:24 +00:00
while (cp3 < cplim)
if ((*cp++ ^ *cp2++) & *cp3++)
add some comments while trying to understand why appletalk gets some things wrong. (part of my continuing "comment it as you understand it" effort :)
1997-08-22 22:47:27 +00:00
goto next; /* next address! */
/*
* If the netmask of what we just found
* is more specific than what we had before
Resolve source address selection in presense of CARP. Add a couple of helper functions: - carp_master() - boolean function which is true if an address is in the MASTER state. - ifa_preferred() - boolean function that compares two addresses, and is aware of CARP. Utilize ifa_preferred() in ifa_ifwithnet(). The previous version of patch also changed source address selection logic in jails using carp_master(), but we failed to negotiate this part with Bjoern. May be we will approach this problem again later. Reported & tested by: Anton Yuzhaninov <citrin citrin.ru> Sponsored by: Nginx, Inc
2013-02-11 10:58:22 +00:00
* (if we had one), or if the virtual status
* of new prefix is better than of the old one,
* then remember the new one before continuing
* to search for an even better one.
add some comments while trying to understand why appletalk gets some things wrong. (part of my continuing "comment it as you understand it" effort :)
1997-08-22 22:47:27 +00:00
*/
Modify most routines returning 'struct ifaddr *' to return references rather than pointers, requiring callers to properly dispose of those references. The following routines now return references: ifaddr_byindex ifa_ifwithaddr ifa_ifwithbroadaddr ifa_ifwithdstaddr ifa_ifwithnet ifaof_ifpforaddr ifa_ifwithroute ifa_ifwithroute_fib rt_getifa rt_getifa_fib IFP_TO_IA ip_rtaddr in6_ifawithifp in6ifa_ifpforlinklocal in6ifa_ifpwithaddr in6_ifadd carp_iamatch6 ip6_getdstifaddr Remove unused macro which didn't have required referencing: IFP_TO_IA6 This closes many small races in which changes to interface or address lists while an ifaddr was in use could lead to use of freed memory (etc). In a few cases, add missing if_addr_list locking required to safely acquire references. Because of a lack of deep copying support, we accept a race in which an in6_ifaddr pointed to by mbuf tags and extracted with ip6_getdstifaddr() doesn't hold a reference while in transmit. Once we have mbuf tag deep copy support, this can be fixed. Reviewed by: bz Obtained from: Apple, Inc. (portions) MFC after: 6 weeks (portions)
2009-06-23 20:19:09 +00:00
if (ifa_maybe == NULL ||
Resolve source address selection in presense of CARP. Add a couple of helper functions: - carp_master() - boolean function which is true if an address is in the MASTER state. - ifa_preferred() - boolean function that compares two addresses, and is aware of CARP. Utilize ifa_preferred() in ifa_ifwithnet(). The previous version of patch also changed source address selection logic in jails using carp_master(), but we failed to negotiate this part with Bjoern. May be we will approach this problem again later. Reported & tested by: Anton Yuzhaninov <citrin citrin.ru> Sponsored by: Nginx, Inc
2013-02-11 10:58:22 +00:00
ifa_preferred(ifa_maybe, ifa) ||
Added a fix for a bug which caused the wrong interface to be selected for broadcasts if point-to-point links shared the same IP address as the ethernet. The fix must be enabled with P2P_LOCALADDR_SHARE option in the kernel config file. This will someday likely be standard, but there isn't sufficient time before release to determine if there are any interoperability problems with routed and/or gated. Reviewed by: Garrett Wollman, and me Submitted by: Peter Wemm
1995-05-27 04:37:24 +00:00
rn_refines((caddr_t)ifa->ifa_netmask,
Modify most routines returning 'struct ifaddr *' to return references rather than pointers, requiring callers to properly dispose of those references. The following routines now return references: ifaddr_byindex ifa_ifwithaddr ifa_ifwithbroadaddr ifa_ifwithdstaddr ifa_ifwithnet ifaof_ifpforaddr ifa_ifwithroute ifa_ifwithroute_fib rt_getifa rt_getifa_fib IFP_TO_IA ip_rtaddr in6_ifawithifp in6ifa_ifpforlinklocal in6ifa_ifpwithaddr in6_ifadd carp_iamatch6 ip6_getdstifaddr Remove unused macro which didn't have required referencing: IFP_TO_IA6 This closes many small races in which changes to interface or address lists while an ifaddr was in use could lead to use of freed memory (etc). In a few cases, add missing if_addr_list locking required to safely acquire references. Because of a lack of deep copying support, we accept a race in which an in6_ifaddr pointed to by mbuf tags and extracted with ip6_getdstifaddr() doesn't hold a reference while in transmit. Once we have mbuf tag deep copy support, this can be fixed. Reviewed by: bz Obtained from: Apple, Inc. (portions) MFC after: 6 weeks (portions)
2009-06-23 20:19:09 +00:00
(caddr_t)ifa_maybe->ifa_netmask)) {
Added a fix for a bug which caused the wrong interface to be selected for broadcasts if point-to-point links shared the same IP address as the ethernet. The fix must be enabled with P2P_LOCALADDR_SHARE option in the kernel config file. This will someday likely be standard, but there isn't sufficient time before release to determine if there are any interoperability problems with routed and/or gated. Reviewed by: Garrett Wollman, and me Submitted by: Peter Wemm
1995-05-27 04:37:24 +00:00
ifa_maybe = ifa;
Modify most routines returning 'struct ifaddr *' to return references rather than pointers, requiring callers to properly dispose of those references. The following routines now return references: ifaddr_byindex ifa_ifwithaddr ifa_ifwithbroadaddr ifa_ifwithdstaddr ifa_ifwithnet ifaof_ifpforaddr ifa_ifwithroute ifa_ifwithroute_fib rt_getifa rt_getifa_fib IFP_TO_IA ip_rtaddr in6_ifawithifp in6ifa_ifpforlinklocal in6ifa_ifpwithaddr in6_ifadd carp_iamatch6 ip6_getdstifaddr Remove unused macro which didn't have required referencing: IFP_TO_IA6 This closes many small races in which changes to interface or address lists while an ifaddr was in use could lead to use of freed memory (etc). In a few cases, add missing if_addr_list locking required to safely acquire references. Because of a lack of deep copying support, we accept a race in which an in6_ifaddr pointed to by mbuf tags and extracted with ip6_getdstifaddr() doesn't hold a reference while in transmit. Once we have mbuf tag deep copy support, this can be fixed. Reviewed by: bz Obtained from: Apple, Inc. (portions) MFC after: 6 weeks (portions)
2009-06-23 20:19:09 +00:00
}
Added a fix for a bug which caused the wrong interface to be selected for broadcasts if point-to-point links shared the same IP address as the ethernet. The fix must be enabled with P2P_LOCALADDR_SHARE option in the kernel config file. This will someday likely be standard, but there isn't sufficient time before release to determine if there are any interoperability problems with routed and/or gated. Reviewed by: Garrett Wollman, and me Submitted by: Peter Wemm
1995-05-27 04:37:24 +00:00
}
}
}
Cosmetic cleanups and rearrangement for code to come. There should be no functional change in this commit.
2001-09-06 00:44:45 +00:00
ifa = ifa_maybe;
Modify most routines returning 'struct ifaddr *' to return references rather than pointers, requiring callers to properly dispose of those references. The following routines now return references: ifaddr_byindex ifa_ifwithaddr ifa_ifwithbroadaddr ifa_ifwithdstaddr ifa_ifwithnet ifaof_ifpforaddr ifa_ifwithroute ifa_ifwithroute_fib rt_getifa rt_getifa_fib IFP_TO_IA ip_rtaddr in6_ifawithifp in6ifa_ifpforlinklocal in6ifa_ifpwithaddr in6_ifadd carp_iamatch6 ip6_getdstifaddr Remove unused macro which didn't have required referencing: IFP_TO_IA6 This closes many small races in which changes to interface or address lists while an ifaddr was in use could lead to use of freed memory (etc). In a few cases, add missing if_addr_list locking required to safely acquire references. Because of a lack of deep copying support, we accept a race in which an in6_ifaddr pointed to by mbuf tags and extracted with ip6_getdstifaddr() doesn't hold a reference while in transmit. Once we have mbuf tag deep copy support, this can be fixed. Reviewed by: bz Obtained from: Apple, Inc. (portions) MFC after: 6 weeks (portions)
2009-06-23 20:19:09 +00:00
ifa_maybe = NULL;
Cosmetic cleanups and rearrangement for code to come. There should be no functional change in this commit.
2001-09-06 00:44:45 +00:00
done:
return (ifa);
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
}
/*
* Find an interface address specific to an interface best matching
* a given address.
*/
struct ifaddr *
Constantify lookup key in ifa_ifwith* functions. Some places in our network stack already have const arguments (like if_output() routines and LLE functions). Code using ifa_ifwith (and similar functins) along with LLE/_output functions is currently bound to use tricks like __DECONST(). Provide a cleaner way by making sockaddr lookup key really constant. MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D3464
2015-09-05 05:33:20 +00:00
ifaof_ifpforaddr(const struct sockaddr *addr, struct ifnet *ifp)
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
{
Clean up whitespace, remove "register" keyword, ANSIfy. No functional changes.
2003-10-23 13:49:10 +00:00
struct ifaddr *ifa;
Constantify lookup key in ifa_ifwith* functions. Some places in our network stack already have const arguments (like if_output() routines and LLE functions). Code using ifa_ifwith (and similar functins) along with LLE/_output functions is currently bound to use tricks like __DECONST(). Provide a cleaner way by making sockaddr lookup key really constant. MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D3464
2015-09-05 05:33:20 +00:00
const char *cp, *cp2, *cp3;
Clean up whitespace, remove "register" keyword, ANSIfy. No functional changes.
2003-10-23 13:49:10 +00:00
char *cplim;
Modify most routines returning 'struct ifaddr *' to return references rather than pointers, requiring callers to properly dispose of those references. The following routines now return references: ifaddr_byindex ifa_ifwithaddr ifa_ifwithbroadaddr ifa_ifwithdstaddr ifa_ifwithnet ifaof_ifpforaddr ifa_ifwithroute ifa_ifwithroute_fib rt_getifa rt_getifa_fib IFP_TO_IA ip_rtaddr in6_ifawithifp in6ifa_ifpforlinklocal in6ifa_ifpwithaddr in6_ifadd carp_iamatch6 ip6_getdstifaddr Remove unused macro which didn't have required referencing: IFP_TO_IA6 This closes many small races in which changes to interface or address lists while an ifaddr was in use could lead to use of freed memory (etc). In a few cases, add missing if_addr_list locking required to safely acquire references. Because of a lack of deep copying support, we accept a race in which an in6_ifaddr pointed to by mbuf tags and extracted with ip6_getdstifaddr() doesn't hold a reference while in transmit. Once we have mbuf tag deep copy support, this can be fixed. Reviewed by: bz Obtained from: Apple, Inc. (portions) MFC after: 6 weeks (portions)
2009-06-23 20:19:09 +00:00
struct ifaddr *ifa_maybe = NULL;
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
u_int af = addr->sa_family;
if (af >= AF_MAX)
Return NULL rather than 0 for a pointer. MFC after: 3 days
2010-07-27 11:54:01 +00:00
return (NULL);
epoch(9): allow preemptible epochs to compose - Add tracker argument to preemptible epochs - Inline epoch read path in kernel and tied modules - Change in_epoch to take an epoch as argument - Simplify tfb_tcp_do_segment to not take a ti_locked argument, there's no longer any benefit to dropping the pcbinfo lock and trying to do so just adds an error prone branchfest to these functions - Remove cases of same function recursion on the epoch as recursing is no longer free. - Remove the the TAILQ_ENTRY and epoch_section from struct thread as the tracker field is now stack or heap allocated as appropriate. Tested by: pho and Limelight Networks Reviewed by: kbowling at llnw dot com Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D16066
2018-07-04 02:47:16 +00:00
MPASS(in_epoch(net_epoch_preempt));
ifnet: Replace if_addr_lock rwlock with epoch + mutex Run on LLNW canaries and tested by pho@ gallatin: Using a 14-core, 28-HTT single socket E5-2697 v3 with a 40GbE MLX5 based ConnectX 4-LX NIC, I see an almost 12% improvement in received packet rate, and a larger improvement in bytes delivered all the way to userspace. When the host receiving 64 streams of netperf -H $DUT -t UDP_STREAM -- -m 1, I see, using nstat -I mce0 1 before the patch: InMpps OMpps InGbs OGbs err TCP Est %CPU syscalls csw irq GBfree 4.98 0.00 4.42 0.00 4235592 33 83.80 4720653 2149771 1235 247.32 4.73 0.00 4.20 0.00 4025260 33 82.99 4724900 2139833 1204 247.32 4.72 0.00 4.20 0.00 4035252 33 82.14 4719162 2132023 1264 247.32 4.71 0.00 4.21 0.00 4073206 33 83.68 4744973 2123317 1347 247.32 4.72 0.00 4.21 0.00 4061118 33 80.82 4713615 2188091 1490 247.32 4.72 0.00 4.21 0.00 4051675 33 85.29 4727399 2109011 1205 247.32 4.73 0.00 4.21 0.00 4039056 33 84.65 4724735 2102603 1053 247.32 After the patch InMpps OMpps InGbs OGbs err TCP Est %CPU syscalls csw irq GBfree 5.43 0.00 4.20 0.00 3313143 33 84.96 5434214 1900162 2656 245.51 5.43 0.00 4.20 0.00 3308527 33 85.24 5439695 1809382 2521 245.51 5.42 0.00 4.19 0.00 3316778 33 87.54 5416028 1805835 2256 245.51 5.42 0.00 4.19 0.00 3317673 33 90.44 5426044 1763056 2332 245.51 5.42 0.00 4.19 0.00 3314839 33 88.11 5435732 1792218 2499 245.52 5.44 0.00 4.19 0.00 3293228 33 91.84 5426301 1668597 2121 245.52 Similarly, netperf reports 230Mb/s before the patch, and 270Mb/s after the patch Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15366
2018-05-18 20:13:34 +00:00
CK_STAILQ_FOREACH(ifa, &ifp->if_addrhead, ifa_link) {
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
if (ifa->ifa_addr->sa_family != af)
continue;
Modify most routines returning 'struct ifaddr *' to return references rather than pointers, requiring callers to properly dispose of those references. The following routines now return references: ifaddr_byindex ifa_ifwithaddr ifa_ifwithbroadaddr ifa_ifwithdstaddr ifa_ifwithnet ifaof_ifpforaddr ifa_ifwithroute ifa_ifwithroute_fib rt_getifa rt_getifa_fib IFP_TO_IA ip_rtaddr in6_ifawithifp in6ifa_ifpforlinklocal in6ifa_ifpwithaddr in6_ifadd carp_iamatch6 ip6_getdstifaddr Remove unused macro which didn't have required referencing: IFP_TO_IA6 This closes many small races in which changes to interface or address lists while an ifaddr was in use could lead to use of freed memory (etc). In a few cases, add missing if_addr_list locking required to safely acquire references. Because of a lack of deep copying support, we accept a race in which an in6_ifaddr pointed to by mbuf tags and extracted with ip6_getdstifaddr() doesn't hold a reference while in transmit. Once we have mbuf tag deep copy support, this can be fixed. Reviewed by: bz Obtained from: Apple, Inc. (portions) MFC after: 6 weeks (portions)
2009-06-23 20:19:09 +00:00
if (ifa_maybe == NULL)
Submitted by: archie@whistle.com This is a patch to sys/net/if.c. What it does is patch the algorithm for finding an IP address on an interface which most closely matches a given IP address. The problem with it is when no address matches, and you have to just pick one at random. Then the code ends up picking the last IP address in the list. This patch changes things so it picks up the first address instead. Usually the first address is more useful as the later ones are aliases.
1996-08-07 04:09:05 +00:00
ifa_maybe = ifa;
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
if (ifa->ifa_netmask == 0) {
Rename equal() macro to sa_equal(), which matches the definitions of sa_equal() in other files, and makes it more clear what equal() is comparing. MFC after: 3 days
2005-07-19 10:03:47 +00:00
if (sa_equal(addr, ifa->ifa_addr) ||
(ifa->ifa_dstaddr &&
sa_equal(addr, ifa->ifa_dstaddr)))
Fix sense of comparison in space test. Also eliminate a compile warning and remove a previously existing off-by-one error.
2001-09-07 05:32:54 +00:00
goto done;
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
continue;
}
Added a fix for a bug which caused the wrong interface to be selected for broadcasts if point-to-point links shared the same IP address as the ethernet. The fix must be enabled with P2P_LOCALADDR_SHARE option in the kernel config file. This will someday likely be standard, but there isn't sufficient time before release to determine if there are any interoperability problems with routed and/or gated. Reviewed by: Garrett Wollman, and me Submitted by: Peter Wemm
1995-05-27 04:37:24 +00:00
if (ifp->if_flags & IFF_POINTOPOINT) {
Rename equal() macro to sa_equal(), which matches the definitions of sa_equal() in other files, and makes it more clear what equal() is comparing. MFC after: 3 days
2005-07-19 10:03:47 +00:00
if (sa_equal(addr, ifa->ifa_dstaddr))
Fix another shortcircuit return() statement that I missed.
2001-09-07 05:39:47 +00:00
goto done;
Took out P2P_LOCALADDR_SHARE option and made it standard.
1995-06-15 00:19:56 +00:00
} else {
Added a fix for a bug which caused the wrong interface to be selected for broadcasts if point-to-point links shared the same IP address as the ethernet. The fix must be enabled with P2P_LOCALADDR_SHARE option in the kernel config file. This will someday likely be standard, but there isn't sufficient time before release to determine if there are any interoperability problems with routed and/or gated. Reviewed by: Garrett Wollman, and me Submitted by: Peter Wemm
1995-05-27 04:37:24 +00:00
cp = addr->sa_data;
cp2 = ifa->ifa_addr->sa_data;
cp3 = ifa->ifa_netmask->sa_data;
cplim = ifa->ifa_netmask->sa_len + (char *)ifa->ifa_netmask;
for (; cp3 < cplim; cp3++)
if ((*cp++ ^ *cp2++) & *cp3)
break;
if (cp3 == cplim)
Fix sense of comparison in space test. Also eliminate a compile warning and remove a previously existing off-by-one error.
2001-09-07 05:32:54 +00:00
goto done;
Added a fix for a bug which caused the wrong interface to be selected for broadcasts if point-to-point links shared the same IP address as the ethernet. The fix must be enabled with P2P_LOCALADDR_SHARE option in the kernel config file. This will someday likely be standard, but there isn't sufficient time before release to determine if there are any interoperability problems with routed and/or gated. Reviewed by: Garrett Wollman, and me Submitted by: Peter Wemm
1995-05-27 04:37:24 +00:00
}
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
}
Wrap array accesses in macros, which also happen to be lvalues: ifnet_addrs[i - 1] -> ifaddr_byindex(i) ifindex2ifnet[i] -> ifnet_byindex(i) This is intended to ease the conversion to SMPng.
2001-09-06 02:40:43 +00:00
ifa = ifa_maybe;
done:
return (ifa);
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
}
Resolve source address selection in presense of CARP. Add a couple of helper functions: - carp_master() - boolean function which is true if an address is in the MASTER state. - ifa_preferred() - boolean function that compares two addresses, and is aware of CARP. Utilize ifa_preferred() in ifa_ifwithnet(). The previous version of patch also changed source address selection logic in jails using carp_master(), but we failed to negotiate this part with Bjoern. May be we will approach this problem again later. Reported & tested by: Anton Yuzhaninov <citrin citrin.ru> Sponsored by: Nginx, Inc
2013-02-11 10:58:22 +00:00
/*
* See whether new ifa is better than current one:
* 1) A non-virtual one is preferred over virtual.
* 2) A virtual in master state preferred over any other state.
*
* Used in several address selecting functions.
*/
int
ifa_preferred(struct ifaddr *cur, struct ifaddr *next)
{
return (cur->ifa_carp && (!next->ifa_carp ||
((*carp_master_p)(next) && !(*carp_master_p)(cur))));
}
Simplify filling sockaddr_dl structure for if_resolvemulti() callback providers. link_init_sdl() function can be used to fill most of the parameters. Use caller stack instead of allocation / freing memory for each request. Do not drop support for extra-long (probably non-existing) link-layer protocols by introducing link_alloc_sdl() (used by if_resolvemulti() callback) and link_free_sdl() (used by caller). Since this change breaks KBI, MFC requires slightly different approach (link_init_sdl() auto-allocating buffer if necessary to handle cases with unmodified if_resolvemulti() callers). MFC after: 2 weeks
2014-01-18 23:24:51 +00:00
struct sockaddr_dl *
link_alloc_sdl(size_t size, int flags)
{
return (malloc(size, M_TEMP, flags));
}
void
link_free_sdl(struct sockaddr *sa)
{
free(sa, M_TEMP);
}
/*
* Fills in given sdl with interface basic info.
* Returns pointer to filled sdl.
*/
struct sockaddr_dl *
link_init_sdl(struct ifnet *ifp, struct sockaddr *paddr, u_char iftype)
{
struct sockaddr_dl *sdl;
sdl = (struct sockaddr_dl *)paddr;
memset(sdl, 0, sizeof(struct sockaddr_dl));
sdl->sdl_len = sizeof(struct sockaddr_dl);
sdl->sdl_family = AF_LINK;
sdl->sdl_index = ifp->if_index;
sdl->sdl_type = iftype;
return (sdl);
}
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
/*
* Mark an interface down and notify protocols of
* the transition.
*/
Make if_(un)route static in if.c as they are called from if_up/if_down only. This is also cleanup to make locking easier. Reviewed by: luigi Approved by: bms(mentor)
2004-04-18 18:59:44 +00:00
static void
Clean up whitespace, remove "register" keyword, ANSIfy. No functional changes.
2003-10-23 13:49:10 +00:00
if_unroute(struct ifnet *ifp, int flag, int fam)
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
{
Clean up whitespace, remove "register" keyword, ANSIfy. No functional changes.
2003-10-23 13:49:10 +00:00
struct ifaddr *ifa;
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
Rename IFF_RUNNING to IFF_DRV_RUNNING, IFF_OACTIVE to IFF_DRV_OACTIVE, and move both flags from ifnet.if_flags to ifnet.if_drv_flags, making and documenting the locking of these flags the responsibility of the device driver, not the network stack. The flags for these two fields will be mutually exclusive so that they can be exposed to user space as though they were stored in the same variable. Provide #defines to provide the old names #ifndef _KERNEL, so that user applications (such as ifconfig) can use the old flag names. Using the old names in a device driver will result in a compile error in order to help device driver writers adopt the new model. When exposing the interface flags to user space, via interface ioctls or routing sockets, or the two fields together. Since the driver flags cannot currently be set for user space, no new logic is currently required to handle this case. Add some assertions that general purpose network stack routines, such as if_setflags(), are not improperly used on driver-owned flags. With this change, a large number of very minor network stack races are closed, subject to correct device driver locking. Most were likely never triggered. Driver sweep to follow; many thanks to pjd and bz for the line-by-line review they gave this patch. Reviewed by: pjd, bz MFC after: 7 days
2005-08-09 10:16:17 +00:00
KASSERT(flag == IFF_UP, ("if_unroute: flag != IFF_UP"));
Generalize the if_up() and if_down() functions under the names if_route() and if_unroute(). This is first step towards sanitizing IFF_UP and IFF_RUNNING
1998-12-16 18:30:43 +00:00
ifp->if_flags &= ~flag;
Use getmicrotime() for if_lastchange, 10msec is plenty precision.
1998-04-06 11:43:12 +00:00
getmicrotime(&ifp->if_lastchange);
ifnet: Replace if_addr_lock rwlock with epoch + mutex Run on LLNW canaries and tested by pho@ gallatin: Using a 14-core, 28-HTT single socket E5-2697 v3 with a 40GbE MLX5 based ConnectX 4-LX NIC, I see an almost 12% improvement in received packet rate, and a larger improvement in bytes delivered all the way to userspace. When the host receiving 64 streams of netperf -H $DUT -t UDP_STREAM -- -m 1, I see, using nstat -I mce0 1 before the patch: InMpps OMpps InGbs OGbs err TCP Est %CPU syscalls csw irq GBfree 4.98 0.00 4.42 0.00 4235592 33 83.80 4720653 2149771 1235 247.32 4.73 0.00 4.20 0.00 4025260 33 82.99 4724900 2139833 1204 247.32 4.72 0.00 4.20 0.00 4035252 33 82.14 4719162 2132023 1264 247.32 4.71 0.00 4.21 0.00 4073206 33 83.68 4744973 2123317 1347 247.32 4.72 0.00 4.21 0.00 4061118 33 80.82 4713615 2188091 1490 247.32 4.72 0.00 4.21 0.00 4051675 33 85.29 4727399 2109011 1205 247.32 4.73 0.00 4.21 0.00 4039056 33 84.65 4724735 2102603 1053 247.32 After the patch InMpps OMpps InGbs OGbs err TCP Est %CPU syscalls csw irq GBfree 5.43 0.00 4.20 0.00 3313143 33 84.96 5434214 1900162 2656 245.51 5.43 0.00 4.20 0.00 3308527 33 85.24 5439695 1809382 2521 245.51 5.42 0.00 4.19 0.00 3316778 33 87.54 5416028 1805835 2256 245.51 5.42 0.00 4.19 0.00 3317673 33 90.44 5426044 1763056 2332 245.51 5.42 0.00 4.19 0.00 3314839 33 88.11 5435732 1792218 2499 245.52 5.44 0.00 4.19 0.00 3293228 33 91.84 5426301 1668597 2121 245.52 Similarly, netperf reports 230Mb/s before the patch, and 270Mb/s after the patch Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15366
2018-05-18 20:13:34 +00:00
CK_STAILQ_FOREACH(ifa, &ifp->if_addrhead, ifa_link)
Generalize the if_up() and if_down() functions under the names if_route() and if_unroute(). This is first step towards sanitizing IFF_UP and IFF_RUNNING
1998-12-16 18:30:43 +00:00
if (fam == PF_UNSPEC || (fam == ifa->ifa_addr->sa_family))
pfctlinput(PRC_IFDOWN, ifa->ifa_addr);
- bump __FreeBSD version to reflect added buf_ring, memory barriers, and ifnet functions - add memory barriers to <machine/atomic.h> - update drivers to only conditionally define their own - add lockless producer / consumer ring buffer - remove ring buffer implementation from cxgb and update its callers - add if_transmit(struct ifnet *ifp, struct mbuf *m) to ifnet to allow drivers to efficiently manage multiple hardware queues (i.e. not serialize all packets through one ifq) - expose if_qflush to allow drivers to flush any driver managed queues This work was supported by Bitgravity Inc. and Chelsio Inc.
2008-11-22 05:55:56 +00:00
ifp->if_qflush(ifp);
export if_qflush for use by driver if_qflush routines only set ifp->if_{transmit, qflush} if not already set KASSERT that neither or both are set
2009-04-16 23:05:10 +00:00
Add CARP (Common Address Redundancy Protocol), which allows multiple hosts to share an IP address, providing high availability and load balancing. Original work on CARP done by Michael Shalayeff, with many additions by Marco Pfatschbacher and Ryan McBride. FreeBSD port done solely by Max Laier. Patch by: mlaier Obtained from: OpenBSD (mickey, mcbride)
2005-02-22 13:04:05 +00:00
if (ifp->if_carp)
Allow carp(4) to be loaded as a kernel module. Follow precedent set by bridge(4), lagg(4) etc. and make use of function pointers and pf_proto_register() to hook carp into the network stack. Currently, because of the uncertainty about whether the unload path is free of race condition panics, unloads are disallowed by default. Compiling with CARPMOD_CAN_UNLOAD in CFLAGS removes this anti foot shooting measure. This commit requires IP6PROTOSPACER, introduced in r211115. Reviewed by: bz, simon Approved by: ken (mentor) MFC after: 2 weeks
2010-08-11 00:51:50 +00:00
(*carp_linkstate_p)(ifp);
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
rt_ifmsg(ifp);
}
/*
* Mark an interface up and notify protocols of
* the transition.
*/
Make if_(un)route static in if.c as they are called from if_up/if_down only. This is also cleanup to make locking easier. Reviewed by: luigi Approved by: bms(mentor)
2004-04-18 18:59:44 +00:00
static void
Clean up whitespace, remove "register" keyword, ANSIfy. No functional changes.
2003-10-23 13:49:10 +00:00
if_route(struct ifnet *ifp, int flag, int fam)
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
{
Clean up whitespace, remove "register" keyword, ANSIfy. No functional changes.
2003-10-23 13:49:10 +00:00
struct ifaddr *ifa;
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
Rename IFF_RUNNING to IFF_DRV_RUNNING, IFF_OACTIVE to IFF_DRV_OACTIVE, and move both flags from ifnet.if_flags to ifnet.if_drv_flags, making and documenting the locking of these flags the responsibility of the device driver, not the network stack. The flags for these two fields will be mutually exclusive so that they can be exposed to user space as though they were stored in the same variable. Provide #defines to provide the old names #ifndef _KERNEL, so that user applications (such as ifconfig) can use the old flag names. Using the old names in a device driver will result in a compile error in order to help device driver writers adopt the new model. When exposing the interface flags to user space, via interface ioctls or routing sockets, or the two fields together. Since the driver flags cannot currently be set for user space, no new logic is currently required to handle this case. Add some assertions that general purpose network stack routines, such as if_setflags(), are not improperly used on driver-owned flags. With this change, a large number of very minor network stack races are closed, subject to correct device driver locking. Most were likely never triggered. Driver sweep to follow; many thanks to pjd and bz for the line-by-line review they gave this patch. Reviewed by: pjd, bz MFC after: 7 days
2005-08-09 10:16:17 +00:00
KASSERT(flag == IFF_UP, ("if_route: flag != IFF_UP"));
Generalize the if_up() and if_down() functions under the names if_route() and if_unroute(). This is first step towards sanitizing IFF_UP and IFF_RUNNING
1998-12-16 18:30:43 +00:00
ifp->if_flags |= flag;
Use getmicrotime() for if_lastchange, 10msec is plenty precision.
1998-04-06 11:43:12 +00:00
getmicrotime(&ifp->if_lastchange);
ifnet: Replace if_addr_lock rwlock with epoch + mutex Run on LLNW canaries and tested by pho@ gallatin: Using a 14-core, 28-HTT single socket E5-2697 v3 with a 40GbE MLX5 based ConnectX 4-LX NIC, I see an almost 12% improvement in received packet rate, and a larger improvement in bytes delivered all the way to userspace. When the host receiving 64 streams of netperf -H $DUT -t UDP_STREAM -- -m 1, I see, using nstat -I mce0 1 before the patch: InMpps OMpps InGbs OGbs err TCP Est %CPU syscalls csw irq GBfree 4.98 0.00 4.42 0.00 4235592 33 83.80 4720653 2149771 1235 247.32 4.73 0.00 4.20 0.00 4025260 33 82.99 4724900 2139833 1204 247.32 4.72 0.00 4.20 0.00 4035252 33 82.14 4719162 2132023 1264 247.32 4.71 0.00 4.21 0.00 4073206 33 83.68 4744973 2123317 1347 247.32 4.72 0.00 4.21 0.00 4061118 33 80.82 4713615 2188091 1490 247.32 4.72 0.00 4.21 0.00 4051675 33 85.29 4727399 2109011 1205 247.32 4.73 0.00 4.21 0.00 4039056 33 84.65 4724735 2102603 1053 247.32 After the patch InMpps OMpps InGbs OGbs err TCP Est %CPU syscalls csw irq GBfree 5.43 0.00 4.20 0.00 3313143 33 84.96 5434214 1900162 2656 245.51 5.43 0.00 4.20 0.00 3308527 33 85.24 5439695 1809382 2521 245.51 5.42 0.00 4.19 0.00 3316778 33 87.54 5416028 1805835 2256 245.51 5.42 0.00 4.19 0.00 3317673 33 90.44 5426044 1763056 2332 245.51 5.42 0.00 4.19 0.00 3314839 33 88.11 5435732 1792218 2499 245.52 5.44 0.00 4.19 0.00 3293228 33 91.84 5426301 1668597 2121 245.52 Similarly, netperf reports 230Mb/s before the patch, and 270Mb/s after the patch Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15366
2018-05-18 20:13:34 +00:00
CK_STAILQ_FOREACH(ifa, &ifp->if_addrhead, ifa_link)
Generalize the if_up() and if_down() functions under the names if_route() and if_unroute(). This is first step towards sanitizing IFF_UP and IFF_RUNNING
1998-12-16 18:30:43 +00:00
if (fam == PF_UNSPEC || (fam == ifa->ifa_addr->sa_family))
pfctlinput(PRC_IFUP, ifa->ifa_addr);
Add CARP (Common Address Redundancy Protocol), which allows multiple hosts to share an IP address, providing high availability and load balancing. Original work on CARP done by Michael Shalayeff, with many additions by Marco Pfatschbacher and Ryan McBride. FreeBSD port done solely by Max Laier. Patch by: mlaier Obtained from: OpenBSD (mickey, mcbride)
2005-02-22 13:04:05 +00:00
if (ifp->if_carp)
Allow carp(4) to be loaded as a kernel module. Follow precedent set by bridge(4), lagg(4) etc. and make use of function pointers and pf_proto_register() to hook carp into the network stack. Currently, because of the uncertainty about whether the unload path is free of race condition panics, unloads are disallowed by default. Compiling with CARPMOD_CAN_UNLOAD in CFLAGS removes this anti foot shooting measure. This commit requires IP6PROTOSPACER, introduced in r211115. Reviewed by: bz, simon Approved by: ken (mentor) MFC after: 2 weeks
2010-08-11 00:51:50 +00:00
(*carp_linkstate_p)(ifp);
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
rt_ifmsg(ifp);
KAME netinet6 basic part(no IPsec,no V6 Multicast Forwarding, no UDP/TCP for IPv6 yet) With this patch, you can assigne IPv6 addr automatically, and can reply to IPv6 ping. Reviewed by: freebsd-arch, cvs-committers Obtained from: KAME project
1999-11-22 02:45:11 +00:00
#ifdef INET6
in6_if_up(ifp);
#endif
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
}
The devices that supported EVFILT_NETDEV kqueue filters were removed in r195175. Remove all definitions, documentation, and usage. fifo_misc.c: Remove all kqueue tests as fifo_io.c performs all those that would have remained. Reviewed by: rwatson MFC after: 3 weeks X-MFC note: don't change vlan_link_state() function signature
2009-12-31 20:29:58 +00:00
void (*vlan_link_state_p)(struct ifnet *); /* XXX: private from if_vlan */
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective work by yar, thompsa and myself. The checksum offloading part also involves work done by Mihail Balikov. The most important changes: o Instead of global linked list of all vlan softc use a per-trunk hash. The size of hash is dynamically adjusted, depending on number of entries. This changes struct ifnet, replacing counter of vlans with a pointer to trunk structure. This change is an improvement for setups with big number of VLANs, several interfaces and several CPUs. It is a small regression for a setup with a single VLAN interface. An alternative to dynamic hash is a per-trunk static array with 4096 entries, which is a compile time option - VLAN_ARRAY. In my experiments the array is not an improvement, probably because such a big trunk structure doesn't fit into CPU cache. o Introduce an UMA zone for VLAN tags. Since drivers depend on it, the zone is declared in kern_mbuf.c, not in optional vlan(4) driver. This change is a big improvement for any setup utilizing vlan(4). o Use rwlock(9) instead of mutex(9) for locking. We are the first ones to do this! :) o Some drivers can do hardware VLAN tagging + hardware checksum offloading. Add an infrastructure for this. Whenever vlan(4) is attached to a parent or parent configuration is changed, the flags on vlan(4) interface are updated. In collaboration with: yar, thompsa In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
void (*vlan_trunk_cap_p)(struct ifnet *); /* XXX: private from if_vlan */
- Merge changes to the base system to support OFED. These include a wider arg2 for sysctl, updates to vlan code, IFT_INFINIBAND, and other miscellaneous small features.
2011-03-21 09:40:01 +00:00
struct ifnet *(*vlan_trunkdev_p)(struct ifnet *);
struct ifnet *(*vlan_devat_p)(struct ifnet *, uint16_t);
int (*vlan_tag_p)(struct ifnet *, uint16_t *);
Add the ability to look up the 3b PCP of a VLAN interface. Use it in toe_l2_resolve to fill up the complete vtag and not just the vid. Reviewed by: kib@ MFC after: 1 week Sponsored by: Chelsio Communications Differential Revision: https://reviews.freebsd.org/D16752
2018-08-16 23:46:38 +00:00
int (*vlan_pcp_p)(struct ifnet *, uint16_t *);
- Merge changes to the base system to support OFED. These include a wider arg2 for sysctl, updates to vlan code, IFT_INFINIBAND, and other miscellaneous small features.
2011-03-21 09:40:01 +00:00
int (*vlan_setcookie_p)(struct ifnet *, void *);
void *(*vlan_cookie_p)(struct ifnet *);
Cleanup link state change notification: o add new if_link_state_change routine that deals with link state changes o change mii to use if_link_state_change
2004-12-08 05:45:59 +00:00
/*
Do not call all link state callbacks directly, but schedule a taskqueue(9) task. This fixes LORs and adds possibility to serve such events pseudorecursively, when link state change of interface causes subsequent change on other interfaces. Sponsored by: Rambler Reviewed by: sam, brooks, mux
2005-04-20 09:30:54 +00:00
* Handle a change in the interface link state. To avoid LORs
* between driver lock and upper layer locks, as well as possible
* recursions, we post event to taskqueue, and all job
* is done in static do_link_state_change().
Cleanup link state change notification: o add new if_link_state_change routine that deals with link state changes o change mii to use if_link_state_change
2004-12-08 05:45:59 +00:00
*/
void
Revert r292275 & r292379 glebius has concerns about these changes so reverting those can be discussed and addressed. Sponsored by: Multiplay
2015-12-17 14:41:30 +00:00
if_link_state_change(struct ifnet *ifp, int link_state)
Cleanup link state change notification: o add new if_link_state_change routine that deals with link state changes o change mii to use if_link_state_change
2004-12-08 05:45:59 +00:00
{
Revert r292275 & r292379 glebius has concerns about these changes so reverting those can be discussed and addressed. Sponsored by: Multiplay
2015-12-17 14:41:30 +00:00
/* Return if state hasn't changed. */
if (ifp->if_link_state == link_state)
- In if_link_state_change() extract function body from if-block, to improve readability. - Call carp_carpdev_state() from if_link_state_change() if interface has associated CARP interface. Sponsored by: Rambler
2005-02-22 14:21:59 +00:00
return;
ifp->if_link_state = link_state;
Widen NET_EPOCH coverage. When epoch(9) was introduced to network stack, it was basically dropped in place of existing locking, which was mutexes and rwlocks. For the sake of performance mutex covered areas were as small as possible, so became epoch covered areas. However, epoch doesn't introduce any contention, it just delays memory reclaim. So, there is no point to minimise epoch covered areas in sense of performance. Meanwhile entering/exiting epoch also has non-zero CPU usage, so doing this less often is a win. Not the least is also code maintainability. In the new paradigm we can assume that at any stage of processing a packet, we are inside network epoch. This makes coding both input and output path way easier. On output path we already enter epoch quite early - in the ip_output(), in the ip6_output(). This patch does the same for the input path. All ISR processing, network related callouts, other ways of packet injection to the network stack shall be performed in net_epoch. Any leaf function that walks network configuration now asserts epoch. Tricky part is configuration code paths - ioctls, sysctls. They also call into leaf functions, so some need to be changed. This patch would introduce more epoch recursions (see EPOCH_TRACE) than we had before. They will be cleaned up separately, as several of them aren't trivial. Note, that unlike a lock recursion the epoch recursion is safe and just wastes a bit of resources. Reviewed by: gallatin, hselasky, cy, adrian, kristof Differential Revision: https://reviews.freebsd.org/D19111
2019-10-07 22:40:05 +00:00
/* XXXGL: reference ifp? */
Do not call all link state callbacks directly, but schedule a taskqueue(9) task. This fixes LORs and adds possibility to serve such events pseudorecursively, when link state change of interface causes subsequent change on other interfaces. Sponsored by: Rambler Reviewed by: sam, brooks, mux
2005-04-20 09:30:54 +00:00
taskqueue_enqueue(taskqueue_swi, &ifp->if_linktask);
}
static void
do_link_state_change(void *arg, int pending)
{
Widen NET_EPOCH coverage. When epoch(9) was introduced to network stack, it was basically dropped in place of existing locking, which was mutexes and rwlocks. For the sake of performance mutex covered areas were as small as possible, so became epoch covered areas. However, epoch doesn't introduce any contention, it just delays memory reclaim. So, there is no point to minimise epoch covered areas in sense of performance. Meanwhile entering/exiting epoch also has non-zero CPU usage, so doing this less often is a win. Not the least is also code maintainability. In the new paradigm we can assume that at any stage of processing a packet, we are inside network epoch. This makes coding both input and output path way easier. On output path we already enter epoch quite early - in the ip_output(), in the ip6_output(). This patch does the same for the input path. All ISR processing, network related callouts, other ways of packet injection to the network stack shall be performed in net_epoch. Any leaf function that walks network configuration now asserts epoch. Tricky part is configuration code paths - ioctls, sysctls. They also call into leaf functions, so some need to be changed. This patch would introduce more epoch recursions (see EPOCH_TRACE) than we had before. They will be cleaned up separately, as several of them aren't trivial. Note, that unlike a lock recursion the epoch recursion is safe and just wastes a bit of resources. Reviewed by: gallatin, hselasky, cy, adrian, kristof Differential Revision: https://reviews.freebsd.org/D19111
2019-10-07 22:40:05 +00:00
struct ifnet *ifp;
int link_state;
ifp = arg;
link_state = ifp->if_link_state;
Do not call all link state callbacks directly, but schedule a taskqueue(9) task. This fixes LORs and adds possibility to serve such events pseudorecursively, when link state change of interface causes subsequent change on other interfaces. Sponsored by: Rambler Reviewed by: sam, brooks, mux
2005-04-20 09:30:54 +00:00
Widen NET_EPOCH coverage. When epoch(9) was introduced to network stack, it was basically dropped in place of existing locking, which was mutexes and rwlocks. For the sake of performance mutex covered areas were as small as possible, so became epoch covered areas. However, epoch doesn't introduce any contention, it just delays memory reclaim. So, there is no point to minimise epoch covered areas in sense of performance. Meanwhile entering/exiting epoch also has non-zero CPU usage, so doing this less often is a win. Not the least is also code maintainability. In the new paradigm we can assume that at any stage of processing a packet, we are inside network epoch. This makes coding both input and output path way easier. On output path we already enter epoch quite early - in the ip_output(), in the ip6_output(). This patch does the same for the input path. All ISR processing, network related callouts, other ways of packet injection to the network stack shall be performed in net_epoch. Any leaf function that walks network configuration now asserts epoch. Tricky part is configuration code paths - ioctls, sysctls. They also call into leaf functions, so some need to be changed. This patch would introduce more epoch recursions (see EPOCH_TRACE) than we had before. They will be cleaned up separately, as several of them aren't trivial. Note, that unlike a lock recursion the epoch recursion is safe and just wastes a bit of resources. Reviewed by: gallatin, hselasky, cy, adrian, kristof Differential Revision: https://reviews.freebsd.org/D19111
2019-10-07 22:40:05 +00:00
CURVNET_SET(ifp->if_vnet);
- In if_link_state_change() extract function body from if-block, to improve readability. - Call carp_carpdev_state() from if_link_state_change() if interface has associated CARP interface. Sponsored by: Rambler
2005-02-22 14:21:59 +00:00
rt_ifmsg(ifp);
Merge the //depot/user/yar/vlan branch into CVS. It contains some collective work by yar, thompsa and myself. The checksum offloading part also involves work done by Mihail Balikov. The most important changes: o Instead of global linked list of all vlan softc use a per-trunk hash. The size of hash is dynamically adjusted, depending on number of entries. This changes struct ifnet, replacing counter of vlans with a pointer to trunk structure. This change is an improvement for setups with big number of VLANs, several interfaces and several CPUs. It is a small regression for a setup with a single VLAN interface. An alternative to dynamic hash is a per-trunk static array with 4096 entries, which is a compile time option - VLAN_ARRAY. In my experiments the array is not an improvement, probably because such a big trunk structure doesn't fit into CPU cache. o Introduce an UMA zone for VLAN tags. Since drivers depend on it, the zone is declared in kern_mbuf.c, not in optional vlan(4) driver. This change is a big improvement for any setup utilizing vlan(4). o Use rwlock(9) instead of mutex(9) for locking. We are the first ones to do this! :) o Some drivers can do hardware VLAN tagging + hardware checksum offloading. Add an infrastructure for this. Whenever vlan(4) is attached to a parent or parent configuration is changed, the flags on vlan(4) interface are updated. In collaboration with: yar, thompsa In collaboration with: Mihail Balikov <mihail.balikov interbgc.com>
2006-01-30 13:45:15 +00:00
if (ifp->if_vlantrunk != NULL)
The devices that supported EVFILT_NETDEV kqueue filters were removed in r195175. Remove all definitions, documentation, and usage. fifo_misc.c: Remove all kqueue tests as fifo_io.c performs all those that would have remained. Reviewed by: rwatson MFC after: 3 weeks X-MFC note: don't change vlan_link_state() function signature
2009-12-31 20:29:58 +00:00
(*vlan_link_state_p)(ifp);
- In if_link_state_change() extract function body from if-block, to improve readability. - Call carp_carpdev_state() from if_link_state_change() if interface has associated CARP interface. Sponsored by: Rambler
2005-02-22 14:21:59 +00:00
if ((ifp->if_type == IFT_ETHER || ifp->if_type == IFT_L2VLAN) &&
Remove struct arpcom. It is unused by most interface types, that allocate it, except Ethernet, where it carried ng_ether(4) pointer. For now carry the pointer in if_l2com directly. Sponsored by: Netflix Sponsored by: Nginx, Inc.
2014-11-07 15:14:10 +00:00
ifp->if_l2com != NULL)
- In if_link_state_change() extract function body from if-block, to improve readability. - Call carp_carpdev_state() from if_link_state_change() if interface has associated CARP interface. Sponsored by: Rambler
2005-02-22 14:21:59 +00:00
(*ng_ether_link_state_p)(ifp, link_state);
if (ifp->if_carp)
Allow carp(4) to be loaded as a kernel module. Follow precedent set by bridge(4), lagg(4) etc. and make use of function pointers and pf_proto_register() to hook carp into the network stack. Currently, because of the uncertainty about whether the unload path is free of race condition panics, unloads are disallowed by default. Compiling with CARPMOD_CAN_UNLOAD in CFLAGS removes this anti foot shooting measure. This commit requires IP6PROTOSPACER, introduced in r211115. Reviewed by: bz, simon Approved by: ken (mentor) MFC after: 2 weeks
2010-08-11 00:51:50 +00:00
(*carp_linkstate_p)(ifp);
Remove KASSERTS, they do not add any value here since the pointer is about to be derefernced anyway.
2012-04-18 01:39:14 +00:00
if (ifp->if_bridge)
Allow different bridge types to coexist if_bridge has a lot of limitations that make it scale poorly to higher data rates. In my projects/VPC branch I leverage the bridge interface between layers for my high speed soft switch as well as for purposes of stacking in general. Reviewed by: sbruno@ Approved by: sbruno@ Differential Revision: https://reviews.freebsd.org/D15344
2018-05-11 05:00:40 +00:00
ifp->if_bridge_linkstate(ifp);
Remove KASSERTS, they do not add any value here since the pointer is about to be derefernced anyway.
2012-04-18 01:39:14 +00:00
if (ifp->if_lagg)
Revert r292275 & r292379 glebius has concerns about these changes so reverting those can be discussed and addressed. Sponsored by: Multiplay
2015-12-17 14:41:30 +00:00
(*lagg_linkstate_p)(ifp, link_state);
Add hooks into the networking layer to support if_bridge. This changes struct ifnet so a buildworld is necessary. Approved by: mlaier (mentor) Obtained from: NetBSD
2005-06-05 03:13:13 +00:00
Change the curvnet variable from a global const struct vnet *, previously always pointing to the default vnet context, to a dynamically changing thread-local one. The currvnet context should be set on entry to networking code via CURVNET_SET() macros, and reverted to previous state via CURVNET_RESTORE(). Recursions on curvnet are permitted, though strongly discuouraged. This change should have no functional impact on nooptions VIMAGE kernel builds, where CURVNET_* macros expand to whitespace. The curthread->td_vnet (aka curvnet) variable's purpose is to be an indicator of the vnet context in which the current network-related operation takes place, in case we cannot deduce the current vnet context from any other source, such as by looking at mbuf's m->m_pkthdr.rcvif->if_vnet, sockets's so->so_vnet etc. Moreover, so far curvnet has turned out to be an invaluable consistency checking aid: it helps to catch cases when sockets, ifnets or any other vnet-aware structures may have leaked from one vnet to another. The exact placement of the CURVNET_SET() / CURVNET_RESTORE() macros was a result of an empirical iterative process, whith an aim to reduce recursions on CURVNET_SET() to a minimum, while still reducing the scope of CURVNET_SET() to networking only operations - the alternative would be calling CURVNET_SET() on each system call entry. In general, curvnet has to be set in three typicall cases: when processing socket-related requests from userspace or from within the kernel; when processing inbound traffic flowing from device drivers to upper layers of the networking stack, and when executing timer-driven networking functions. This change also introduces a DDB subcommand to show the list of all vnet instances. Approved by: julian (mentor)
2009-05-05 10:56:12 +00:00
if (IS_DEFAULT_VNET(curvnet))
devctl_notify("IFNET", ifp->if_xname,
(link_state == LINK_STATE_UP) ? "LINK_UP" : "LINK_DOWN",
NULL);
Do not call all link state callbacks directly, but schedule a taskqueue(9) task. This fixes LORs and adds possibility to serve such events pseudorecursively, when link state change of interface causes subsequent change on other interfaces. Sponsored by: Rambler Reviewed by: sam, brooks, mux
2005-04-20 09:30:54 +00:00
if (pending > 1)
if_printf(ifp, "%d link states coalesced\n", pending);
Add a sysctl net.link.log_link_state_change, which allows to suppress logging of interface link state changes. Requested by: sam, kan
2005-03-12 12:58:03 +00:00
if (log_link_state_change)
Slight cleanup of interface event logging. Make if_printf() use vlog() instead of vprintf(). This means it can no longer return the number of characters printed, as it used to, but every single call to if_printf() in the entire kernel ignores the return value anyway; just return 0 so we don't have to change the prototype. Consistently use if_printf() throughout sys/net/if.c, instead of a mixture of if_printf() and log(). In ifa_maintain_loopback_route(), don't needlessly log an error if we either failed to add a route because it already existed or failed to remove one because it did not. We still return an error code, though. MFC after: 1 week
2018-05-11 00:19:49 +00:00
if_printf(ifp, "link state changed to %s\n",
Add a sysctl net.link.log_link_state_change, which allows to suppress logging of interface link state changes. Requested by: sam, kan
2005-03-12 12:58:03 +00:00
(link_state == LINK_STATE_UP) ? "UP" : "DOWN" );
ifnet: Use if_link_state snapshot to invoke ifnet_link_event So that everyone in this task have consistent view of link state. Reviewed by: ae MFC after: 1 week Sponsored by: Microsoft Differential Revision: https://reviews.freebsd.org/D8214
2016-10-12 01:52:29 +00:00
EVENTHANDLER_INVOKE(ifnet_link_event, ifp, link_state);
Step 1.5 of importing the network stack virtualization infrastructure from the vimage project, as per plan established at devsummit 08/08: http://wiki.freebsd.org/Image/Notes200808DevSummit Introduce INIT_VNET_*() initializer macros, VNET_FOREACH() iterator macros, and CURVNET_SET() context setting macros, all currently resolving to NOPs. Prepare for virtualization of selected SYSCTL objects by introducing a family of SYSCTL_V_*() macros, currently resolving to their global counterparts, i.e. SYSCTL_V_INT() == SYSCTL_INT(). Move selected #defines from sys/sys/vimage.h to newly introduced header files specific to virtualized subsystems (sys/net/vnet.h, sys/netinet/vinet.h etc.). All the changes are verified to have zero functional impact at this point in time by doing MD5 comparision between pre- and post-change object files(*). (*) netipsec/keysock.c did not validate depending on compile time options. Implemented by: julian, bz, brooks, zec Reviewed by: julian, bz, brooks, kris, rwatson, ... Approved by: julian (mentor) Obtained from: //depot/projects/vimage-commit2/... X-MFC after: never Sponsored by: NLnet Foundation, The FreeBSD Foundation
2008-10-02 15:37:58 +00:00
CURVNET_RESTORE();
Cleanup link state change notification: o add new if_link_state_change routine that deals with link state changes o change mii to use if_link_state_change
2004-12-08 05:45:59 +00:00
}
Generalize the if_up() and if_down() functions under the names if_route() and if_unroute(). This is first step towards sanitizing IFF_UP and IFF_RUNNING
1998-12-16 18:30:43 +00:00
/*
* Mark an interface down and notify protocols of
* the transition.
*/
void
Clean up whitespace, remove "register" keyword, ANSIfy. No functional changes.
2003-10-23 13:49:10 +00:00
if_down(struct ifnet *ifp)
Generalize the if_up() and if_down() functions under the names if_route() and if_unroute(). This is first step towards sanitizing IFF_UP and IFF_RUNNING
1998-12-16 18:30:43 +00:00
{
ifnet: introduce event handlers for ifup/ifdown events Hyper-V's NIC SR-IOV implementation needs a Hyper-V synthetic NIC and a VF NIC to work together, mainly to support seamless live migration. When the VF device becomes UP (or DOWN), the synthetic NIC driver needs to switch the data path from the synthetic NIC to the VF (or the opposite). So the synthetic NIC driver needs to know when a VF device is becoming UP or DOWN and hence the patch is made. Reviewed by: sephe Approved by: sephe (mentor) MFC after: 2 weeks Sponsored by: Microsoft Differential Revision: https://reviews.freebsd.org/D8963
2017-01-24 09:19:46 +00:00
EVENTHANDLER_INVOKE(ifnet_event, ifp, IFNET_EVENT_DOWN);
Generalize the if_up() and if_down() functions under the names if_route() and if_unroute(). This is first step towards sanitizing IFF_UP and IFF_RUNNING
1998-12-16 18:30:43 +00:00
if_unroute(ifp, IFF_UP, AF_UNSPEC);
}
/*
* Mark an interface up and notify protocols of
* the transition.
*/
void
Clean up whitespace, remove "register" keyword, ANSIfy. No functional changes.
2003-10-23 13:49:10 +00:00
if_up(struct ifnet *ifp)
Generalize the if_up() and if_down() functions under the names if_route() and if_unroute(). This is first step towards sanitizing IFF_UP and IFF_RUNNING
1998-12-16 18:30:43 +00:00
{
if_route(ifp, IFF_UP, AF_UNSPEC);
ifnet: introduce event handlers for ifup/ifdown events Hyper-V's NIC SR-IOV implementation needs a Hyper-V synthetic NIC and a VF NIC to work together, mainly to support seamless live migration. When the VF device becomes UP (or DOWN), the synthetic NIC driver needs to switch the data path from the synthetic NIC to the VF (or the opposite). So the synthetic NIC driver needs to know when a VF device is becoming UP or DOWN and hence the patch is made. Reviewed by: sephe Approved by: sephe (mentor) MFC after: 2 weeks Sponsored by: Microsoft Differential Revision: https://reviews.freebsd.org/D8963
2017-01-24 09:19:46 +00:00
EVENTHANDLER_INVOKE(ifnet_event, ifp, IFNET_EVENT_UP);
Generalize the if_up() and if_down() functions under the names if_route() and if_unroute(). This is first step towards sanitizing IFF_UP and IFF_RUNNING
1998-12-16 18:30:43 +00:00
}
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
/*
* Flush an interface queue.
*/
export if_qflush for use by driver if_qflush routines only set ifp->if_{transmit, qflush} if not already set KASSERT that neither or both are set
2009-04-16 23:05:10 +00:00
void
- bump __FreeBSD version to reflect added buf_ring, memory barriers, and ifnet functions - add memory barriers to <machine/atomic.h> - update drivers to only conditionally define their own - add lockless producer / consumer ring buffer - remove ring buffer implementation from cxgb and update its callers - add if_transmit(struct ifnet *ifp, struct mbuf *m) to ifnet to allow drivers to efficiently manage multiple hardware queues (i.e. not serialize all packets through one ifq) - expose if_qflush to allow drivers to flush any driver managed queues This work was supported by Bitgravity Inc. and Chelsio Inc.
2008-11-22 05:55:56 +00:00
if_qflush(struct ifnet *ifp)
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
{
Clean up whitespace, remove "register" keyword, ANSIfy. No functional changes.
2003-10-23 13:49:10 +00:00
struct mbuf *m, *n;
- bump __FreeBSD version to reflect added buf_ring, memory barriers, and ifnet functions - add memory barriers to <machine/atomic.h> - update drivers to only conditionally define their own - add lockless producer / consumer ring buffer - remove ring buffer implementation from cxgb and update its callers - add if_transmit(struct ifnet *ifp, struct mbuf *m) to ifnet to allow drivers to efficiently manage multiple hardware queues (i.e. not serialize all packets through one ifq) - expose if_qflush to allow drivers to flush any driver managed queues This work was supported by Bitgravity Inc. and Chelsio Inc.
2008-11-22 05:55:56 +00:00
struct ifaltq *ifq;
ifq = &ifp->if_snd;
Fix an assertion when if_down()ing a ALTQ managed interface. The lock should have been in place all the time the mtx_assert in the ALTQ code just discovered the shortcoming. PR: i386/71195 Tested by: Bettan (PR originator), myself MFC after: 5 days
2004-09-01 19:56:47 +00:00
IFQ_LOCK(ifq);
Link ALTQ to the build and break with ABI for struct ifnet. Please recompile your (network) modules as well as any userland that might make sense of sizeof(struct ifnet). This does not change the queueing yet. These changes will follow in a seperate commit. Same with the driver changes, which need case by case evaluation. __FreeBSD_version bump will follow. Tested-by: (i386)LINT
2004-06-13 17:29:10 +00:00
#ifdef ALTQ
if (ALTQ_IS_ENABLED(ifq))
ALTQ_PURGE(ifq);
#endif
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
n = ifq->ifq_head;
sys/net* : for pointers replace 0 with NULL. Mostly cosmetical, no functional change. Found with devel/coccinelle.
2016-04-15 17:30:33 +00:00
while ((m = n) != NULL) {
Deprecate m_act. Use m_nextpkt always.
2014-07-17 05:21:16 +00:00
n = m->m_nextpkt;
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
m_freem(m);
}
ifq->ifq_head = 0;
ifq->ifq_tail = 0;
ifq->ifq_len = 0;
Fix an assertion when if_down()ing a ALTQ managed interface. The lock should have been in place all the time the mtx_assert in the ALTQ code just discovered the shortcoming. PR: i386/71195 Tested by: Bettan (PR originator), myself MFC after: 5 days
2004-09-01 19:56:47 +00:00
IFQ_UNLOCK(ifq);
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
}
/*
Add ifunit_ref(), a version of ifunit(), that returns not just an interface pointer, but also a reference to it. Modify ifioctl() to use ifunit_ref(), holding the reference until all ioctls, etc, have completed. This closes a class of reader-writer races in which interfaces could be removed during long-running ioctls, leading to crashes. Many other consumers of ifunit() should now use ifunit_ref() to avoid similar races. MFC after: 3 weeks
2009-04-23 13:08:47 +00:00
* Map interface name to interface structure pointer, with or without
* returning a reference.
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
*/
Add ifunit_ref(), a version of ifunit(), that returns not just an interface pointer, but also a reference to it. Modify ifioctl() to use ifunit_ref(), holding the reference until all ioctls, etc, have completed. This closes a class of reader-writer races in which interfaces could be removed during long-running ioctls, leading to crashes. Many other consumers of ifunit() should now use ifunit_ref() to avoid similar races. MFC after: 3 weeks
2009-04-23 13:08:47 +00:00
struct ifnet *
ifunit_ref(const char *name)
{
Mechanical cleanup of epoch(9) usage in network stack. - Remove macros that covertly create epoch_tracker on thread stack. Such macros a quite unsafe, e.g. will produce a buggy code if same macro is used in embedded scopes. Explicitly declare epoch_tracker always. - Unmask interface list IFNET_RLOCK_NOSLEEP(), interface address list IF_ADDR_RLOCK() and interface AF specific data IF_AFDATA_RLOCK() read locking macros to what they actually are - the net_epoch. Keeping them as is is very misleading. They all are named FOO_RLOCK(), while they no longer have lock semantics. Now they allow recursion and what's more important they now no longer guarantee protection against their companion WLOCK macros. Note: INP_HASH_RLOCK() has same problems, but not touched by this commit. This is non functional mechanical change. The only functionally changed functions are ni6_addrs() and ni6_store_addrs(), where we no longer enter epoch recursively. Discussed with: jtl, gallatin
2019-01-09 01:11:19 +00:00
struct epoch_tracker et;
Add ifunit_ref(), a version of ifunit(), that returns not just an interface pointer, but also a reference to it. Modify ifioctl() to use ifunit_ref(), holding the reference until all ioctls, etc, have completed. This closes a class of reader-writer races in which interfaces could be removed during long-running ioctls, leading to crashes. Many other consumers of ifunit() should now use ifunit_ref() to avoid similar races. MFC after: 3 weeks
2009-04-23 13:08:47 +00:00
struct ifnet *ifp;
Mechanical cleanup of epoch(9) usage in network stack. - Remove macros that covertly create epoch_tracker on thread stack. Such macros a quite unsafe, e.g. will produce a buggy code if same macro is used in embedded scopes. Explicitly declare epoch_tracker always. - Unmask interface list IFNET_RLOCK_NOSLEEP(), interface address list IF_ADDR_RLOCK() and interface AF specific data IF_AFDATA_RLOCK() read locking macros to what they actually are - the net_epoch. Keeping them as is is very misleading. They all are named FOO_RLOCK(), while they no longer have lock semantics. Now they allow recursion and what's more important they now no longer guarantee protection against their companion WLOCK macros. Note: INP_HASH_RLOCK() has same problems, but not touched by this commit. This is non functional mechanical change. The only functionally changed functions are ni6_addrs() and ni6_store_addrs(), where we no longer enter epoch recursively. Discussed with: jtl, gallatin
2019-01-09 01:11:19 +00:00
NET_EPOCH_ENTER(et);
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
CK_STAILQ_FOREACH(ifp, &V_ifnet, if_link) {
As with ifnet_byindex_ref(), don't return IFF_DYING interfaces from ifunit_ref(). ifunit() continues to return them. MFC after: 3 weeks
2009-04-23 15:56:01 +00:00
if (strncmp(name, ifp->if_xname, IFNAMSIZ) == 0 &&
!(ifp->if_flags & IFF_DYING))
Add ifunit_ref(), a version of ifunit(), that returns not just an interface pointer, but also a reference to it. Modify ifioctl() to use ifunit_ref(), holding the reference until all ioctls, etc, have completed. This closes a class of reader-writer races in which interfaces could be removed during long-running ioctls, leading to crashes. Many other consumers of ifunit() should now use ifunit_ref() to avoid similar races. MFC after: 3 weeks
2009-04-23 13:08:47 +00:00
break;
}
if (ifp != NULL)
if_ref(ifp);
Mechanical cleanup of epoch(9) usage in network stack. - Remove macros that covertly create epoch_tracker on thread stack. Such macros a quite unsafe, e.g. will produce a buggy code if same macro is used in embedded scopes. Explicitly declare epoch_tracker always. - Unmask interface list IFNET_RLOCK_NOSLEEP(), interface address list IF_ADDR_RLOCK() and interface AF specific data IF_AFDATA_RLOCK() read locking macros to what they actually are - the net_epoch. Keeping them as is is very misleading. They all are named FOO_RLOCK(), while they no longer have lock semantics. Now they allow recursion and what's more important they now no longer guarantee protection against their companion WLOCK macros. Note: INP_HASH_RLOCK() has same problems, but not touched by this commit. This is non functional mechanical change. The only functionally changed functions are ni6_addrs() and ni6_store_addrs(), where we no longer enter epoch recursively. Discussed with: jtl, gallatin
2019-01-09 01:11:19 +00:00
NET_EPOCH_EXIT(et);
Add ifunit_ref(), a version of ifunit(), that returns not just an interface pointer, but also a reference to it. Modify ifioctl() to use ifunit_ref(), holding the reference until all ioctls, etc, have completed. This closes a class of reader-writer races in which interfaces could be removed during long-running ioctls, leading to crashes. Many other consumers of ifunit() should now use ifunit_ref() to avoid similar races. MFC after: 3 weeks
2009-04-23 13:08:47 +00:00
return (ifp);
}
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
struct ifnet *
Add kernel infrastructure for network device cloning. Reviewed by: ru, ume Obtained from: NetBSD MFC after: 1 week
2001-07-02 20:49:25 +00:00
ifunit(const char *name)
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
{
Mechanical cleanup of epoch(9) usage in network stack. - Remove macros that covertly create epoch_tracker on thread stack. Such macros a quite unsafe, e.g. will produce a buggy code if same macro is used in embedded scopes. Explicitly declare epoch_tracker always. - Unmask interface list IFNET_RLOCK_NOSLEEP(), interface address list IF_ADDR_RLOCK() and interface AF specific data IF_AFDATA_RLOCK() read locking macros to what they actually are - the net_epoch. Keeping them as is is very misleading. They all are named FOO_RLOCK(), while they no longer have lock semantics. Now they allow recursion and what's more important they now no longer guarantee protection against their companion WLOCK macros. Note: INP_HASH_RLOCK() has same problems, but not touched by this commit. This is non functional mechanical change. The only functionally changed functions are ni6_addrs() and ni6_store_addrs(), where we no longer enter epoch recursively. Discussed with: jtl, gallatin
2019-01-09 01:11:19 +00:00
struct epoch_tracker et;
Allow ifunit() routine to understand names like ed0f2. Also fix a bug caused by using bcmp() instead of strcmp(). Reviewed by: Garrett Wollman <wollman@khavrinen.lcs.mit.edu>
1999-12-13 15:57:11 +00:00
struct ifnet *ifp;
Cleanup ifunit(), so it uses the dev_named() function to map an interface name into a device.
2001-10-17 18:58:14 +00:00
Mechanical cleanup of epoch(9) usage in network stack. - Remove macros that covertly create epoch_tracker on thread stack. Such macros a quite unsafe, e.g. will produce a buggy code if same macro is used in embedded scopes. Explicitly declare epoch_tracker always. - Unmask interface list IFNET_RLOCK_NOSLEEP(), interface address list IF_ADDR_RLOCK() and interface AF specific data IF_AFDATA_RLOCK() read locking macros to what they actually are - the net_epoch. Keeping them as is is very misleading. They all are named FOO_RLOCK(), while they no longer have lock semantics. Now they allow recursion and what's more important they now no longer guarantee protection against their companion WLOCK macros. Note: INP_HASH_RLOCK() has same problems, but not touched by this commit. This is non functional mechanical change. The only functionally changed functions are ni6_addrs() and ni6_store_addrs(), where we no longer enter epoch recursively. Discussed with: jtl, gallatin
2019-01-09 01:11:19 +00:00
NET_EPOCH_ENTER(et);
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
CK_STAILQ_FOREACH(ifp, &V_ifnet, if_link) {
Add the kernel side of network interface renaming support. The basic process is to send a routing socket announcement that the interface has departed, change if_xname, update the sockaddr_dl associated with the interface, and announce the arrival of the interface on the routing socket. As part of this change, ifunit() is greatly simplified by testing if_xname directly. if_clone_destroy() now uses if_dname to look up the cloner for the interface and if_dunit to identify the unit number. Reviewed by: ru, sam (concept) Vincent Jardin <vjardin AT free.fr> Max Laier <max AT love2party.net>
2004-02-04 02:54:25 +00:00
if (strncmp(name, ifp->if_xname, IFNAMSIZ) == 0)
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
break;
}
Mechanical cleanup of epoch(9) usage in network stack. - Remove macros that covertly create epoch_tracker on thread stack. Such macros a quite unsafe, e.g. will produce a buggy code if same macro is used in embedded scopes. Explicitly declare epoch_tracker always. - Unmask interface list IFNET_RLOCK_NOSLEEP(), interface address list IF_ADDR_RLOCK() and interface AF specific data IF_AFDATA_RLOCK() read locking macros to what they actually are - the net_epoch. Keeping them as is is very misleading. They all are named FOO_RLOCK(), while they no longer have lock semantics. Now they allow recursion and what's more important they now no longer guarantee protection against their companion WLOCK macros. Note: INP_HASH_RLOCK() has same problems, but not touched by this commit. This is non functional mechanical change. The only functionally changed functions are ni6_addrs() and ni6_store_addrs(), where we no longer enter epoch recursively. Discussed with: jtl, gallatin
2019-01-09 01:11:19 +00:00
NET_EPOCH_EXIT(et);
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
return (ifp);
}
Fix access to ifru_buffer on freebsd32. Make all kernel accesses to ifru_buffer go via access functions which take the process ABI into account and use an appropriate union to access members in the correct place in struct ifreq. Reviewed by: kib Obtained from: CheriBSD MFC after: 1 week Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D14846
2018-03-27 18:26:50 +00:00
static void *
Remove the thread argument from ifr_buffer_*() accessors. They are always used in a context where curthread is the correct thread. This makes them more similar to the ifr_data_get_ptr() accessor.
2018-04-06 23:25:54 +00:00
ifr_buffer_get_buffer(void *data)
Fix access to ifru_buffer on freebsd32. Make all kernel accesses to ifru_buffer go via access functions which take the process ABI into account and use an appropriate union to access members in the correct place in struct ifreq. Reviewed by: kib Obtained from: CheriBSD MFC after: 1 week Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D14846
2018-03-27 18:26:50 +00:00
{
union ifreq_union *ifrup;
ifrup = data;
#ifdef COMPAT_FREEBSD32
Remove the thread argument from ifr_buffer_*() accessors. They are always used in a context where curthread is the correct thread. This makes them more similar to the ifr_data_get_ptr() accessor.
2018-04-06 23:25:54 +00:00
if (SV_CURPROC_FLAG(SV_ILP32))
Fix access to ifru_buffer on freebsd32. Make all kernel accesses to ifru_buffer go via access functions which take the process ABI into account and use an appropriate union to access members in the correct place in struct ifreq. Reviewed by: kib Obtained from: CheriBSD MFC after: 1 week Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D14846
2018-03-27 18:26:50 +00:00
return ((void *)(uintptr_t)
ifrup->ifr32.ifr_ifru.ifru_buffer.buffer);
#endif
return (ifrup->ifr.ifr_ifru.ifru_buffer.buffer);
}
static void
Remove the thread argument from ifr_buffer_*() accessors. They are always used in a context where curthread is the correct thread. This makes them more similar to the ifr_data_get_ptr() accessor.
2018-04-06 23:25:54 +00:00
ifr_buffer_set_buffer_null(void *data)
Fix access to ifru_buffer on freebsd32. Make all kernel accesses to ifru_buffer go via access functions which take the process ABI into account and use an appropriate union to access members in the correct place in struct ifreq. Reviewed by: kib Obtained from: CheriBSD MFC after: 1 week Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D14846
2018-03-27 18:26:50 +00:00
{
union ifreq_union *ifrup;
ifrup = data;
#ifdef COMPAT_FREEBSD32
Remove the thread argument from ifr_buffer_*() accessors. They are always used in a context where curthread is the correct thread. This makes them more similar to the ifr_data_get_ptr() accessor.
2018-04-06 23:25:54 +00:00
if (SV_CURPROC_FLAG(SV_ILP32))
Fix access to ifru_buffer on freebsd32. Make all kernel accesses to ifru_buffer go via access functions which take the process ABI into account and use an appropriate union to access members in the correct place in struct ifreq. Reviewed by: kib Obtained from: CheriBSD MFC after: 1 week Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D14846
2018-03-27 18:26:50 +00:00
ifrup->ifr32.ifr_ifru.ifru_buffer.buffer = 0;
else
#endif
ifrup->ifr.ifr_ifru.ifru_buffer.buffer = NULL;
}
static size_t
Remove the thread argument from ifr_buffer_*() accessors. They are always used in a context where curthread is the correct thread. This makes them more similar to the ifr_data_get_ptr() accessor.
2018-04-06 23:25:54 +00:00
ifr_buffer_get_length(void *data)
Fix access to ifru_buffer on freebsd32. Make all kernel accesses to ifru_buffer go via access functions which take the process ABI into account and use an appropriate union to access members in the correct place in struct ifreq. Reviewed by: kib Obtained from: CheriBSD MFC after: 1 week Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D14846
2018-03-27 18:26:50 +00:00
{
union ifreq_union *ifrup;
ifrup = data;
#ifdef COMPAT_FREEBSD32
Remove the thread argument from ifr_buffer_*() accessors. They are always used in a context where curthread is the correct thread. This makes them more similar to the ifr_data_get_ptr() accessor.
2018-04-06 23:25:54 +00:00
if (SV_CURPROC_FLAG(SV_ILP32))
Fix access to ifru_buffer on freebsd32. Make all kernel accesses to ifru_buffer go via access functions which take the process ABI into account and use an appropriate union to access members in the correct place in struct ifreq. Reviewed by: kib Obtained from: CheriBSD MFC after: 1 week Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D14846
2018-03-27 18:26:50 +00:00
return (ifrup->ifr32.ifr_ifru.ifru_buffer.length);
#endif
return (ifrup->ifr.ifr_ifru.ifru_buffer.length);
}
static void
Remove the thread argument from ifr_buffer_*() accessors. They are always used in a context where curthread is the correct thread. This makes them more similar to the ifr_data_get_ptr() accessor.
2018-04-06 23:25:54 +00:00
ifr_buffer_set_length(void *data, size_t len)
Fix access to ifru_buffer on freebsd32. Make all kernel accesses to ifru_buffer go via access functions which take the process ABI into account and use an appropriate union to access members in the correct place in struct ifreq. Reviewed by: kib Obtained from: CheriBSD MFC after: 1 week Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D14846
2018-03-27 18:26:50 +00:00
{
union ifreq_union *ifrup;
ifrup = data;
#ifdef COMPAT_FREEBSD32
Remove the thread argument from ifr_buffer_*() accessors. They are always used in a context where curthread is the correct thread. This makes them more similar to the ifr_data_get_ptr() accessor.
2018-04-06 23:25:54 +00:00
if (SV_CURPROC_FLAG(SV_ILP32))
Fix access to ifru_buffer on freebsd32. Make all kernel accesses to ifru_buffer go via access functions which take the process ABI into account and use an appropriate union to access members in the correct place in struct ifreq. Reviewed by: kib Obtained from: CheriBSD MFC after: 1 week Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D14846
2018-03-27 18:26:50 +00:00
ifrup->ifr32.ifr_ifru.ifru_buffer.length = len;
else
#endif
ifrup->ifr.ifr_ifru.ifru_buffer.length = len;
}
Use an accessor function to access ifr_data. This fixes 32-bit compat (no ioctl command defintions are required as struct ifreq is the same size). This is believed to be sufficent to fully support ifconfig on 32-bit systems. Reviewed by: kib Obtained from: CheriBSD MFC after: 1 week Relnotes: yes Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D14900
2018-03-30 18:50:13 +00:00
void *
ifr_data_get_ptr(void *ifrp)
{
union ifreq_union *ifrup;
ifrup = ifrp;
#ifdef COMPAT_FREEBSD32
if (SV_CURPROC_FLAG(SV_ILP32))
return ((void *)(uintptr_t)
ifrup->ifr32.ifr_ifru.ifru_data);
#endif
return (ifrup->ifr.ifr_ifru.ifru_data);
}
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
/*
Introduce network device nodes. Network devices will now automatically appear in /dev. Interface hardware ioctls (not protocol or routing) can be performed on the descriptor. The SIOCGIFCONF ioctl may be performed on the special /dev/network node.
2001-09-29 05:55:04 +00:00
* Hardware specific interface ioctls.
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
*/
For changing the MTU on tun/tap devices, it should not matter whether it is done via using ifconfig, which uses a SIOCSIFMTU ioctl() command, or doing it using a TUNSIFINFO/TAPSIFINFO ioctl() command. Without this patch, for IPv6 the new MTU is not used when creating routes. Especially, when initiating TCP connections after increasing the MTU, the old MTU is still used to compute the MSS. Thanks to ae@ and bz@ for helping to improve the patch. Reviewed by: ae@, bz@ Approved by: re (kib@) MFC after: 1 week Sponsored by: Netflix, Inc. Differential Revision: https://reviews.freebsd.org/D17180
2018-09-29 13:01:23 +00:00
int
Introduce network device nodes. Network devices will now automatically appear in /dev. Interface hardware ioctls (not protocol or routing) can be performed on the descriptor. The SIOCGIFCONF ioctl may be performed on the special /dev/network node.
2001-09-29 05:55:04 +00:00
ifhwioctl(u_long cmd, struct ifnet *ifp, caddr_t data, struct thread *td)
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
{
Introduce network device nodes. Network devices will now automatically appear in /dev. Interface hardware ioctls (not protocol or routing) can be performed on the descriptor. The SIOCGIFCONF ioctl may be performed on the special /dev/network node.
2001-09-29 05:55:04 +00:00
struct ifreq *ifr;
if: Defer the if_up until the ifnet.if_ioctl is called. This ensures the interface is initialized by the interface driver before it can be used by the rest of the system. Reviewed by: jhb, karels, gnn MFC after: 3 week Sponsored by: Microsoft Differential Revision: https://reviews.freebsd.org/D8905
2017-01-06 05:10:49 +00:00
int error = 0, do_ifup = 0;
Rename IFF_RUNNING to IFF_DRV_RUNNING, IFF_OACTIVE to IFF_DRV_OACTIVE, and move both flags from ifnet.if_flags to ifnet.if_drv_flags, making and documenting the locking of these flags the responsibility of the device driver, not the network stack. The flags for these two fields will be mutually exclusive so that they can be exposed to user space as though they were stored in the same variable. Provide #defines to provide the old names #ifndef _KERNEL, so that user applications (such as ifconfig) can use the old flag names. Using the old names in a device driver will result in a compile error in order to help device driver writers adopt the new model. When exposing the interface flags to user space, via interface ioctls or routing sockets, or the two fields together. Since the driver flags cannot currently be set for user space, no new logic is currently required to handle this case. Add some assertions that general purpose network stack routines, such as if_setflags(), are not improperly used on driver-owned flags. With this change, a large number of very minor network stack races are closed, subject to correct device driver locking. Most were likely never triggered. Driver sweep to follow; many thanks to pjd and bz for the line-by-line review they gave this patch. Reviewed by: pjd, bz MFC after: 7 days
2005-08-09 10:16:17 +00:00
int new_flags, temp_flags;
Add the kernel side of network interface renaming support. The basic process is to send a routing socket announcement that the interface has departed, change if_xname, update the sockaddr_dl associated with the interface, and announce the arrival of the interface on the routing socket. As part of this change, ifunit() is greatly simplified by testing if_xname directly. if_clone_destroy() now uses if_dname to look up the cloner for the interface and if_dunit to identify the unit number. Reviewed by: ru, sam (concept) Vincent Jardin <vjardin AT free.fr> Max Laier <max AT love2party.net>
2004-02-04 02:54:25 +00:00
size_t namelen, onamelen;
Revised revision 199201 (add interface description capability as inspired by OpenBSD), based on comments from many, including rwatson, jhb, brooks and others. Sponsored by: iXsystems, Inc. MFC after: 1 month
2010-01-27 00:30:07 +00:00
size_t descrlen;
char *descrbuf, *odescrbuf;
Add the kernel side of network interface renaming support. The basic process is to send a routing socket announcement that the interface has departed, change if_xname, update the sockaddr_dl associated with the interface, and announce the arrival of the interface on the routing socket. As part of this change, ifunit() is greatly simplified by testing if_xname directly. if_clone_destroy() now uses if_dname to look up the cloner for the interface and if_dunit to identify the unit number. Reviewed by: ru, sam (concept) Vincent Jardin <vjardin AT free.fr> Max Laier <max AT love2party.net>
2004-02-04 02:54:25 +00:00
char new_name[IFNAMSIZ];
struct ifaddr *ifa;
struct sockaddr_dl *sdl;
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
ifr = (struct ifreq *)data;
switch (cmd) {
Add a SIOCGIFINDEX ioctl, which returns the index of a named interface. This will be used to more efficiently support if_nametoindex(3).
2001-10-17 19:40:44 +00:00
case SIOCGIFINDEX:
ifr->ifr_index = ifp->if_index;
break;
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
case SIOCGIFFLAGS:
Rename IFF_RUNNING to IFF_DRV_RUNNING, IFF_OACTIVE to IFF_DRV_OACTIVE, and move both flags from ifnet.if_flags to ifnet.if_drv_flags, making and documenting the locking of these flags the responsibility of the device driver, not the network stack. The flags for these two fields will be mutually exclusive so that they can be exposed to user space as though they were stored in the same variable. Provide #defines to provide the old names #ifndef _KERNEL, so that user applications (such as ifconfig) can use the old flag names. Using the old names in a device driver will result in a compile error in order to help device driver writers adopt the new model. When exposing the interface flags to user space, via interface ioctls or routing sockets, or the two fields together. Since the driver flags cannot currently be set for user space, no new logic is currently required to handle this case. Add some assertions that general purpose network stack routines, such as if_setflags(), are not improperly used on driver-owned flags. With this change, a large number of very minor network stack races are closed, subject to correct device driver locking. Most were likely never triggered. Driver sweep to follow; many thanks to pjd and bz for the line-by-line review they gave this patch. Reviewed by: pjd, bz MFC after: 7 days
2005-08-09 10:16:17 +00:00
temp_flags = ifp->if_flags | ifp->if_drv_flags;
ifr->ifr_flags = temp_flags & 0xffff;
ifr->ifr_flagshigh = temp_flags >> 16;
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
break;
Add two fields to the ifnet structure indicating what extra capabilities a network device has, and which ones are enabled.
2001-09-18 17:41:42 +00:00
case SIOCGIFCAP:
ifr->ifr_reqcap = ifp->if_capabilities;
ifr->ifr_curcap = ifp->if_capenable;
break;
Introduce support for Mandatory Access Control and extensible kernel access control. Introduce two ioctls, SIOCGIFMAC, SIOCSIFMAC, which permit user processes to manage the MAC labels on network interfaces. Note that this is part of the user process API/ABI that will be revised prior to 5.0-RELEASE. Obtained from: TrustedBSD Project Sponsored by: DARPA, NAI Labs
2002-08-01 21:15:53 +00:00
#ifdef MAC
case SIOCGIFMAC:
Merge first in a series of TrustedBSD MAC Framework KPI changes from Mac OS X Leopard--rationalize naming for entry points to the following general forms: mac_<object>_<method/action> mac_<object>_check_<method/action> The previous naming scheme was inconsistent and mostly reversed from the new scheme. Also, make object types more consistent and remove spaces from object types that contain multiple parts ("posix_sem" -> "posixsem") to make mechanical parsing easier. Introduce a new "netinet" object type for certain IPv4/IPv6-related methods. Also simplify, slightly, some entry point names. All MAC policy modules will need to be recompiled, and modules not updates as part of this commit will need to be modified to conform to the new KPI. Sponsored by: SPARTA (original patches against Mac OS X) Obtained from: TrustedBSD Project, Apple Computer
2007-10-24 19:04:04 +00:00
error = mac_ifnet_ioctl_get(td->td_ucred, ifr, ifp);
Introduce support for Mandatory Access Control and extensible kernel access control. Introduce two ioctls, SIOCGIFMAC, SIOCSIFMAC, which permit user processes to manage the MAC labels on network interfaces. Note that this is part of the user process API/ABI that will be revised prior to 5.0-RELEASE. Obtained from: TrustedBSD Project Sponsored by: DARPA, NAI Labs
2002-08-01 21:15:53 +00:00
break;
#endif
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
case SIOCGIFMETRIC:
ifr->ifr_metric = ifp->if_metric;
break;
Added ioctl support for SIOCGIFMTU and SIOCSIFMTU. These set the per- interface MTU.
1994-08-08 10:49:26 +00:00
case SIOCGIFMTU:
ifr->ifr_mtu = ifp->if_mtu;
break;
Add generic part of generic multiple-physical-interface support (the successor of IFF_ALTPHYS).
1994-12-21 22:57:05 +00:00
case SIOCGIFPHYS:
o Remove struct if_data from struct ifnet. Now it is merely API structure for route(4) socket and ifmib(4) sysctl. o Move fields from if_data to ifnet, but keep all statistic counters separate, since they should disappear later. o Provide function if_data_copy() to fill if_data, utilize it in routing socket and ifmib handler. o Provide overridable ifnet(9) method to fetch counters. If no provided, if_get_counters_compat() would be used, that returns old counters. Sponsored by: Netflix Sponsored by: Nginx, Inc.
2014-08-31 06:46:21 +00:00
/* XXXGL: did this ever worked? */
ifr->ifr_phys = 0;
Add generic part of generic multiple-physical-interface support (the successor of IFF_ALTPHYS).
1994-12-21 22:57:05 +00:00
break;
Revised revision 199201 (add interface description capability as inspired by OpenBSD), based on comments from many, including rwatson, jhb, brooks and others. Sponsored by: iXsystems, Inc. MFC after: 1 month
2010-01-27 00:30:07 +00:00
case SIOCGIFDESCR:
error = 0;
sx_slock(&ifdescr_sx);
When an underlying ioctl(2) handler returns an error, our ioctl(2) interface considers that it hits a fatal error, and will not copyout the request structure back for _IOW and _IOWR ioctls, keeping them untouched. The previous implementation of the SIOCGIFDESCR ioctl intends to feed the buffer length back to userland. However, if we return an error, the feedback would be defeated and ifconfig(8) would trap into an infinite loop. This commit changes SIOCGIFDESCR to set buffer field to NULL to indicate the previous ENAMETOOLONG case. Reported by: bschmidt MFC after: 2 weeks
2010-04-14 22:02:19 +00:00
if (ifp->if_description == NULL)
Revised revision 199201 (add interface description capability as inspired by OpenBSD), based on comments from many, including rwatson, jhb, brooks and others. Sponsored by: iXsystems, Inc. MFC after: 1 month
2010-01-27 00:30:07 +00:00
error = ENOMSG;
When an underlying ioctl(2) handler returns an error, our ioctl(2) interface considers that it hits a fatal error, and will not copyout the request structure back for _IOW and _IOWR ioctls, keeping them untouched. The previous implementation of the SIOCGIFDESCR ioctl intends to feed the buffer length back to userland. However, if we return an error, the feedback would be defeated and ifconfig(8) would trap into an infinite loop. This commit changes SIOCGIFDESCR to set buffer field to NULL to indicate the previous ENAMETOOLONG case. Reported by: bschmidt MFC after: 2 weeks
2010-04-14 22:02:19 +00:00
else {
Revised revision 199201 (add interface description capability as inspired by OpenBSD), based on comments from many, including rwatson, jhb, brooks and others. Sponsored by: iXsystems, Inc. MFC after: 1 month
2010-01-27 00:30:07 +00:00
/* space for terminating nul */
descrlen = strlen(ifp->if_description) + 1;
Remove the thread argument from ifr_buffer_*() accessors. They are always used in a context where curthread is the correct thread. This makes them more similar to the ifr_data_get_ptr() accessor.
2018-04-06 23:25:54 +00:00
if (ifr_buffer_get_length(ifr) < descrlen)
ifr_buffer_set_buffer_null(ifr);
Revised revision 199201 (add interface description capability as inspired by OpenBSD), based on comments from many, including rwatson, jhb, brooks and others. Sponsored by: iXsystems, Inc. MFC after: 1 month
2010-01-27 00:30:07 +00:00
else
error = copyout(ifp->if_description,
Remove the thread argument from ifr_buffer_*() accessors. They are always used in a context where curthread is the correct thread. This makes them more similar to the ifr_data_get_ptr() accessor.
2018-04-06 23:25:54 +00:00
ifr_buffer_get_buffer(ifr), descrlen);
ifr_buffer_set_length(ifr, descrlen);
Revised revision 199201 (add interface description capability as inspired by OpenBSD), based on comments from many, including rwatson, jhb, brooks and others. Sponsored by: iXsystems, Inc. MFC after: 1 month
2010-01-27 00:30:07 +00:00
}
sx_sunlock(&ifdescr_sx);
break;
case SIOCSIFDESCR:
error = priv_check(td, PRIV_NET_SETIFDESCR);
if (error)
return (error);
/*
* Copy only (length-1) bytes to make sure that
* if_description is always nul terminated. The
* length parameter is supposed to count the
* terminating nul in.
*/
Remove the thread argument from ifr_buffer_*() accessors. They are always used in a context where curthread is the correct thread. This makes them more similar to the ifr_data_get_ptr() accessor.
2018-04-06 23:25:54 +00:00
if (ifr_buffer_get_length(ifr) > ifdescr_maxlen)
Revised revision 199201 (add interface description capability as inspired by OpenBSD), based on comments from many, including rwatson, jhb, brooks and others. Sponsored by: iXsystems, Inc. MFC after: 1 month
2010-01-27 00:30:07 +00:00
return (ENAMETOOLONG);
Remove the thread argument from ifr_buffer_*() accessors. They are always used in a context where curthread is the correct thread. This makes them more similar to the ifr_data_get_ptr() accessor.
2018-04-06 23:25:54 +00:00
else if (ifr_buffer_get_length(ifr) == 0)
Revised revision 199201 (add interface description capability as inspired by OpenBSD), based on comments from many, including rwatson, jhb, brooks and others. Sponsored by: iXsystems, Inc. MFC after: 1 month
2010-01-27 00:30:07 +00:00
descrbuf = NULL;
else {
Remove the thread argument from ifr_buffer_*() accessors. They are always used in a context where curthread is the correct thread. This makes them more similar to the ifr_data_get_ptr() accessor.
2018-04-06 23:25:54 +00:00
descrbuf = malloc(ifr_buffer_get_length(ifr),
Fix access to ifru_buffer on freebsd32. Make all kernel accesses to ifru_buffer go via access functions which take the process ABI into account and use an appropriate union to access members in the correct place in struct ifreq. Reviewed by: kib Obtained from: CheriBSD MFC after: 1 week Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D14846
2018-03-27 18:26:50 +00:00
M_IFDESCR, M_WAITOK | M_ZERO);
Remove the thread argument from ifr_buffer_*() accessors. They are always used in a context where curthread is the correct thread. This makes them more similar to the ifr_data_get_ptr() accessor.
2018-04-06 23:25:54 +00:00
error = copyin(ifr_buffer_get_buffer(ifr), descrbuf,
ifr_buffer_get_length(ifr) - 1);
Revised revision 199201 (add interface description capability as inspired by OpenBSD), based on comments from many, including rwatson, jhb, brooks and others. Sponsored by: iXsystems, Inc. MFC after: 1 month
2010-01-27 00:30:07 +00:00
if (error) {
free(descrbuf, M_IFDESCR);
break;
}
}
sx_xlock(&ifdescr_sx);
odescrbuf = ifp->if_description;
ifp->if_description = descrbuf;
sx_xunlock(&ifdescr_sx);
getmicrotime(&ifp->if_lastchange);
free(odescrbuf, M_IFDESCR);
break;
Add generic part of generic multiple-physical-interface support (the successor of IFF_ALTPHYS).
1994-12-21 22:57:05 +00:00
Add infrastructure to allow all frames/packets received on an interface to be assigned to a non-default FIB instance. You may need to recompile world or ports due to the change of struct ifnet. Submitted by: cjsp Submitted by: Alexander V. Chernikov (melifaro ipfw.ru) (original versions) Reviewed by: julian Reviewed by: Alexander V. Chernikov (melifaro ipfw.ru) MFC after: 2 weeks X-MFC: use spare in struct ifnet
2011-07-03 12:22:02 +00:00
case SIOCGIFFIB:
ifr->ifr_fib = ifp->if_fib;
break;
case SIOCSIFFIB:
error = priv_check(td, PRIV_NET_SETIFFIB);
if (error)
return (error);
if (ifr->ifr_fib >= rt_numfibs)
return (EINVAL);
ifp->if_fib = ifr->ifr_fib;
break;
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
case SIOCSIFFLAGS:
Sweep kernel replacing suser(9) calls with priv(9) calls, assigning specific privilege names to a broad range of privileges. These may require some future tweaking. Sponsored by: nCircle Network Security, Inc. Obtained from: TrustedBSD Project Discussed on: arch@ Reviewed (at least in part) by: mlaier, jmg, pjd, bde, ceri, Alex Lyashkov <umka at sevcity dot net>, Skip Ford <skip dot ford at verizon dot net>, Antoine Brodin <antoine dot brodin at laposte dot net>
2006-11-06 13:42:10 +00:00
error = priv_check(td, PRIV_NET_SETIFFLAGS);
Mostly Cosmetics. Some of the procedures in if_sl.c was void, but should be int. I made them int, and let them return 0. Will have to find out what the return-val is used for.
1994-10-08 01:40:23 +00:00
if (error)
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
return (error);
Rename IFF_RUNNING to IFF_DRV_RUNNING, IFF_OACTIVE to IFF_DRV_OACTIVE, and move both flags from ifnet.if_flags to ifnet.if_drv_flags, making and documenting the locking of these flags the responsibility of the device driver, not the network stack. The flags for these two fields will be mutually exclusive so that they can be exposed to user space as though they were stored in the same variable. Provide #defines to provide the old names #ifndef _KERNEL, so that user applications (such as ifconfig) can use the old flag names. Using the old names in a device driver will result in a compile error in order to help device driver writers adopt the new model. When exposing the interface flags to user space, via interface ioctls or routing sockets, or the two fields together. Since the driver flags cannot currently be set for user space, no new logic is currently required to handle this case. Add some assertions that general purpose network stack routines, such as if_setflags(), are not improperly used on driver-owned flags. With this change, a large number of very minor network stack races are closed, subject to correct device driver locking. Most were likely never triggered. Driver sweep to follow; many thanks to pjd and bz for the line-by-line review they gave this patch. Reviewed by: pjd, bz MFC after: 7 days
2005-08-09 10:16:17 +00:00
/*
* Currently, no driver owned flags pass the IFF_CANTCHANGE
* check, so we don't need special handling here yet.
*/
Increase size of ifnet.if_flags from 16 bits (short) to 32 bits (int). To avoid breaking application ABI use unused ifreq.ifru_flags[1] for upper 16 bits in SIOCSIFFLAGS and SIOCGIFFLAGS ioctl's. Reviewed by: -hackers, -net
2002-08-18 07:05:00 +00:00
new_flags = (ifr->ifr_flags & 0xffff) |
(ifr->ifr_flagshigh << 16);
Axe IFF_SMART. Fortunately this layering violating flag was never used, it was just declared.
2013-11-05 12:52:56 +00:00
if (ifp->if_flags & IFF_UP &&
Increase size of ifnet.if_flags from 16 bits (short) to 32 bits (int). To avoid breaking application ABI use unused ifreq.ifru_flags[1] for upper 16 bits in SIOCSIFFLAGS and SIOCGIFFLAGS ioctl's. Reviewed by: -hackers, -net
2002-08-18 07:05:00 +00:00
(new_flags & IFF_UP) == 0) {
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
if_down(ifp);
Increase size of ifnet.if_flags from 16 bits (short) to 32 bits (int). To avoid breaking application ABI use unused ifreq.ifru_flags[1] for upper 16 bits in SIOCSIFFLAGS and SIOCGIFFLAGS ioctl's. Reviewed by: -hackers, -net
2002-08-18 07:05:00 +00:00
} else if (new_flags & IFF_UP &&
Introduce IFF_SMART bit. This means that the driver will add/delete routes when it knows it is up/down, rather than have the generic code belive it is up if configured. This is probably most useful for serial lines, although many PHY chips could probably tell us if we're connected to the cable/hub as well.
1999-06-06 09:17:51 +00:00
(ifp->if_flags & IFF_UP) == 0) {
if: Defer the if_up until the ifnet.if_ioctl is called. This ensures the interface is initialized by the interface driver before it can be used by the rest of the system. Reviewed by: jhb, karels, gnn MFC after: 3 week Sponsored by: Microsoft Differential Revision: https://reviews.freebsd.org/D8905
2017-01-06 05:10:49 +00:00
do_ifup = 1;
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
}
Log a message about entering or leaving permanently promiscuous mode, as it is done for usual promiscuous mode already. This info is important because promiscuous mode in the hands of a malicious party can jeopardize the whole network.
2005-10-03 01:47:43 +00:00
/* See if permanently promiscuous mode bit is about to flip */
if ((ifp->if_flags ^ new_flags) & IFF_PPROMISC) {
if (new_flags & IFF_PPROMISC)
ifp->if_flags |= IFF_PROMISC;
else if (ifp->if_pcount == 0)
ifp->if_flags &= ~IFF_PROMISC;
Allow silencing of 'promiscuous mode enabled/disabled' messages. PR: 166255 Submitted by: eugen.grosbein.net Obtained from: eugen.grosbein.net MFC after: 1 week
2016-05-12 19:42:13 +00:00
if (log_promisc_mode_change)
Slight cleanup of interface event logging. Make if_printf() use vlog() instead of vprintf(). This means it can no longer return the number of characters printed, as it used to, but every single call to if_printf() in the entire kernel ignores the return value anyway; just return 0 so we don't have to change the prototype. Consistently use if_printf() throughout sys/net/if.c, instead of a mixture of if_printf() and log(). In ifa_maintain_loopback_route(), don't needlessly log an error if we either failed to add a route because it already existed or failed to remove one because it did not. We still return an error code, though. MFC after: 1 week
2018-05-11 00:19:49 +00:00
if_printf(ifp, "permanently promiscuous mode %s\n",
Allow silencing of 'promiscuous mode enabled/disabled' messages. PR: 166255 Submitted by: eugen.grosbein.net Obtained from: eugen.grosbein.net MFC after: 1 week
2016-05-12 19:42:13 +00:00
((new_flags & IFF_PPROMISC) ?
"enabled" : "disabled"));
Log a message about entering or leaving permanently promiscuous mode, as it is done for usual promiscuous mode already. This info is important because promiscuous mode in the hands of a malicious party can jeopardize the whole network.
2005-10-03 01:47:43 +00:00
}
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
ifp->if_flags = (ifp->if_flags & IFF_CANTCHANGE) |
Increase size of ifnet.if_flags from 16 bits (short) to 32 bits (int). To avoid breaking application ABI use unused ifreq.ifru_flags[1] for upper 16 bits in SIOCSIFFLAGS and SIOCGIFFLAGS ioctl's. Reviewed by: -hackers, -net
2002-08-18 07:05:00 +00:00
(new_flags &~ IFF_CANTCHANGE);
Define IFF_LOCKGIANT() and IFF_UNLOCKGIANT() macros, which conditionally acquire Giant if the passed interface has IFF_NEEDSGIANT set on it. Modify calls into (ifp)->if_ioctl() in if.c to use these macros in order to ensure that Giant is held. MFC after: 3 days Bumped into by: jmg
2004-10-19 18:11:55 +00:00
if (ifp->if_ioctl) {
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
(void) (*ifp->if_ioctl)(ifp, cmd, data);
Define IFF_LOCKGIANT() and IFF_UNLOCKGIANT() macros, which conditionally acquire Giant if the passed interface has IFF_NEEDSGIANT set on it. Modify calls into (ifp)->if_ioctl() in if.c to use these macros in order to ensure that Giant is held. MFC after: 3 days Bumped into by: jmg
2004-10-19 18:11:55 +00:00
}
if: Defer the if_up until the ifnet.if_ioctl is called. This ensures the interface is initialized by the interface driver before it can be used by the rest of the system. Reviewed by: jhb, karels, gnn MFC after: 3 week Sponsored by: Microsoft Differential Revision: https://reviews.freebsd.org/D8905
2017-01-06 05:10:49 +00:00
if (do_ifup)
if_up(ifp);
Use getmicrotime() for if_lastchange, 10msec is plenty precision.
1998-04-06 11:43:12 +00:00
getmicrotime(&ifp->if_lastchange);
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
break;
Add two fields to the ifnet structure indicating what extra capabilities a network device has, and which ones are enabled.
2001-09-18 17:41:42 +00:00
case SIOCSIFCAP:
Sweep kernel replacing suser(9) calls with priv(9) calls, assigning specific privilege names to a broad range of privileges. These may require some future tweaking. Sponsored by: nCircle Network Security, Inc. Obtained from: TrustedBSD Project Discussed on: arch@ Reviewed (at least in part) by: mlaier, jmg, pjd, bde, ceri, Alex Lyashkov <umka at sevcity dot net>, Skip Ford <skip dot ford at verizon dot net>, Antoine Brodin <antoine dot brodin at laposte dot net>
2006-11-06 13:42:10 +00:00
error = priv_check(td, PRIV_NET_SETIFCAP);
Add two fields to the ifnet structure indicating what extra capabilities a network device has, and which ones are enabled.
2001-09-18 17:41:42 +00:00
if (error)
return (error);
Improve the SIOCSIFCAP handler a bit: - allow for ifp->if_ioctl being NULL, as the rest of ifioctl() does; - give the interface driver a chance to report a error to the caller; - don't forget to update ifp->if_lastchange upon successful modification of interface operation parameters.
2004-02-21 12:48:25 +00:00
if (ifp->if_ioctl == NULL)
return (EOPNOTSUPP);
Add two fields to the ifnet structure indicating what extra capabilities a network device has, and which ones are enabled.
2001-09-18 17:41:42 +00:00
if (ifr->ifr_reqcap & ~ifp->if_capabilities)
return (EINVAL);
Improve the SIOCSIFCAP handler a bit: - allow for ifp->if_ioctl being NULL, as the rest of ifioctl() does; - give the interface driver a chance to report a error to the caller; - don't forget to update ifp->if_lastchange upon successful modification of interface operation parameters.
2004-02-21 12:48:25 +00:00
error = (*ifp->if_ioctl)(ifp, cmd, data);
if (error == 0)
getmicrotime(&ifp->if_lastchange);
Add two fields to the ifnet structure indicating what extra capabilities a network device has, and which ones are enabled.
2001-09-18 17:41:42 +00:00
break;
Introduce support for Mandatory Access Control and extensible kernel access control. Introduce two ioctls, SIOCGIFMAC, SIOCSIFMAC, which permit user processes to manage the MAC labels on network interfaces. Note that this is part of the user process API/ABI that will be revised prior to 5.0-RELEASE. Obtained from: TrustedBSD Project Sponsored by: DARPA, NAI Labs
2002-08-01 21:15:53 +00:00
#ifdef MAC
case SIOCSIFMAC:
Merge first in a series of TrustedBSD MAC Framework KPI changes from Mac OS X Leopard--rationalize naming for entry points to the following general forms: mac_<object>_<method/action> mac_<object>_check_<method/action> The previous naming scheme was inconsistent and mostly reversed from the new scheme. Also, make object types more consistent and remove spaces from object types that contain multiple parts ("posix_sem" -> "posixsem") to make mechanical parsing easier. Introduce a new "netinet" object type for certain IPv4/IPv6-related methods. Also simplify, slightly, some entry point names. All MAC policy modules will need to be recompiled, and modules not updates as part of this commit will need to be modified to conform to the new KPI. Sponsored by: SPARTA (original patches against Mac OS X) Obtained from: TrustedBSD Project, Apple Computer
2007-10-24 19:04:04 +00:00
error = mac_ifnet_ioctl_set(td->td_ucred, ifr, ifp);
Introduce support for Mandatory Access Control and extensible kernel access control. Introduce two ioctls, SIOCGIFMAC, SIOCSIFMAC, which permit user processes to manage the MAC labels on network interfaces. Note that this is part of the user process API/ABI that will be revised prior to 5.0-RELEASE. Obtained from: TrustedBSD Project Sponsored by: DARPA, NAI Labs
2002-08-01 21:15:53 +00:00
break;
#endif
Add the kernel side of network interface renaming support. The basic process is to send a routing socket announcement that the interface has departed, change if_xname, update the sockaddr_dl associated with the interface, and announce the arrival of the interface on the routing socket. As part of this change, ifunit() is greatly simplified by testing if_xname directly. if_clone_destroy() now uses if_dname to look up the cloner for the interface and if_dunit to identify the unit number. Reviewed by: ru, sam (concept) Vincent Jardin <vjardin AT free.fr> Max Laier <max AT love2party.net>
2004-02-04 02:54:25 +00:00
case SIOCSIFNAME:
Sweep kernel replacing suser(9) calls with priv(9) calls, assigning specific privilege names to a broad range of privileges. These may require some future tweaking. Sponsored by: nCircle Network Security, Inc. Obtained from: TrustedBSD Project Discussed on: arch@ Reviewed (at least in part) by: mlaier, jmg, pjd, bde, ceri, Alex Lyashkov <umka at sevcity dot net>, Skip Ford <skip dot ford at verizon dot net>, Antoine Brodin <antoine dot brodin at laposte dot net>
2006-11-06 13:42:10 +00:00
error = priv_check(td, PRIV_NET_SETIFNAME);
if (error)
Add the kernel side of network interface renaming support. The basic process is to send a routing socket announcement that the interface has departed, change if_xname, update the sockaddr_dl associated with the interface, and announce the arrival of the interface on the routing socket. As part of this change, ifunit() is greatly simplified by testing if_xname directly. if_clone_destroy() now uses if_dname to look up the cloner for the interface and if_dunit to identify the unit number. Reviewed by: ru, sam (concept) Vincent Jardin <vjardin AT free.fr> Max Laier <max AT love2party.net>
2004-02-04 02:54:25 +00:00
return (error);
Use an accessor function to access ifr_data. This fixes 32-bit compat (no ioctl command defintions are required as struct ifreq is the same size). This is believed to be sufficent to fully support ifconfig on 32-bit systems. Reviewed by: kib Obtained from: CheriBSD MFC after: 1 week Relnotes: yes Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D14900
2018-03-30 18:50:13 +00:00
error = copyinstr(ifr_data_get_ptr(ifr), new_name, IFNAMSIZ,
NULL);
Don't allow interfaces to be renamed to the empty string. While I'm here, errors aren't bools. Pointed out by: hmp
2004-03-13 02:35:03 +00:00
if (error != 0)
Add the kernel side of network interface renaming support. The basic process is to send a routing socket announcement that the interface has departed, change if_xname, update the sockaddr_dl associated with the interface, and announce the arrival of the interface on the routing socket. As part of this change, ifunit() is greatly simplified by testing if_xname directly. if_clone_destroy() now uses if_dname to look up the cloner for the interface and if_dunit to identify the unit number. Reviewed by: ru, sam (concept) Vincent Jardin <vjardin AT free.fr> Max Laier <max AT love2party.net>
2004-02-04 02:54:25 +00:00
return (error);
Don't allow interfaces to be renamed to the empty string. While I'm here, errors aren't bools. Pointed out by: hmp
2004-03-13 02:35:03 +00:00
if (new_name[0] == '\0')
return (EINVAL);
When handling SIOCSIFNAME ensure that the new interface name is NUL terminated. Reject the rename attempt if the name is too long. MFC after: 1 week
2016-05-15 21:37:36 +00:00
if (new_name[IFNAMSIZ-1] != '\0') {
new_name[IFNAMSIZ-1] = '\0';
if (strlen(new_name) == IFNAMSIZ-1)
return (EINVAL);
}
SIOCSIFNAME: Do nothing if we're not actually changing Instead of throwing EEXIST, just succeed if the name isn't actually changing. We don't need to trigger departure or any of that because there's no change from consumers' perspective. PR: 240539 Reviewed by: brooks MFC after: 5 days Differential Revision: https://reviews.freebsd.org/D21618
2019-09-12 15:36:48 +00:00
if (strcmp(new_name, ifp->if_xname) == 0)
break;
Add the kernel side of network interface renaming support. The basic process is to send a routing socket announcement that the interface has departed, change if_xname, update the sockaddr_dl associated with the interface, and announce the arrival of the interface on the routing socket. As part of this change, ifunit() is greatly simplified by testing if_xname directly. if_clone_destroy() now uses if_dname to look up the cloner for the interface and if_dunit to identify the unit number. Reviewed by: ru, sam (concept) Vincent Jardin <vjardin AT free.fr> Max Laier <max AT love2party.net>
2004-02-04 02:54:25 +00:00
if (ifunit(new_name) != NULL)
return (EEXIST);
Change vlan interfaces to cope more usefully with the parent interface being renamed. Previously the vlan interfaces would lose their configuration as if the parent interface had been physically removed. Now vlan interfaces ignore rename events. - Add a new ifnet flag (IFF_RENAMING) that is set while an ifnet is being renamed. This flag can be checked in ifnet departure/arrival event handlers to treat rename events differently. - Change the ifnet departure event handler in the if_vlan(4) driver to ignore departure events due to a trunk interface being renamed. Reviewed by: brooks, rwatson MFC after: 1 week
2009-12-29 13:35:18 +00:00
/*
* XXX: Locking. Nothing else seems to lock if_flags,
* and there are numerous other races with the
* ifunit() checks not being atomic with namespace
* changes (renames, vmoves, if_attach, etc).
*/
ifp->if_flags |= IFF_RENAMING;
Add the kernel side of network interface renaming support. The basic process is to send a routing socket announcement that the interface has departed, change if_xname, update the sockaddr_dl associated with the interface, and announce the arrival of the interface on the routing socket. As part of this change, ifunit() is greatly simplified by testing if_xname directly. if_clone_destroy() now uses if_dname to look up the cloner for the interface and if_dunit to identify the unit number. Reviewed by: ru, sam (concept) Vincent Jardin <vjardin AT free.fr> Max Laier <max AT love2party.net>
2004-02-04 02:54:25 +00:00
/* Announce the departure of the interface. */
rt_ifannouncemsg(ifp, IFAN_DEPARTURE);
Move eventhandler for 'ifnet_departure_event' at the end of the progress. Some of the (IPv6) cleanup functions send packets to inform peers of the departure. These packets confused users of ifnet_departure_event (pf at the moment). PR: kern/80627 Tested by: Divacky Roman MFC after: 1 week
2005-07-14 20:26:43 +00:00
EVENTHANDLER_INVOKE(ifnet_departure_event, ifp);
Add the kernel side of network interface renaming support. The basic process is to send a routing socket announcement that the interface has departed, change if_xname, update the sockaddr_dl associated with the interface, and announce the arrival of the interface on the routing socket. As part of this change, ifunit() is greatly simplified by testing if_xname directly. if_clone_destroy() now uses if_dname to look up the cloner for the interface and if_dunit to identify the unit number. Reviewed by: ru, sam (concept) Vincent Jardin <vjardin AT free.fr> Max Laier <max AT love2party.net>
2004-02-04 02:54:25 +00:00
Slight cleanup of interface event logging. Make if_printf() use vlog() instead of vprintf(). This means it can no longer return the number of characters printed, as it used to, but every single call to if_printf() in the entire kernel ignores the return value anyway; just return 0 so we don't have to change the prototype. Consistently use if_printf() throughout sys/net/if.c, instead of a mixture of if_printf() and log(). In ifa_maintain_loopback_route(), don't needlessly log an error if we either failed to add a route because it already existed or failed to remove one because it did not. We still return an error code, though. MFC after: 1 week
2018-05-11 00:19:49 +00:00
if_printf(ifp, "changing name to '%s'\n", new_name);
Log the renaming of an interface. This should make it easier to follow kernel log files.
2004-09-18 05:02:08 +00:00
Remove ifa_mtx. It was used only in one place in kernel, and ifnet's ifaddr lock can substitute it there. Discussed with: melifaro, ae Sponsored by: Netflix Sponsored by: Nginx, Inc.
2013-10-15 10:41:22 +00:00
IF_ADDR_WLOCK(ifp);
Add the kernel side of network interface renaming support. The basic process is to send a routing socket announcement that the interface has departed, change if_xname, update the sockaddr_dl associated with the interface, and announce the arrival of the interface on the routing socket. As part of this change, ifunit() is greatly simplified by testing if_xname directly. if_clone_destroy() now uses if_dname to look up the cloner for the interface and if_dunit to identify the unit number. Reviewed by: ru, sam (concept) Vincent Jardin <vjardin AT free.fr> Max Laier <max AT love2party.net>
2004-02-04 02:54:25 +00:00
strlcpy(ifp->if_xname, new_name, sizeof(ifp->if_xname));
- Store pointer to the link-level address right in "struct ifnet" rather than in ifindex_table[]; all (except one) accesses are through ifp anyway. IF_LLADDR() works faster, and all (except one) ifaddr_byindex() users were converted to use ifp->if_addr. - Stop storing a (pointer to) Ethernet address in "struct arpcom", and drop the IFP2ENADDR() macro; all users have been converted to use IF_LLADDR() instead.
2005-11-11 16:04:59 +00:00
ifa = ifp->if_addr;
Add the kernel side of network interface renaming support. The basic process is to send a routing socket announcement that the interface has departed, change if_xname, update the sockaddr_dl associated with the interface, and announce the arrival of the interface on the routing socket. As part of this change, ifunit() is greatly simplified by testing if_xname directly. if_clone_destroy() now uses if_dname to look up the cloner for the interface and if_dunit to identify the unit number. Reviewed by: ru, sam (concept) Vincent Jardin <vjardin AT free.fr> Max Laier <max AT love2party.net>
2004-02-04 02:54:25 +00:00
sdl = (struct sockaddr_dl *)ifa->ifa_addr;
namelen = strlen(new_name);
onamelen = sdl->sdl_nlen;
/*
* Move the address if needed. This is safe because we
* allocate space for a name of length IFNAMSIZ when we
* create this in if_attach().
*/
if (namelen != onamelen) {
bcopy(sdl->sdl_data + onamelen,
sdl->sdl_data + namelen, sdl->sdl_alen);
}
bcopy(new_name, sdl->sdl_data, namelen);
sdl->sdl_nlen = namelen;
sdl = (struct sockaddr_dl *)ifa->ifa_netmask;
bzero(sdl->sdl_data, onamelen);
while (namelen != 0)
sdl->sdl_data[--namelen] = 0xff;
Remove ifa_mtx. It was used only in one place in kernel, and ifnet's ifaddr lock can substitute it there. Discussed with: melifaro, ae Sponsored by: Netflix Sponsored by: Nginx, Inc.
2013-10-15 10:41:22 +00:00
IF_ADDR_WUNLOCK(ifp);
Add the kernel side of network interface renaming support. The basic process is to send a routing socket announcement that the interface has departed, change if_xname, update the sockaddr_dl associated with the interface, and announce the arrival of the interface on the routing socket. As part of this change, ifunit() is greatly simplified by testing if_xname directly. if_clone_destroy() now uses if_dname to look up the cloner for the interface and if_dunit to identify the unit number. Reviewed by: ru, sam (concept) Vincent Jardin <vjardin AT free.fr> Max Laier <max AT love2party.net>
2004-02-04 02:54:25 +00:00
Bring eventhandler callbacks for pf. This enables pf to track dynamic address changes on interfaces (dailup) with the "on (<ifname>)"-syntax. This also brings hooks in anticipation of tracking cloned interfaces, which will be in future versions of pf. Approved by: bms(mentor)
2004-02-26 04:27:55 +00:00
EVENTHANDLER_INVOKE(ifnet_arrival_event, ifp);
Add the kernel side of network interface renaming support. The basic process is to send a routing socket announcement that the interface has departed, change if_xname, update the sockaddr_dl associated with the interface, and announce the arrival of the interface on the routing socket. As part of this change, ifunit() is greatly simplified by testing if_xname directly. if_clone_destroy() now uses if_dname to look up the cloner for the interface and if_dunit to identify the unit number. Reviewed by: ru, sam (concept) Vincent Jardin <vjardin AT free.fr> Max Laier <max AT love2party.net>
2004-02-04 02:54:25 +00:00
/* Announce the return of the interface. */
rt_ifannouncemsg(ifp, IFAN_ARRIVAL);
Change vlan interfaces to cope more usefully with the parent interface being renamed. Previously the vlan interfaces would lose their configuration as if the parent interface had been physically removed. Now vlan interfaces ignore rename events. - Add a new ifnet flag (IFF_RENAMING) that is set while an ifnet is being renamed. This flag can be checked in ifnet departure/arrival event handlers to treat rename events differently. - Change the ifnet departure event handler in the if_vlan(4) driver to ignore departure events due to a trunk interface being renamed. Reviewed by: brooks, rwatson MFC after: 1 week
2009-12-29 13:35:18 +00:00
ifp->if_flags &= ~IFF_RENAMING;
Add the kernel side of network interface renaming support. The basic process is to send a routing socket announcement that the interface has departed, change if_xname, update the sockaddr_dl associated with the interface, and announce the arrival of the interface on the routing socket. As part of this change, ifunit() is greatly simplified by testing if_xname directly. if_clone_destroy() now uses if_dname to look up the cloner for the interface and if_dunit to identify the unit number. Reviewed by: ru, sam (concept) Vincent Jardin <vjardin AT free.fr> Max Laier <max AT love2party.net>
2004-02-04 02:54:25 +00:00
break;
Manage vnets via the jail system. If a jail is given the boolean parameter "vnet" when it is created, a new vnet instance will be created along with the jail. Networks interfaces can be moved between prisons with an ioctl similar to the one that moves them between vimages. For now vnets will co-exist under both jails and vimages, but soon struct vimage will be going away. Reviewed by: zec, julian Approved by: bz (mentor)
2009-06-15 18:59:29 +00:00
#ifdef VIMAGE
case SIOCSIFVNET:
error = priv_check(td, PRIV_NET_SETIFVNET);
if (error)
return (error);
Make the in-kernel logic for the SIOCSIFVNET, SIOCSIFRVNET ioctls (ifconfig ifN (-)vnet <jname|jid>) work correctly. Move vi_if_move to if.c and split it up into two functions(*), one for each ioctl. In the reclaim case, correctly set the vnet before calling if_vmove. Instead of silently allowing a move of an interface from the current vnet to the current vnet, return an error. (*) There is some duplicate interface name checking before actually moving the interface between network stacks without locking and thus race prone. Ideally if_vmove will correctly and automagically handle these in the future. Suggested by: rwatson (*) Approved by: re (kib)
2009-07-26 11:29:26 +00:00
error = if_vmove_loan(td, ifp, ifr->ifr_name, ifr->ifr_jid);
Manage vnets via the jail system. If a jail is given the boolean parameter "vnet" when it is created, a new vnet instance will be created along with the jail. Networks interfaces can be moved between prisons with an ioctl similar to the one that moves them between vimages. For now vnets will co-exist under both jails and vimages, but soon struct vimage will be going away. Reviewed by: zec, julian Approved by: bz (mentor)
2009-06-15 18:59:29 +00:00
break;
#endif
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
case SIOCSIFMETRIC:
Sweep kernel replacing suser(9) calls with priv(9) calls, assigning specific privilege names to a broad range of privileges. These may require some future tweaking. Sponsored by: nCircle Network Security, Inc. Obtained from: TrustedBSD Project Discussed on: arch@ Reviewed (at least in part) by: mlaier, jmg, pjd, bde, ceri, Alex Lyashkov <umka at sevcity dot net>, Skip Ford <skip dot ford at verizon dot net>, Antoine Brodin <antoine dot brodin at laposte dot net>
2006-11-06 13:42:10 +00:00
error = priv_check(td, PRIV_NET_SETIFMETRIC);
Mostly Cosmetics. Some of the procedures in if_sl.c was void, but should be int. I made them int, and let them return 0. Will have to find out what the return-val is used for.
1994-10-08 01:40:23 +00:00
if (error)
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
return (error);
ifp->if_metric = ifr->ifr_metric;
Use getmicrotime() for if_lastchange, 10msec is plenty precision.
1998-04-06 11:43:12 +00:00
getmicrotime(&ifp->if_lastchange);
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
break;
Add generic part of generic multiple-physical-interface support (the successor of IFF_ALTPHYS).
1994-12-21 22:57:05 +00:00
case SIOCSIFPHYS:
Sweep kernel replacing suser(9) calls with priv(9) calls, assigning specific privilege names to a broad range of privileges. These may require some future tweaking. Sponsored by: nCircle Network Security, Inc. Obtained from: TrustedBSD Project Discussed on: arch@ Reviewed (at least in part) by: mlaier, jmg, pjd, bde, ceri, Alex Lyashkov <umka at sevcity dot net>, Skip Ford <skip dot ford at verizon dot net>, Antoine Brodin <antoine dot brodin at laposte dot net>
2006-11-06 13:42:10 +00:00
error = priv_check(td, PRIV_NET_SETIFPHYS);
Change the use if ifnet.if_lastchange to be more in line with SNMP requirements. Update description of ifnet.if_lastchange in if.h to indicate this.
1996-06-10 23:07:36 +00:00
if (error)
Minor beautifications related to style(9) and code consistency. No functional changes.
2004-02-21 12:56:09 +00:00
return (error);
if (ifp->if_ioctl == NULL)
return (EOPNOTSUPP);
Change the use if ifnet.if_lastchange to be more in line with SNMP requirements. Update description of ifnet.if_lastchange in if.h to indicate this.
1996-06-10 23:07:36 +00:00
error = (*ifp->if_ioctl)(ifp, cmd, data);
if (error == 0)
Use getmicrotime() for if_lastchange, 10msec is plenty precision.
1998-04-06 11:43:12 +00:00
getmicrotime(&ifp->if_lastchange);
Minor beautifications related to style(9) and code consistency. No functional changes.
2004-02-21 12:56:09 +00:00
break;
Add generic part of generic multiple-physical-interface support (the successor of IFF_ALTPHYS).
1994-12-21 22:57:05 +00:00
Added ioctl support for SIOCGIFMTU and SIOCSIFMTU. These set the per- interface MTU.
1994-08-08 10:49:26 +00:00
case SIOCSIFMTU:
KAME netinet6 basic part(no IPsec,no V6 Multicast Forwarding, no UDP/TCP for IPv6 yet) With this patch, you can assigne IPv6 addr automatically, and can reply to IPv6 ping. Reviewed by: freebsd-arch, cvs-committers Obtained from: KAME project
1999-11-22 02:45:11 +00:00
{
u_long oldmtu = ifp->if_mtu;
Sweep kernel replacing suser(9) calls with priv(9) calls, assigning specific privilege names to a broad range of privileges. These may require some future tweaking. Sponsored by: nCircle Network Security, Inc. Obtained from: TrustedBSD Project Discussed on: arch@ Reviewed (at least in part) by: mlaier, jmg, pjd, bde, ceri, Alex Lyashkov <umka at sevcity dot net>, Skip Ford <skip dot ford at verizon dot net>, Antoine Brodin <antoine dot brodin at laposte dot net>
2006-11-06 13:42:10 +00:00
error = priv_check(td, PRIV_NET_SETIFMTU);
Mostly Cosmetics. Some of the procedures in if_sl.c was void, but should be int. I made them int, and let them return 0. Will have to find out what the return-val is used for.
1994-10-08 01:40:23 +00:00
if (error)
Added ioctl support for SIOCGIFMTU and SIOCSIFMTU. These set the per- interface MTU.
1994-08-08 10:49:26 +00:00
return (error);
Define IF_MAXMTU and IF_MINMTU and don't allow MTUs with out-of-range values. ``comparison is always 0'' warnings are silly ! Ok'd by: wollman, dg Advised by: bde
1999-08-06 13:53:03 +00:00
if (ifr->ifr_mtu < IF_MINMTU || ifr->ifr_mtu > IF_MAXMTU)
Enforce the mtu to between the range 1-65535 before calling the driver ioctl routine.
1994-08-08 10:58:30 +00:00
return (EINVAL);
Introduce network device nodes. Network devices will now automatically appear in /dev. Interface hardware ioctls (not protocol or routing) can be performed on the descriptor. The SIOCGIFCONF ioctl may be performed on the special /dev/network node.
2001-09-29 05:55:04 +00:00
if (ifp->if_ioctl == NULL)
return (EOPNOTSUPP);
Change the use if ifnet.if_lastchange to be more in line with SNMP requirements. Update description of ifnet.if_lastchange in if.h to indicate this.
1996-06-10 23:07:36 +00:00
error = (*ifp->if_ioctl)(ifp, cmd, data);
Notify user processes about interface's MTU change. Reviewed by: wollman, freebsd-net
2000-01-24 08:53:39 +00:00
if (error == 0) {
Use getmicrotime() for if_lastchange, 10msec is plenty precision.
1998-04-06 11:43:12 +00:00
getmicrotime(&ifp->if_lastchange);
Notify user processes about interface's MTU change. Reviewed by: wollman, freebsd-net
2000-01-24 08:53:39 +00:00
rt_ifmsg(ifp);
Import the netdump client code. This is a component of a system which lets the kernel dump core to a remote host after a panic, rather than to a local storage device. The server component is available in the ports tree. netdump is particularly useful on diskless systems. The netdump(4) man page contains some details describing the protocol. Support for configuring netdump will be added to dumpon(8) in a future commit. To use netdump, the kernel must have been compiled with the NETDUMP option. The initial revision of netdump was written by Darrell Anderson and was integrated into Sandvine's OS, from which this version was derived. Reviewed by: bdrewery, cem (earlier versions), julian, sbruno MFC after: 1 month X-MFC note: use a spare field in struct ifnet Sponsored by: Dell EMC Isilon Differential Revision: https://reviews.freebsd.org/D15253
2018-05-06 00:38:29 +00:00
#ifdef INET
NETDUMP_REINIT(ifp);
#endif
Notify user processes about interface's MTU change. Reviewed by: wollman, freebsd-net
2000-01-24 08:53:39 +00:00
}
KAME netinet6 basic part(no IPsec,no V6 Multicast Forwarding, no UDP/TCP for IPv6 yet) With this patch, you can assigne IPv6 addr automatically, and can reply to IPv6 ping. Reviewed by: freebsd-arch, cvs-committers Obtained from: KAME project
1999-11-22 02:45:11 +00:00
/*
* If the link MTU changed, do network layer specific procedure.
*/
if (ifp->if_mtu != oldmtu) {
#ifdef INET6
nd6_setmtu(ifp);
#endif
Finish r274175: do control plane MTU tracking. Update route MTU in case of ifnet MTU change. Add new RTF_FIXEDMTU to track explicitly specified MTU. Old behavior: ifconfig em0 mtu 1500->9000 -> all routes traversing em0 do not change MTU. User has to manually update all routes. ifconfig em0 mtu 9000->1500 -> all routes traversing em0 do not change MTU. However, if ip[6]_output finds route with rt_mtu > interface mtu, rt_mtu gets updated. New behavior: ifconfig em0 mtu 1500->9000 -> all interface routes in all fibs gets updated with new MTU unless RTF_FIXEDMTU flag set on them. ifconfig em0 mtu 9000->1500 -> all routes in all fibs gets updated with new MTU unless RTF_FIXEDMTU flag set on them AND rt_mtu is less than ifp mtu. route add ... -mtu XXX automatically sets RTF_FIXEDMTU flag. route change .. -mtu 0 automatically removes RTF_FIXEDMTU flag. PR: 194238 MFC after: 1 month CR: D1125
2014-11-17 01:05:29 +00:00
rt_updatemtu(ifp);
KAME netinet6 basic part(no IPsec,no V6 Multicast Forwarding, no UDP/TCP for IPv6 yet) With this patch, you can assigne IPv6 addr automatically, and can reply to IPv6 ping. Reviewed by: freebsd-arch, cvs-committers Obtained from: KAME project
1999-11-22 02:45:11 +00:00
}
Introduce network device nodes. Network devices will now automatically appear in /dev. Interface hardware ioctls (not protocol or routing) can be performed on the descriptor. The SIOCGIFCONF ioctl may be performed on the special /dev/network node.
2001-09-29 05:55:04 +00:00
break;
KAME netinet6 basic part(no IPsec,no V6 Multicast Forwarding, no UDP/TCP for IPv6 yet) With this patch, you can assigne IPv6 addr automatically, and can reply to IPv6 ping. Reviewed by: freebsd-arch, cvs-committers Obtained from: KAME project
1999-11-22 02:45:11 +00:00
}
Added ioctl support for SIOCGIFMTU and SIOCSIFMTU. These set the per- interface MTU.
1994-08-08 10:49:26 +00:00
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
case SIOCADDMULTI:
case SIOCDELMULTI:
Sweep kernel replacing suser(9) calls with priv(9) calls, assigning specific privilege names to a broad range of privileges. These may require some future tweaking. Sponsored by: nCircle Network Security, Inc. Obtained from: TrustedBSD Project Discussed on: arch@ Reviewed (at least in part) by: mlaier, jmg, pjd, bde, ceri, Alex Lyashkov <umka at sevcity dot net>, Skip Ford <skip dot ford at verizon dot net>, Antoine Brodin <antoine dot brodin at laposte dot net>
2006-11-06 13:42:10 +00:00
if (cmd == SIOCADDMULTI)
error = priv_check(td, PRIV_NET_ADDMULTI);
else
error = priv_check(td, PRIV_NET_DELMULTI);
Mostly Cosmetics. Some of the procedures in if_sl.c was void, but should be int. I made them int, and let them return 0. Will have to find out what the return-val is used for.
1994-10-08 01:40:23 +00:00
if (error)
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
return (error);
Use the new if_multiaddrs list for multicast addresses rather than the previous hackery involving struct in_ifaddr and arpcom. Get rid of the abominable multi_kludge. Update all network interfaces to use the new machanism. Distressingly few Ethernet drivers program the multicast filter properly (assuming the hardware has one, which it usually does).
1997-01-13 21:26:53 +00:00
/* Don't allow group membership on non-multicast interfaces. */
if ((ifp->if_flags & IFF_MULTICAST) == 0)
Introduce network device nodes. Network devices will now automatically appear in /dev. Interface hardware ioctls (not protocol or routing) can be performed on the descriptor. The SIOCGIFCONF ioctl may be performed on the special /dev/network node.
2001-09-29 05:55:04 +00:00
return (EOPNOTSUPP);
Use the new if_multiaddrs list for multicast addresses rather than the previous hackery involving struct in_ifaddr and arpcom. Get rid of the abominable multi_kludge. Update all network interfaces to use the new machanism. Distressingly few Ethernet drivers program the multicast filter properly (assuming the hardware has one, which it usually does).
1997-01-13 21:26:53 +00:00
/* Don't let users screw up protocols' entries. */
if (ifr->ifr_addr.sa_family != AF_LINK)
Introduce network device nodes. Network devices will now automatically appear in /dev. Interface hardware ioctls (not protocol or routing) can be performed on the descriptor. The SIOCGIFCONF ioctl may be performed on the special /dev/network node.
2001-09-29 05:55:04 +00:00
return (EINVAL);
Use the new if_multiaddrs list for multicast addresses rather than the previous hackery involving struct in_ifaddr and arpcom. Get rid of the abominable multi_kludge. Update all network interfaces to use the new machanism. Distressingly few Ethernet drivers program the multicast filter properly (assuming the hardware has one, which it usually does).
1997-01-13 21:26:53 +00:00
if (cmd == SIOCADDMULTI) {
Mechanical cleanup of epoch(9) usage in network stack. - Remove macros that covertly create epoch_tracker on thread stack. Such macros a quite unsafe, e.g. will produce a buggy code if same macro is used in embedded scopes. Explicitly declare epoch_tracker always. - Unmask interface list IFNET_RLOCK_NOSLEEP(), interface address list IF_ADDR_RLOCK() and interface AF specific data IF_AFDATA_RLOCK() read locking macros to what they actually are - the net_epoch. Keeping them as is is very misleading. They all are named FOO_RLOCK(), while they no longer have lock semantics. Now they allow recursion and what's more important they now no longer guarantee protection against their companion WLOCK macros. Note: INP_HASH_RLOCK() has same problems, but not touched by this commit. This is non functional mechanical change. The only functionally changed functions are ni6_addrs() and ni6_store_addrs(), where we no longer enter epoch recursively. Discussed with: jtl, gallatin
2019-01-09 01:11:19 +00:00
struct epoch_tracker et;
Use the new if_multiaddrs list for multicast addresses rather than the previous hackery involving struct in_ifaddr and arpcom. Get rid of the abominable multi_kludge. Update all network interfaces to use the new machanism. Distressingly few Ethernet drivers program the multicast filter properly (assuming the hardware has one, which it usually does).
1997-01-13 21:26:53 +00:00
struct ifmultiaddr *ifma;
Implement reference counting for ifmultiaddr, in_multi, and in6_multi structures. Detect when ifnet instances are detached from the network stack and perform appropriate cleanup to prevent memory leaks. This has been implemented in such a way as to be backwards ABI compatible. Kernel consumers are changed to use if_delmulti_ifma(); in_delmulti() is unable to detect interface removal by design, as it performs searches on structures which are removed with the interface. With this architectural change, the panics FreeBSD users have experienced with carp and pfsync should be resolved. Obtained from: p4 branch bms_netdev Reviewed by: andre Sponsored by: Garance A Drosehn Idea from: NetBSD MFC after: 1 month
2007-03-20 00:36:10 +00:00
/*
* Userland is only permitted to join groups once
* via the if_addmulti() KPI, because it cannot hold
* struct ifmultiaddr * between calls. It may also
* lose a race while we check if the membership
* already exists.
*/
Mechanical cleanup of epoch(9) usage in network stack. - Remove macros that covertly create epoch_tracker on thread stack. Such macros a quite unsafe, e.g. will produce a buggy code if same macro is used in embedded scopes. Explicitly declare epoch_tracker always. - Unmask interface list IFNET_RLOCK_NOSLEEP(), interface address list IF_ADDR_RLOCK() and interface AF specific data IF_AFDATA_RLOCK() read locking macros to what they actually are - the net_epoch. Keeping them as is is very misleading. They all are named FOO_RLOCK(), while they no longer have lock semantics. Now they allow recursion and what's more important they now no longer guarantee protection against their companion WLOCK macros. Note: INP_HASH_RLOCK() has same problems, but not touched by this commit. This is non functional mechanical change. The only functionally changed functions are ni6_addrs() and ni6_store_addrs(), where we no longer enter epoch recursively. Discussed with: jtl, gallatin
2019-01-09 01:11:19 +00:00
NET_EPOCH_ENTER(et);
Implement reference counting for ifmultiaddr, in_multi, and in6_multi structures. Detect when ifnet instances are detached from the network stack and perform appropriate cleanup to prevent memory leaks. This has been implemented in such a way as to be backwards ABI compatible. Kernel consumers are changed to use if_delmulti_ifma(); in_delmulti() is unable to detect interface removal by design, as it performs searches on structures which are removed with the interface. With this architectural change, the panics FreeBSD users have experienced with carp and pfsync should be resolved. Obtained from: p4 branch bms_netdev Reviewed by: andre Sponsored by: Garance A Drosehn Idea from: NetBSD MFC after: 1 month
2007-03-20 00:36:10 +00:00
ifma = if_findmulti(ifp, &ifr->ifr_addr);
Mechanical cleanup of epoch(9) usage in network stack. - Remove macros that covertly create epoch_tracker on thread stack. Such macros a quite unsafe, e.g. will produce a buggy code if same macro is used in embedded scopes. Explicitly declare epoch_tracker always. - Unmask interface list IFNET_RLOCK_NOSLEEP(), interface address list IF_ADDR_RLOCK() and interface AF specific data IF_AFDATA_RLOCK() read locking macros to what they actually are - the net_epoch. Keeping them as is is very misleading. They all are named FOO_RLOCK(), while they no longer have lock semantics. Now they allow recursion and what's more important they now no longer guarantee protection against their companion WLOCK macros. Note: INP_HASH_RLOCK() has same problems, but not touched by this commit. This is non functional mechanical change. The only functionally changed functions are ni6_addrs() and ni6_store_addrs(), where we no longer enter epoch recursively. Discussed with: jtl, gallatin
2019-01-09 01:11:19 +00:00
NET_EPOCH_EXIT(et);
Implement reference counting for ifmultiaddr, in_multi, and in6_multi structures. Detect when ifnet instances are detached from the network stack and perform appropriate cleanup to prevent memory leaks. This has been implemented in such a way as to be backwards ABI compatible. Kernel consumers are changed to use if_delmulti_ifma(); in_delmulti() is unable to detect interface removal by design, as it performs searches on structures which are removed with the interface. With this architectural change, the panics FreeBSD users have experienced with carp and pfsync should be resolved. Obtained from: p4 branch bms_netdev Reviewed by: andre Sponsored by: Garance A Drosehn Idea from: NetBSD MFC after: 1 month
2007-03-20 00:36:10 +00:00
if (ifma != NULL)
error = EADDRINUSE;
else
error = if_addmulti(ifp, &ifr->ifr_addr, &ifma);
Use the new if_multiaddrs list for multicast addresses rather than the previous hackery involving struct in_ifaddr and arpcom. Get rid of the abominable multi_kludge. Update all network interfaces to use the new machanism. Distressingly few Ethernet drivers program the multicast filter properly (assuming the hardware has one, which it usually does).
1997-01-13 21:26:53 +00:00
} else {
error = if_delmulti(ifp, &ifr->ifr_addr);
}
if (error == 0)
Use getmicrotime() for if_lastchange, 10msec is plenty precision.
1998-04-06 11:43:12 +00:00
getmicrotime(&ifp->if_lastchange);
Introduce network device nodes. Network devices will now automatically appear in /dev. Interface hardware ioctls (not protocol or routing) can be performed on the descriptor. The SIOCGIFCONF ioctl may be performed on the special /dev/network node.
2001-09-29 05:55:04 +00:00
break;
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
make sure we have root priv on SIOCSIFPHY*. from thorpej@netbsd
2000-10-04 23:16:29 +00:00
case SIOCSIFPHYADDR:
case SIOCDIFPHYADDR:
#ifdef INET6
case SIOCSIFPHYADDR_IN6:
#endif
Clean up whitespace, remove "register" keyword, ANSIfy. No functional changes.
2003-10-23 13:49:10 +00:00
case SIOCSIFMEDIA:
Ooops, this should have made it into the same commit, but didn't. Introduce the SIOC[SG]IFGENERIC hooks that can be used to pass an arbritrary ioctl subcommand into an interface driver. Surprisingly enough, there was no provision for this already present (except of the option of abusing SIOC[SG]IFMEDIA for this). The idea is that an interface driver can establish ioctl subcommands of its own that can't be meaningfully interpreted by the upper layer interface ioctl function. Something like this is required to implement a clean solution of passing down things like CHAP secrets or PPP options to the /sys/net/if_sppp* files. (Yes, my CHAP is now finally working with it, but i gotta update my kernel to the new callout interface before being able to commit _that_.) Reviewed by: peter [long ago, actually]
1997-10-07 07:40:35 +00:00
case SIOCSIFGENERIC:
Sweep kernel replacing suser(9) calls with priv(9) calls, assigning specific privilege names to a broad range of privileges. These may require some future tweaking. Sponsored by: nCircle Network Security, Inc. Obtained from: TrustedBSD Project Discussed on: arch@ Reviewed (at least in part) by: mlaier, jmg, pjd, bde, ceri, Alex Lyashkov <umka at sevcity dot net>, Skip Ford <skip dot ford at verizon dot net>, Antoine Brodin <antoine dot brodin at laposte dot net>
2006-11-06 13:42:10 +00:00
error = priv_check(td, PRIV_NET_HWIOCTL);
add SIOC{S,G}IFMEDIA ioctl support
1997-05-03 21:07:13 +00:00
if (error)
return (error);
Introduce network device nodes. Network devices will now automatically appear in /dev. Interface hardware ioctls (not protocol or routing) can be performed on the descriptor. The SIOCGIFCONF ioctl may be performed on the special /dev/network node.
2001-09-29 05:55:04 +00:00
if (ifp->if_ioctl == NULL)
add SIOC{S,G}IFMEDIA ioctl support
1997-05-03 21:07:13 +00:00
return (EOPNOTSUPP);
error = (*ifp->if_ioctl)(ifp, cmd, data);
if (error == 0)
Use getmicrotime() for if_lastchange, 10msec is plenty precision.
1998-04-06 11:43:12 +00:00
getmicrotime(&ifp->if_lastchange);
Introduce network device nodes. Network devices will now automatically appear in /dev. Interface hardware ioctls (not protocol or routing) can be performed on the descriptor. The SIOCGIFCONF ioctl may be performed on the special /dev/network node.
2001-09-29 05:55:04 +00:00
break;
add SIOC{S,G}IFMEDIA ioctl support
1997-05-03 21:07:13 +00:00
Add a new interface ioctl, to return "aux status". This is inteded for to allow ifconfig to print various unstructured information from an interface. The data is returned from the kernel in ASCII form, see the comment in if.h for some technicalities. Canonical cut&paste example to be found in if_tun.c Initial use: Now tun* interfaces tell the PID of the process which opened them. Future uses could be (volounteers welcome!): Have ppp/slip interfaces tell which tty they use. Make sync interfaces return their media state: red/yellow/blue alarm, timeslot assignment and so on. Make ethernets warn about missing heartbeats and/or cables
1999-06-19 18:42:31 +00:00
case SIOCGIFSTATUS:
Sync with recent KAME. This work was based on kame-20010528-freebsd43-snap.tgz and some critical problem after the snap was out were fixed. There are many many changes since last KAME merge. TODO: - The definitions of SADB_* in sys/net/pfkeyv2.h are still different from RFC2407/IANA assignment because of binary compatibility issue. It should be fixed under 5-CURRENT. - ip6po_m member of struct ip6_pktopts is no longer used. But, it is still there because of binary compatibility issue. It should be removed under 5-CURRENT. Reviewed by: itojun Obtained from: KAME MFC after: 3 weeks
2001-06-11 12:39:29 +00:00
case SIOCGIFPSRCADDR:
case SIOCGIFPDSTADDR:
add SIOC{S,G}IFMEDIA ioctl support
1997-05-03 21:07:13 +00:00
case SIOCGIFMEDIA:
ifmedia changes: - Extend the number of available subtypes for Ethernet media by using some of the ifmedia word's option bits to help denote subtypes. As a result, the number of possible Ethernet subtype values increases from 31 to 511. - Use some of those new values to define new media types. - lacp_compose_key() recgonizes the new Ethernet media types added. (Change made as required by a comment in if_media.h) - New ioctl, SIOGIFXMEDIA, to handle getting the new extended media types. SIOCGIFMEDIA is retained for backwards compatibility. - Changes to ifconfig to allow it to handle the new extended media types. Submitted by: mike@karels.net (original), hselasky Reviewed by: jfvogel, gnn, hselasky Approved by: jfvogel (mentor), gnn (mentor) Differential Revision: http://reviews.freebsd.org/D1965
2015-04-07 21:31:17 +00:00
case SIOCGIFXMEDIA:
Ooops, this should have made it into the same commit, but didn't. Introduce the SIOC[SG]IFGENERIC hooks that can be used to pass an arbritrary ioctl subcommand into an interface driver. Surprisingly enough, there was no provision for this already present (except of the option of abusing SIOC[SG]IFMEDIA for this). The idea is that an interface driver can establish ioctl subcommands of its own that can't be meaningfully interpreted by the upper layer interface ioctl function. Something like this is required to implement a clean solution of passing down things like CHAP secrets or PPP options to the /sys/net/if_sppp* files. (Yes, my CHAP is now finally working with it, but i gotta update my kernel to the new callout interface before being able to commit _that_.) Reviewed by: peter [long ago, actually]
1997-10-07 07:40:35 +00:00
case SIOCGIFGENERIC:
if: Add ioctls to get RSS key and hash type/function. It will be needed by hn(4) to configure its RSS key and hash type/function in the transparent VF mode in order to match VF's RSS settings. The description of the transparent VF mode and the RSS hash value issue are here: https://svnweb.freebsd.org/base?view=revision&revision=322299 https://svnweb.freebsd.org/base?view=revision&revision=322485 These are generic enough to promise two independent IOCs instead of abusing SIOCGDRVSPEC. Setting RSS key and hash type/function is a different story, which probably requires more discussion. Comment about UDP_{IPV4,IPV6,IPV6_EX} were only in the patch in the review request; these hash types are standardized now. Reviewed by: gallatin MFC after: 1 week Sponsored by: Microsoft Differential Revision: https://reviews.freebsd.org/D12174
2017-09-05 05:28:52 +00:00
case SIOCGIFRSSKEY:
case SIOCGIFRSSHASH:
Add SIOCGIFDOWNREASON. The ioctl(2) is intended to provide more details about the cause of the down for the link. Eventually we might define a comprehensive list of codes for the situations. But interface also allows the driver to provide free-form null-terminated ASCII string to provide arbitrary non-formalized information. Sample implementation exists for mlx5(4), where the string is fetched from firmware controlling the port. Reviewed by: hselasky, rrs Sponsored by: Mellanox Technologies MFC after: 1 week Differential revision: https://reviews.freebsd.org/D21527
2019-09-17 18:49:13 +00:00
case SIOCGIFDOWNREASON:
Minor beautifications related to style(9) and code consistency. No functional changes.
2004-02-21 12:56:09 +00:00
if (ifp->if_ioctl == NULL)
add SIOC{S,G}IFMEDIA ioctl support
1997-05-03 21:07:13 +00:00
return (EOPNOTSUPP);
Introduce network device nodes. Network devices will now automatically appear in /dev. Interface hardware ioctls (not protocol or routing) can be performed on the descriptor. The SIOCGIFCONF ioctl may be performed on the special /dev/network node.
2001-09-29 05:55:04 +00:00
error = (*ifp->if_ioctl)(ifp, cmd, data);
break;
add SIOC{S,G}IFMEDIA ioctl support
1997-05-03 21:07:13 +00:00
Implement SIOCSIFLLADDR, which allows you to change the link-level address on an interface. This basically allows you to do what my little setmac module/utility does via ifconfig. This involves the following changes: socket.h: define SIOCSIFLLADDR if.c: add support for SIOCSIFLLADDR, which resets the values in the arpcom struct and sockaddr_dl for the specified interface. Note that if the interface is already up, we need to down/up it in order to program the underlying hardware's receive filter. ifconfig.c: add lladdr command ifconfig.8: document lladdr command You can now force the MAC address on any ethernet interface to be whatever you want. (The change is not sticky across reboots of course: we don't actually reprogram the EEPROM or anything.) Actually, you can reprogram the MAC address on other kinds of interfaces too; this shouldn't be ethernet-specific (though at the moment it's limited to 6 bytes of address data). Nobody ran up to me and said "this is the politically correct way to do this!" so I don't want to hear any complaints from people who think I could have done it more elegantly. Consider yourselves lucky I didn't do it by having ifconfig tread all over /dev/kmem.
2000-06-16 20:14:43 +00:00
case SIOCSIFLLADDR:
Sweep kernel replacing suser(9) calls with priv(9) calls, assigning specific privilege names to a broad range of privileges. These may require some future tweaking. Sponsored by: nCircle Network Security, Inc. Obtained from: TrustedBSD Project Discussed on: arch@ Reviewed (at least in part) by: mlaier, jmg, pjd, bde, ceri, Alex Lyashkov <umka at sevcity dot net>, Skip Ford <skip dot ford at verizon dot net>, Antoine Brodin <antoine dot brodin at laposte dot net>
2006-11-06 13:42:10 +00:00
error = priv_check(td, PRIV_NET_SETLLADDR);
Implement SIOCSIFLLADDR, which allows you to change the link-level address on an interface. This basically allows you to do what my little setmac module/utility does via ifconfig. This involves the following changes: socket.h: define SIOCSIFLLADDR if.c: add support for SIOCSIFLLADDR, which resets the values in the arpcom struct and sockaddr_dl for the specified interface. Note that if the interface is already up, we need to down/up it in order to program the underlying hardware's receive filter. ifconfig.c: add lladdr command ifconfig.8: document lladdr command You can now force the MAC address on any ethernet interface to be whatever you want. (The change is not sticky across reboots of course: we don't actually reprogram the EEPROM or anything.) Actually, you can reprogram the MAC address on other kinds of interfaces too; this shouldn't be ethernet-specific (though at the moment it's limited to 6 bytes of address data). Nobody ran up to me and said "this is the politically correct way to do this!" so I don't want to hear any complaints from people who think I could have done it more elegantly. Consider yourselves lucky I didn't do it by having ifconfig tread all over /dev/kmem.
2000-06-16 20:14:43 +00:00
if (error)
return (error);
Introduce network device nodes. Network devices will now automatically appear in /dev. Interface hardware ioctls (not protocol or routing) can be performed on the descriptor. The SIOCGIFCONF ioctl may be performed on the special /dev/network node.
2001-09-29 05:55:04 +00:00
error = if_setlladdr(ifp,
Export the functionality of SIOCSIFLLADDR with if_setlladdr() and add some more rigorous sanity checking in the process. Reviewed by: freebsd-net
2000-08-15 00:48:38 +00:00
ifr->ifr_addr.sa_data, ifr->ifr_addr.sa_len);
Introduce network device nodes. Network devices will now automatically appear in /dev. Interface hardware ioctls (not protocol or routing) can be performed on the descriptor. The SIOCGIFCONF ioctl may be performed on the special /dev/network node.
2001-09-29 05:55:04 +00:00
break;
Export the functionality of SIOCSIFLLADDR with if_setlladdr() and add some more rigorous sanity checking in the process. Reviewed by: freebsd-net
2000-08-15 00:48:38 +00:00
Persistently store NIC's hardware MAC address, and add a way to retrive it The MAC address reported by `ifconfig ${nic} ether' does not always match the address in the hardware, as reported by the driver during attach. In particular, NICs which are components of a lagg(4) interface all report the same MAC. When attaching, the NIC driver passes the MAC address it read from the hardware as an argument to ether_ifattach(). Keep a second copy of it, and create ioctl(SIOCGHWADDR) to return it. Teach `ifconfig' to report it along with the active MAC address. PR: 194386 Reviewed by: glebius MFC after: 1 week Sponsored by: Panasas Differential Revision: https://reviews.freebsd.org/D10609
2017-05-10 22:13:47 +00:00
case SIOCGHWADDR:
error = if_gethwaddr(ifp, ifr);
break;
Rework CASE_IOC_IFGROUPREQ() to require a case before the macro. This is more compatible with formatting tools and looks more normal. Reported by: jhb (on a different review) Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D18442
2019-01-22 17:39:26 +00:00
case CASE_IOC_IFGROUPREQ(SIOCAIFGROUP):
Sweep kernel replacing suser(9) calls with priv(9) calls, assigning specific privilege names to a broad range of privileges. These may require some future tweaking. Sponsored by: nCircle Network Security, Inc. Obtained from: TrustedBSD Project Discussed on: arch@ Reviewed (at least in part) by: mlaier, jmg, pjd, bde, ceri, Alex Lyashkov <umka at sevcity dot net>, Skip Ford <skip dot ford at verizon dot net>, Antoine Brodin <antoine dot brodin at laposte dot net>
2006-11-06 13:42:10 +00:00
error = priv_check(td, PRIV_NET_ADDIFGROUP);
Import interface groups from OpenBSD. This allows to group interfaces in order to - for example - apply firewall rules to a whole group of interfaces. This is required for importing pf from OpenBSD 3.9 Obtained from: OpenBSD (with changes) Discussed on: -net (back in April)
2006-06-19 22:20:45 +00:00
if (error)
return (error);
Add 32-bit compat for ioctls that take struct ifgroupreq. Use an accessor to access ifgr_group and ifgr_groups. Use an macro CASE_IOC_IFGROUPREQ(cmd) in place of case statements such as "case SIOCAIFGROUP:". This avoids poluting the switch statements with large numbers of #ifdefs. Reviewed by: kib Obtained from: CheriBSD MFC after: 1 week Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D14960
2018-04-05 22:14:55 +00:00
if ((error = if_addgroup(ifp,
ifgr_group_get((struct ifgroupreq *)data))))
Import interface groups from OpenBSD. This allows to group interfaces in order to - for example - apply firewall rules to a whole group of interfaces. This is required for importing pf from OpenBSD 3.9 Obtained from: OpenBSD (with changes) Discussed on: -net (back in April)
2006-06-19 22:20:45 +00:00
return (error);
break;
Rework CASE_IOC_IFGROUPREQ() to require a case before the macro. This is more compatible with formatting tools and looks more normal. Reported by: jhb (on a different review) Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D18442
2019-01-22 17:39:26 +00:00
case CASE_IOC_IFGROUPREQ(SIOCGIFGROUP):
Widen NET_EPOCH coverage. When epoch(9) was introduced to network stack, it was basically dropped in place of existing locking, which was mutexes and rwlocks. For the sake of performance mutex covered areas were as small as possible, so became epoch covered areas. However, epoch doesn't introduce any contention, it just delays memory reclaim. So, there is no point to minimise epoch covered areas in sense of performance. Meanwhile entering/exiting epoch also has non-zero CPU usage, so doing this less often is a win. Not the least is also code maintainability. In the new paradigm we can assume that at any stage of processing a packet, we are inside network epoch. This makes coding both input and output path way easier. On output path we already enter epoch quite early - in the ip_output(), in the ip6_output(). This patch does the same for the input path. All ISR processing, network related callouts, other ways of packet injection to the network stack shall be performed in net_epoch. Any leaf function that walks network configuration now asserts epoch. Tricky part is configuration code paths - ioctls, sysctls. They also call into leaf functions, so some need to be changed. This patch would introduce more epoch recursions (see EPOCH_TRACE) than we had before. They will be cleaned up separately, as several of them aren't trivial. Note, that unlike a lock recursion the epoch recursion is safe and just wastes a bit of resources. Reviewed by: gallatin, hselasky, cy, adrian, kristof Differential Revision: https://reviews.freebsd.org/D19111
2019-10-07 22:40:05 +00:00
{
struct epoch_tracker et;
NET_EPOCH_ENTER(et);
error = if_getgroup((struct ifgroupreq *)data, ifp);
NET_EPOCH_EXIT(et);
Import interface groups from OpenBSD. This allows to group interfaces in order to - for example - apply firewall rules to a whole group of interfaces. This is required for importing pf from OpenBSD 3.9 Obtained from: OpenBSD (with changes) Discussed on: -net (back in April)
2006-06-19 22:20:45 +00:00
break;
Widen NET_EPOCH coverage. When epoch(9) was introduced to network stack, it was basically dropped in place of existing locking, which was mutexes and rwlocks. For the sake of performance mutex covered areas were as small as possible, so became epoch covered areas. However, epoch doesn't introduce any contention, it just delays memory reclaim. So, there is no point to minimise epoch covered areas in sense of performance. Meanwhile entering/exiting epoch also has non-zero CPU usage, so doing this less often is a win. Not the least is also code maintainability. In the new paradigm we can assume that at any stage of processing a packet, we are inside network epoch. This makes coding both input and output path way easier. On output path we already enter epoch quite early - in the ip_output(), in the ip6_output(). This patch does the same for the input path. All ISR processing, network related callouts, other ways of packet injection to the network stack shall be performed in net_epoch. Any leaf function that walks network configuration now asserts epoch. Tricky part is configuration code paths - ioctls, sysctls. They also call into leaf functions, so some need to be changed. This patch would introduce more epoch recursions (see EPOCH_TRACE) than we had before. They will be cleaned up separately, as several of them aren't trivial. Note, that unlike a lock recursion the epoch recursion is safe and just wastes a bit of resources. Reviewed by: gallatin, hselasky, cy, adrian, kristof Differential Revision: https://reviews.freebsd.org/D19111
2019-10-07 22:40:05 +00:00
}
Import interface groups from OpenBSD. This allows to group interfaces in order to - for example - apply firewall rules to a whole group of interfaces. This is required for importing pf from OpenBSD 3.9 Obtained from: OpenBSD (with changes) Discussed on: -net (back in April)
2006-06-19 22:20:45 +00:00
Rework CASE_IOC_IFGROUPREQ() to require a case before the macro. This is more compatible with formatting tools and looks more normal. Reported by: jhb (on a different review) Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D18442
2019-01-22 17:39:26 +00:00
case CASE_IOC_IFGROUPREQ(SIOCDIFGROUP):
Sweep kernel replacing suser(9) calls with priv(9) calls, assigning specific privilege names to a broad range of privileges. These may require some future tweaking. Sponsored by: nCircle Network Security, Inc. Obtained from: TrustedBSD Project Discussed on: arch@ Reviewed (at least in part) by: mlaier, jmg, pjd, bde, ceri, Alex Lyashkov <umka at sevcity dot net>, Skip Ford <skip dot ford at verizon dot net>, Antoine Brodin <antoine dot brodin at laposte dot net>
2006-11-06 13:42:10 +00:00
error = priv_check(td, PRIV_NET_DELIFGROUP);
Import interface groups from OpenBSD. This allows to group interfaces in order to - for example - apply firewall rules to a whole group of interfaces. This is required for importing pf from OpenBSD 3.9 Obtained from: OpenBSD (with changes) Discussed on: -net (back in April)
2006-06-19 22:20:45 +00:00
if (error)
return (error);
Add 32-bit compat for ioctls that take struct ifgroupreq. Use an accessor to access ifgr_group and ifgr_groups. Use an macro CASE_IOC_IFGROUPREQ(cmd) in place of case statements such as "case SIOCAIFGROUP:". This avoids poluting the switch statements with large numbers of #ifdefs. Reviewed by: kib Obtained from: CheriBSD MFC after: 1 week Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D14960
2018-04-05 22:14:55 +00:00
if ((error = if_delgroup(ifp,
ifgr_group_get((struct ifgroupreq *)data))))
Import interface groups from OpenBSD. This allows to group interfaces in order to - for example - apply firewall rules to a whole group of interfaces. This is required for importing pf from OpenBSD 3.9 Obtained from: OpenBSD (with changes) Discussed on: -net (back in April)
2006-06-19 22:20:45 +00:00
return (error);
break;
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
default:
Introduce network device nodes. Network devices will now automatically appear in /dev. Interface hardware ioctls (not protocol or routing) can be performed on the descriptor. The SIOCGIFCONF ioctl may be performed on the special /dev/network node.
2001-09-29 05:55:04 +00:00
error = ENOIOCTL;
break;
}
return (error);
}
Reshuffle SIOCGIFCONF32 handler from r155224. - move all the chunks into one file, which allows to hide SIOCGIFCONF32 global definition as well. - replace __amd64__ with proper COMPAT_FREEBSD32 around. - handle 32bit capacity before going into the handler itself instead of doing internal 32bit specific changes within it (e.g. as it's done for SIOCGDEFIFACE32_IN6). - use explicitely sized types for ABI compat. Approved by: kib (mentor) MFC after: 2 weeks
2010-10-21 16:20:48 +00:00
#ifdef COMPAT_FREEBSD32
struct ifconf32 {
int32_t ifc_len;
union {
uint32_t ifcu_buf;
uint32_t ifcu_req;
} ifc_ifcu;
};
#define SIOCGIFCONF32 _IOWR('i', 36, struct ifconf32)
#endif
Translate 32-bit ifmedia requests into native ones. We use transformation rather than accessors as virtually ever driver implements SIOCGIFMEDIA and all would have to be touched. Keep the code readable by always performing copies and (possiably no-op) transforms. Reviewed by: jhb, kib Obtained from: CheriBSD MFC after: 1 week Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D14996
2018-04-25 15:30:42 +00:00
#ifdef COMPAT_FREEBSD32
static void
ifmr_init(struct ifmediareq *ifmr, caddr_t data)
{
struct ifmediareq32 *ifmr32;
ifmr32 = (struct ifmediareq32 *)data;
memcpy(ifmr->ifm_name, ifmr32->ifm_name,
sizeof(ifmr->ifm_name));
ifmr->ifm_current = ifmr32->ifm_current;
ifmr->ifm_mask = ifmr32->ifm_mask;
ifmr->ifm_status = ifmr32->ifm_status;
ifmr->ifm_active = ifmr32->ifm_active;
ifmr->ifm_count = ifmr32->ifm_count;
ifmr->ifm_ulist = (int *)(uintptr_t)ifmr32->ifm_ulist;
}
static void
ifmr_update(const struct ifmediareq *ifmr, caddr_t data)
{
struct ifmediareq32 *ifmr32;
ifmr32 = (struct ifmediareq32 *)data;
ifmr32->ifm_current = ifmr->ifm_current;
ifmr32->ifm_mask = ifmr->ifm_mask;
ifmr32->ifm_status = ifmr->ifm_status;
ifmr32->ifm_active = ifmr->ifm_active;
ifmr32->ifm_count = ifmr->ifm_count;
}
#endif
Introduce network device nodes. Network devices will now automatically appear in /dev. Interface hardware ioctls (not protocol or routing) can be performed on the descriptor. The SIOCGIFCONF ioctl may be performed on the special /dev/network node.
2001-09-29 05:55:04 +00:00
/*
* Interface ioctls.
*/
int
Clean up whitespace, remove "register" keyword, ANSIfy. No functional changes.
2003-10-23 13:49:10 +00:00
ifioctl(struct socket *so, u_long cmd, caddr_t data, struct thread *td)
Introduce network device nodes. Network devices will now automatically appear in /dev. Interface hardware ioctls (not protocol or routing) can be performed on the descriptor. The SIOCGIFCONF ioctl may be performed on the special /dev/network node.
2001-09-29 05:55:04 +00:00
{
Translate 32-bit ifmedia requests into native ones. We use transformation rather than accessors as virtually ever driver implements SIOCGIFMEDIA and all would have to be touched. Keep the code readable by always performing copies and (possiably no-op) transforms. Reviewed by: jhb, kib Obtained from: CheriBSD MFC after: 1 week Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D14996
2018-04-25 15:30:42 +00:00
#ifdef COMPAT_FREEBSD32
net: fix uninitialized variable warning
2018-05-19 19:00:04 +00:00
caddr_t saved_data = NULL;
Translate 32-bit ifmedia requests into native ones. We use transformation rather than accessors as virtually ever driver implements SIOCGIFMEDIA and all would have to be touched. Keep the code readable by always performing copies and (possiably no-op) transforms. Reviewed by: jhb, kib Obtained from: CheriBSD MFC after: 1 week Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D14996
2018-04-25 15:30:42 +00:00
struct ifmediareq ifmr;
Widen NET_EPOCH coverage. When epoch(9) was introduced to network stack, it was basically dropped in place of existing locking, which was mutexes and rwlocks. For the sake of performance mutex covered areas were as small as possible, so became epoch covered areas. However, epoch doesn't introduce any contention, it just delays memory reclaim. So, there is no point to minimise epoch covered areas in sense of performance. Meanwhile entering/exiting epoch also has non-zero CPU usage, so doing this less often is a win. Not the least is also code maintainability. In the new paradigm we can assume that at any stage of processing a packet, we are inside network epoch. This makes coding both input and output path way easier. On output path we already enter epoch quite early - in the ip_output(), in the ip6_output(). This patch does the same for the input path. All ISR processing, network related callouts, other ways of packet injection to the network stack shall be performed in net_epoch. Any leaf function that walks network configuration now asserts epoch. Tricky part is configuration code paths - ioctls, sysctls. They also call into leaf functions, so some need to be changed. This patch would introduce more epoch recursions (see EPOCH_TRACE) than we had before. They will be cleaned up separately, as several of them aren't trivial. Note, that unlike a lock recursion the epoch recursion is safe and just wastes a bit of resources. Reviewed by: gallatin, hselasky, cy, adrian, kristof Differential Revision: https://reviews.freebsd.org/D19111
2019-10-07 22:40:05 +00:00
struct ifmediareq *ifmrp = NULL;
struct ifmediareq *ifmrp is only used in the COMPAT_FREEBSD32 parts of ifioctl(). Move it inside the proper #ifdef. This was throwing a valid "Assigned but unused" warning with gcc. Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D16063
2018-07-07 13:35:06 +00:00
#endif
Introduce network device nodes. Network devices will now automatically appear in /dev. Interface hardware ioctls (not protocol or routing) can be performed on the descriptor. The SIOCGIFCONF ioctl may be performed on the special /dev/network node.
2001-09-29 05:55:04 +00:00
struct ifnet *ifp;
struct ifreq *ifr;
int error;
Increase size of ifnet.if_flags from 16 bits (short) to 32 bits (int). To avoid breaking application ABI use unused ifreq.ifru_flags[1] for upper 16 bits in SIOCSIFFLAGS and SIOCGIFFLAGS ioctl's. Reviewed by: -hackers, -net
2002-08-18 07:05:00 +00:00
int oif_flags;
Introduce network device nodes. Network devices will now automatically appear in /dev. Interface hardware ioctls (not protocol or routing) can be performed on the descriptor. The SIOCGIFCONF ioctl may be performed on the special /dev/network node.
2001-09-29 05:55:04 +00:00
Mfp4 CH=177274,177280,177284-177285,177297,177324-177325 VNET socket push back: try to minimize the number of places where we have to switch vnets and narrow down the time we stay switched. Add assertions to the socket code to catch possibly unset vnets as seen in r204147. While this reduces the number of vnet recursion in some places like NFS, POSIX local sockets and some netgraph, .. recursions are impossible to fix. The current expectations are documented at the beginning of uipc_socket.c along with the other information there. Sponsored by: The FreeBSD Foundation Sponsored by: CK Software GmbH Reviewed by: jhb Tested by: zec Tested by: Mikolaj Golub (to.my.trociny gmail.com) MFC after: 2 weeks
2011-02-16 21:29:13 +00:00
CURVNET_SET(so->so_vnet);
Get closer to a VIMAGE network stack teardown from top to bottom rather than removing the network interfaces first. This change is rather larger and convoluted as the ordering requirements cannot be separated. Move the pfil(9) framework to SI_SUB_PROTO_PFIL, move Firewalls and related modules to their own SI_SUB_PROTO_FIREWALL. Move initialization of "physical" interfaces to SI_SUB_DRIVERS, move virtual (cloned) interfaces to SI_SUB_PSEUDO. Move Multicast to SI_SUB_PROTO_MC. Re-work parts of multicast initialisation and teardown, not taking the huge amount of memory into account if used as a module yet. For interface teardown we try to do as many of them as we can on SI_SUB_INIT_IF, but for some this makes no sense, e.g., when tunnelling over a higher layer protocol such as IP. In that case the interface has to go along (or before) the higher layer protocol is shutdown. Kernel hhooks need to go last on teardown as they may be used at various higher layers and we cannot remove them before we cleaned up the higher layers. For interface teardown there are multiple paths: (a) a cloned interface is destroyed (inside a VIMAGE or in the base system), (b) any interface is moved from a virtual network stack to a different network stack ("vmove"), or (c) a virtual network stack is being shut down. All code paths go through if_detach_internal() where we, depending on the vmove flag or the vnet state, make a decision on how much to shut down; in case we are destroying a VNET the individual protocol layers will cleanup their own parts thus we cannot do so again for each interface as we end up with, e.g., double-frees, destroying locks twice or acquiring already destroyed locks. When calling into protocol cleanups we equally have to tell them whether they need to detach upper layer protocols ("ulp") or not (e.g., in6_ifdetach()). Provide or enahnce helper functions to do proper cleanup at a protocol rather than at an interface level. Approved by: re (hrs) Obtained from: projects/vnet Reviewed by: gnn, jhb Sponsored by: The FreeBSD Foundation MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D6747
2016-06-21 13:48:49 +00:00
#ifdef VIMAGE
/* Make sure the VNET is stable. */
Factor out VNET shutdown check into an own vnet structure field. Remove the now obsolete vnet_state field. This greatly simplifies the detection of VNET shutdown and avoids code duplication. Discussed with: bz@ MFC after: 1 week Sponsored by: Mellanox Technologies
2019-10-07 14:15:41 +00:00
if (so->so_vnet->vnet_shutdown) {
Get closer to a VIMAGE network stack teardown from top to bottom rather than removing the network interfaces first. This change is rather larger and convoluted as the ordering requirements cannot be separated. Move the pfil(9) framework to SI_SUB_PROTO_PFIL, move Firewalls and related modules to their own SI_SUB_PROTO_FIREWALL. Move initialization of "physical" interfaces to SI_SUB_DRIVERS, move virtual (cloned) interfaces to SI_SUB_PSEUDO. Move Multicast to SI_SUB_PROTO_MC. Re-work parts of multicast initialisation and teardown, not taking the huge amount of memory into account if used as a module yet. For interface teardown we try to do as many of them as we can on SI_SUB_INIT_IF, but for some this makes no sense, e.g., when tunnelling over a higher layer protocol such as IP. In that case the interface has to go along (or before) the higher layer protocol is shutdown. Kernel hhooks need to go last on teardown as they may be used at various higher layers and we cannot remove them before we cleaned up the higher layers. For interface teardown there are multiple paths: (a) a cloned interface is destroyed (inside a VIMAGE or in the base system), (b) any interface is moved from a virtual network stack to a different network stack ("vmove"), or (c) a virtual network stack is being shut down. All code paths go through if_detach_internal() where we, depending on the vmove flag or the vnet state, make a decision on how much to shut down; in case we are destroying a VNET the individual protocol layers will cleanup their own parts thus we cannot do so again for each interface as we end up with, e.g., double-frees, destroying locks twice or acquiring already destroyed locks. When calling into protocol cleanups we equally have to tell them whether they need to detach upper layer protocols ("ulp") or not (e.g., in6_ifdetach()). Provide or enahnce helper functions to do proper cleanup at a protocol rather than at an interface level. Approved by: re (hrs) Obtained from: projects/vnet Reviewed by: gnn, jhb Sponsored by: The FreeBSD Foundation MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D6747
2016-06-21 13:48:49 +00:00
CURVNET_RESTORE();
return (EBUSY);
}
#endif
Introduce network device nodes. Network devices will now automatically appear in /dev. Interface hardware ioctls (not protocol or routing) can be performed on the descriptor. The SIOCGIFCONF ioctl may be performed on the special /dev/network node.
2001-09-29 05:55:04 +00:00
switch (cmd) {
case SIOCGIFCONF:
Mfp4 CH=177274,177280,177284-177285,177297,177324-177325 VNET socket push back: try to minimize the number of places where we have to switch vnets and narrow down the time we stay switched. Add assertions to the socket code to catch possibly unset vnets as seen in r204147. While this reduces the number of vnet recursion in some places like NFS, POSIX local sockets and some netgraph, .. recursions are impossible to fix. The current expectations are documented at the beginning of uipc_socket.c along with the other information there. Sponsored by: The FreeBSD Foundation Sponsored by: CK Software GmbH Reviewed by: jhb Tested by: zec Tested by: Mikolaj Golub (to.my.trociny gmail.com) MFC after: 2 weeks
2011-02-16 21:29:13 +00:00
error = ifconf(cmd, data);
Widen NET_EPOCH coverage. When epoch(9) was introduced to network stack, it was basically dropped in place of existing locking, which was mutexes and rwlocks. For the sake of performance mutex covered areas were as small as possible, so became epoch covered areas. However, epoch doesn't introduce any contention, it just delays memory reclaim. So, there is no point to minimise epoch covered areas in sense of performance. Meanwhile entering/exiting epoch also has non-zero CPU usage, so doing this less often is a win. Not the least is also code maintainability. In the new paradigm we can assume that at any stage of processing a packet, we are inside network epoch. This makes coding both input and output path way easier. On output path we already enter epoch quite early - in the ip_output(), in the ip6_output(). This patch does the same for the input path. All ISR processing, network related callouts, other ways of packet injection to the network stack shall be performed in net_epoch. Any leaf function that walks network configuration now asserts epoch. Tricky part is configuration code paths - ioctls, sysctls. They also call into leaf functions, so some need to be changed. This patch would introduce more epoch recursions (see EPOCH_TRACE) than we had before. They will be cleaned up separately, as several of them aren't trivial. Note, that unlike a lock recursion the epoch recursion is safe and just wastes a bit of resources. Reviewed by: gallatin, hselasky, cy, adrian, kristof Differential Revision: https://reviews.freebsd.org/D19111
2019-10-07 22:40:05 +00:00
goto out_noref;
Reshuffle SIOCGIFCONF32 handler from r155224. - move all the chunks into one file, which allows to hide SIOCGIFCONF32 global definition as well. - replace __amd64__ with proper COMPAT_FREEBSD32 around. - handle 32bit capacity before going into the handler itself instead of doing internal 32bit specific changes within it (e.g. as it's done for SIOCGDEFIFACE32_IN6). - use explicitely sized types for ABI compat. Approved by: kib (mentor) MFC after: 2 weeks
2010-10-21 16:20:48 +00:00
#ifdef COMPAT_FREEBSD32
Implement SIOCGIFCONF for 32bit binaries.
2006-02-02 19:58:37 +00:00
case SIOCGIFCONF32:
Reshuffle SIOCGIFCONF32 handler from r155224. - move all the chunks into one file, which allows to hide SIOCGIFCONF32 global definition as well. - replace __amd64__ with proper COMPAT_FREEBSD32 around. - handle 32bit capacity before going into the handler itself instead of doing internal 32bit specific changes within it (e.g. as it's done for SIOCGDEFIFACE32_IN6). - use explicitely sized types for ABI compat. Approved by: kib (mentor) MFC after: 2 weeks
2010-10-21 16:20:48 +00:00
{
struct ifconf32 *ifc32;
struct ifconf ifc;
ifc32 = (struct ifconf32 *)data;
ifc.ifc_len = ifc32->ifc_len;
ifc.ifc_buf = PTRIN(ifc32->ifc_buf);
Mfp4 CH=177274,177280,177284-177285,177297,177324-177325 VNET socket push back: try to minimize the number of places where we have to switch vnets and narrow down the time we stay switched. Add assertions to the socket code to catch possibly unset vnets as seen in r204147. While this reduces the number of vnet recursion in some places like NFS, POSIX local sockets and some netgraph, .. recursions are impossible to fix. The current expectations are documented at the beginning of uipc_socket.c along with the other information there. Sponsored by: The FreeBSD Foundation Sponsored by: CK Software GmbH Reviewed by: jhb Tested by: zec Tested by: Mikolaj Golub (to.my.trociny gmail.com) MFC after: 2 weeks
2011-02-16 21:29:13 +00:00
error = ifconf(SIOCGIFCONF, (void *)&ifc);
Update ifc_len field of struct ifconf passed for the ioctl SIOCGIFCONF32 (i.e. under COMPAT_FREEBSD32) in case ifconf() returned success to match the native SIOCGIFCONF behavior. PR: kern/158369 Reported by: Paul Procacci <pprocacci att gmail com> MFC after: 1 week
2011-06-28 08:41:44 +00:00
if (error == 0)
ifc32->ifc_len = ifc.ifc_len;
Widen NET_EPOCH coverage. When epoch(9) was introduced to network stack, it was basically dropped in place of existing locking, which was mutexes and rwlocks. For the sake of performance mutex covered areas were as small as possible, so became epoch covered areas. However, epoch doesn't introduce any contention, it just delays memory reclaim. So, there is no point to minimise epoch covered areas in sense of performance. Meanwhile entering/exiting epoch also has non-zero CPU usage, so doing this less often is a win. Not the least is also code maintainability. In the new paradigm we can assume that at any stage of processing a packet, we are inside network epoch. This makes coding both input and output path way easier. On output path we already enter epoch quite early - in the ip_output(), in the ip6_output(). This patch does the same for the input path. All ISR processing, network related callouts, other ways of packet injection to the network stack shall be performed in net_epoch. Any leaf function that walks network configuration now asserts epoch. Tricky part is configuration code paths - ioctls, sysctls. They also call into leaf functions, so some need to be changed. This patch would introduce more epoch recursions (see EPOCH_TRACE) than we had before. They will be cleaned up separately, as several of them aren't trivial. Note, that unlike a lock recursion the epoch recursion is safe and just wastes a bit of resources. Reviewed by: gallatin, hselasky, cy, adrian, kristof Differential Revision: https://reviews.freebsd.org/D19111
2019-10-07 22:40:05 +00:00
goto out_noref;
Reshuffle SIOCGIFCONF32 handler from r155224. - move all the chunks into one file, which allows to hide SIOCGIFCONF32 global definition as well. - replace __amd64__ with proper COMPAT_FREEBSD32 around. - handle 32bit capacity before going into the handler itself instead of doing internal 32bit specific changes within it (e.g. as it's done for SIOCGDEFIFACE32_IN6). - use explicitely sized types for ABI compat. Approved by: kib (mentor) MFC after: 2 weeks
2010-10-21 16:20:48 +00:00
}
Implement SIOCGIFCONF for 32bit binaries.
2006-02-02 19:58:37 +00:00
#endif
Introduce network device nodes. Network devices will now automatically appear in /dev. Interface hardware ioctls (not protocol or routing) can be performed on the descriptor. The SIOCGIFCONF ioctl may be performed on the special /dev/network node.
2001-09-29 05:55:04 +00:00
}
Translate 32-bit ifmedia requests into native ones. We use transformation rather than accessors as virtually ever driver implements SIOCGIFMEDIA and all would have to be touched. Keep the code readable by always performing copies and (possiably no-op) transforms. Reviewed by: jhb, kib Obtained from: CheriBSD MFC after: 1 week Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D14996
2018-04-25 15:30:42 +00:00
#ifdef COMPAT_FREEBSD32
switch (cmd) {
case SIOCGIFMEDIA32:
case SIOCGIFXMEDIA32:
ifmrp = &ifmr;
ifmr_init(ifmrp, data);
cmd = _IOC_NEWTYPE(cmd, struct ifmediareq);
saved_data = data;
data = (caddr_t)ifmrp;
}
#endif
ifr = (struct ifreq *)data;
Introduce network device nodes. Network devices will now automatically appear in /dev. Interface hardware ioctls (not protocol or routing) can be performed on the descriptor. The SIOCGIFCONF ioctl may be performed on the special /dev/network node.
2001-09-29 05:55:04 +00:00
switch (cmd) {
Introduce an interm userland-kernel API for creating vnets and assigning ifnets from one vnet to another. Deletion of vnets is not yet supported. The interface is implemented as an ioctl extension so that no syscalls had to be introduced. This should be acceptable given that the new interface will be used for a short / interim period only, until the new jail management framwork gains the capability of managing vnets. This method for managing vimages / vnets has been in use for the past 7 years without any observable issues. The userland tool to be used in conjunction with the interim API can be found in p4: //depot/projects/vimage-commit2/src/usr.sbin/vimage/... and will most probably never get commited to svn. While here, bump copyright notices in kern_vimage.c and vimage.h to cover work done in year 2009. Approved by: julian (mentor) Discussed with: bz, rwatson
2009-05-31 12:10:04 +00:00
#ifdef VIMAGE
Manage vnets via the jail system. If a jail is given the boolean parameter "vnet" when it is created, a new vnet instance will be created along with the jail. Networks interfaces can be moved between prisons with an ioctl similar to the one that moves them between vimages. For now vnets will co-exist under both jails and vimages, but soon struct vimage will be going away. Reviewed by: zec, julian Approved by: bz (mentor)
2009-06-15 18:59:29 +00:00
case SIOCSIFRVNET:
error = priv_check(td, PRIV_NET_SETIFVNET);
Mfp4 CH=177274,177280,177284-177285,177297,177324-177325 VNET socket push back: try to minimize the number of places where we have to switch vnets and narrow down the time we stay switched. Add assertions to the socket code to catch possibly unset vnets as seen in r204147. While this reduces the number of vnet recursion in some places like NFS, POSIX local sockets and some netgraph, .. recursions are impossible to fix. The current expectations are documented at the beginning of uipc_socket.c along with the other information there. Sponsored by: The FreeBSD Foundation Sponsored by: CK Software GmbH Reviewed by: jhb Tested by: zec Tested by: Mikolaj Golub (to.my.trociny gmail.com) MFC after: 2 weeks
2011-02-16 21:29:13 +00:00
if (error == 0)
error = if_vmove_reclaim(td, ifr->ifr_name,
ifr->ifr_jid);
Translate 32-bit ifmedia requests into native ones. We use transformation rather than accessors as virtually ever driver implements SIOCGIFMEDIA and all would have to be touched. Keep the code readable by always performing copies and (possiably no-op) transforms. Reviewed by: jhb, kib Obtained from: CheriBSD MFC after: 1 week Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D14996
2018-04-25 15:30:42 +00:00
goto out_noref;
Introduce an interm userland-kernel API for creating vnets and assigning ifnets from one vnet to another. Deletion of vnets is not yet supported. The interface is implemented as an ioctl extension so that no syscalls had to be introduced. This should be acceptable given that the new interface will be used for a short / interim period only, until the new jail management framwork gains the capability of managing vnets. This method for managing vimages / vnets has been in use for the past 7 years without any observable issues. The userland tool to be used in conjunction with the interim API can be found in p4: //depot/projects/vimage-commit2/src/usr.sbin/vimage/... and will most probably never get commited to svn. While here, bump copyright notices in kern_vimage.c and vimage.h to cover work done in year 2009. Approved by: julian (mentor) Discussed with: bz, rwatson
2009-05-31 12:10:04 +00:00
#endif
Introduce network device nodes. Network devices will now automatically appear in /dev. Interface hardware ioctls (not protocol or routing) can be performed on the descriptor. The SIOCGIFCONF ioctl may be performed on the special /dev/network node.
2001-09-29 05:55:04 +00:00
case SIOCIFCREATE:
Revise network interface cloning to take an optional opaque parameter that can specify configuration parameters: o rev cloner api's to add optional parameter block o add SIOCCREATE2 that accepts parameter data o rev vlan support to use new api (maintain old code) Reviewed by: arch@
2006-07-09 06:04:01 +00:00
case SIOCIFCREATE2:
Sweep kernel replacing suser(9) calls with priv(9) calls, assigning specific privilege names to a broad range of privileges. These may require some future tweaking. Sponsored by: nCircle Network Security, Inc. Obtained from: TrustedBSD Project Discussed on: arch@ Reviewed (at least in part) by: mlaier, jmg, pjd, bde, ceri, Alex Lyashkov <umka at sevcity dot net>, Skip Ford <skip dot ford at verizon dot net>, Antoine Brodin <antoine dot brodin at laposte dot net>
2006-11-06 13:42:10 +00:00
error = priv_check(td, PRIV_NET_IFCREATE);
Mfp4 CH=177274,177280,177284-177285,177297,177324-177325 VNET socket push back: try to minimize the number of places where we have to switch vnets and narrow down the time we stay switched. Add assertions to the socket code to catch possibly unset vnets as seen in r204147. While this reduces the number of vnet recursion in some places like NFS, POSIX local sockets and some netgraph, .. recursions are impossible to fix. The current expectations are documented at the beginning of uipc_socket.c along with the other information there. Sponsored by: The FreeBSD Foundation Sponsored by: CK Software GmbH Reviewed by: jhb Tested by: zec Tested by: Mikolaj Golub (to.my.trociny gmail.com) MFC after: 2 weeks
2011-02-16 21:29:13 +00:00
if (error == 0)
error = if_clone_create(ifr->ifr_name,
Use an accessor function to access ifr_data. This fixes 32-bit compat (no ioctl command defintions are required as struct ifreq is the same size). This is believed to be sufficent to fully support ifconfig on 32-bit systems. Reviewed by: kib Obtained from: CheriBSD MFC after: 1 week Relnotes: yes Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D14900
2018-03-30 18:50:13 +00:00
sizeof(ifr->ifr_name), cmd == SIOCIFCREATE2 ?
ifr_data_get_ptr(ifr) : NULL);
Translate 32-bit ifmedia requests into native ones. We use transformation rather than accessors as virtually ever driver implements SIOCGIFMEDIA and all would have to be touched. Keep the code readable by always performing copies and (possiably no-op) transforms. Reviewed by: jhb, kib Obtained from: CheriBSD MFC after: 1 week Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D14996
2018-04-25 15:30:42 +00:00
goto out_noref;
Introduce network device nodes. Network devices will now automatically appear in /dev. Interface hardware ioctls (not protocol or routing) can be performed on the descriptor. The SIOCGIFCONF ioctl may be performed on the special /dev/network node.
2001-09-29 05:55:04 +00:00
case SIOCIFDESTROY:
Sweep kernel replacing suser(9) calls with priv(9) calls, assigning specific privilege names to a broad range of privileges. These may require some future tweaking. Sponsored by: nCircle Network Security, Inc. Obtained from: TrustedBSD Project Discussed on: arch@ Reviewed (at least in part) by: mlaier, jmg, pjd, bde, ceri, Alex Lyashkov <umka at sevcity dot net>, Skip Ford <skip dot ford at verizon dot net>, Antoine Brodin <antoine dot brodin at laposte dot net>
2006-11-06 13:42:10 +00:00
error = priv_check(td, PRIV_NET_IFDESTROY);
Mfp4 CH=177274,177280,177284-177285,177297,177324-177325 VNET socket push back: try to minimize the number of places where we have to switch vnets and narrow down the time we stay switched. Add assertions to the socket code to catch possibly unset vnets as seen in r204147. While this reduces the number of vnet recursion in some places like NFS, POSIX local sockets and some netgraph, .. recursions are impossible to fix. The current expectations are documented at the beginning of uipc_socket.c along with the other information there. Sponsored by: The FreeBSD Foundation Sponsored by: CK Software GmbH Reviewed by: jhb Tested by: zec Tested by: Mikolaj Golub (to.my.trociny gmail.com) MFC after: 2 weeks
2011-02-16 21:29:13 +00:00
if (error == 0)
error = if_clone_destroy(ifr->ifr_name);
Translate 32-bit ifmedia requests into native ones. We use transformation rather than accessors as virtually ever driver implements SIOCGIFMEDIA and all would have to be touched. Keep the code readable by always performing copies and (possiably no-op) transforms. Reviewed by: jhb, kib Obtained from: CheriBSD MFC after: 1 week Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D14996
2018-04-25 15:30:42 +00:00
goto out_noref;
Clean up whitespace, remove "register" keyword, ANSIfy. No functional changes.
2003-10-23 13:49:10 +00:00
Introduce network device nodes. Network devices will now automatically appear in /dev. Interface hardware ioctls (not protocol or routing) can be performed on the descriptor. The SIOCGIFCONF ioctl may be performed on the special /dev/network node.
2001-09-29 05:55:04 +00:00
case SIOCIFGCLONERS:
Mfp4 CH=177274,177280,177284-177285,177297,177324-177325 VNET socket push back: try to minimize the number of places where we have to switch vnets and narrow down the time we stay switched. Add assertions to the socket code to catch possibly unset vnets as seen in r204147. While this reduces the number of vnet recursion in some places like NFS, POSIX local sockets and some netgraph, .. recursions are impossible to fix. The current expectations are documented at the beginning of uipc_socket.c along with the other information there. Sponsored by: The FreeBSD Foundation Sponsored by: CK Software GmbH Reviewed by: jhb Tested by: zec Tested by: Mikolaj Golub (to.my.trociny gmail.com) MFC after: 2 weeks
2011-02-16 21:29:13 +00:00
error = if_clone_list((struct if_clonereq *)data);
Translate 32-bit ifmedia requests into native ones. We use transformation rather than accessors as virtually ever driver implements SIOCGIFMEDIA and all would have to be touched. Keep the code readable by always performing copies and (possiably no-op) transforms. Reviewed by: jhb, kib Obtained from: CheriBSD MFC after: 1 week Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D14996
2018-04-25 15:30:42 +00:00
goto out_noref;
Rework CASE_IOC_IFGROUPREQ() to require a case before the macro. This is more compatible with formatting tools and looks more normal. Reported by: jhb (on a different review) Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D18442
2019-01-22 17:39:26 +00:00
case CASE_IOC_IFGROUPREQ(SIOCGIFGMEMB):
Mfp4 CH=177274,177280,177284-177285,177297,177324-177325 VNET socket push back: try to minimize the number of places where we have to switch vnets and narrow down the time we stay switched. Add assertions to the socket code to catch possibly unset vnets as seen in r204147. While this reduces the number of vnet recursion in some places like NFS, POSIX local sockets and some netgraph, .. recursions are impossible to fix. The current expectations are documented at the beginning of uipc_socket.c along with the other information there. Sponsored by: The FreeBSD Foundation Sponsored by: CK Software GmbH Reviewed by: jhb Tested by: zec Tested by: Mikolaj Golub (to.my.trociny gmail.com) MFC after: 2 weeks
2011-02-16 21:29:13 +00:00
error = if_getgroupmembers((struct ifgroupreq *)data);
Translate 32-bit ifmedia requests into native ones. We use transformation rather than accessors as virtually ever driver implements SIOCGIFMEDIA and all would have to be touched. Keep the code readable by always performing copies and (possiably no-op) transforms. Reviewed by: jhb, kib Obtained from: CheriBSD MFC after: 1 week Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D14996
2018-04-25 15:30:42 +00:00
goto out_noref;
A major overhaul of the CARP implementation. The ip_carp.c was started from scratch, copying needed functionality from the old implemenation on demand, with a thorough review of all code. The main change is that interface layer has been removed from the CARP. Now redundant addresses are configured exactly on the interfaces, they run on. The CARP configuration itself is, as before, configured and read via SIOCSVH/SIOCGVH ioctls. A new prefix created with SIOCAIFADDR or SIOCAIFADDR_IN6 may now be configured to a particular virtual host id, which makes the prefix redundant. ifconfig(8) semantics has been changed too: now one doesn't need to clone carpXX interface, he/she should directly configure a vhid on a Ethernet interface. To supply vhid data from the kernel to an application the getifaddrs(8) function had been changed to pass ifam_data with each address. [1] The new implementation definitely closes all PRs related to carp(4) being an interface, and may close several others. It also allows to run a single redundant IP per interface. Big thanks to Bjoern Zeeb for his help with inet6 part of patch, for idea on using ifam_data and for several rounds of reviewing! PR: kern/117000, kern/126945, kern/126714, kern/120130, kern/117448 Reviewed by: bz Submitted by: bz [1]
2011-12-16 12:16:56 +00:00
#if defined(INET) || defined(INET6)
case SIOCSVH:
case SIOCGVH:
if (carp_ioctl_p == NULL)
error = EPROTONOSUPPORT;
else
error = (*carp_ioctl_p)(ifr, cmd, td);
Translate 32-bit ifmedia requests into native ones. We use transformation rather than accessors as virtually ever driver implements SIOCGIFMEDIA and all would have to be touched. Keep the code readable by always performing copies and (possiably no-op) transforms. Reviewed by: jhb, kib Obtained from: CheriBSD MFC after: 1 week Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D14996
2018-04-25 15:30:42 +00:00
goto out_noref;
A major overhaul of the CARP implementation. The ip_carp.c was started from scratch, copying needed functionality from the old implemenation on demand, with a thorough review of all code. The main change is that interface layer has been removed from the CARP. Now redundant addresses are configured exactly on the interfaces, they run on. The CARP configuration itself is, as before, configured and read via SIOCSVH/SIOCGVH ioctls. A new prefix created with SIOCAIFADDR or SIOCAIFADDR_IN6 may now be configured to a particular virtual host id, which makes the prefix redundant. ifconfig(8) semantics has been changed too: now one doesn't need to clone carpXX interface, he/she should directly configure a vhid on a Ethernet interface. To supply vhid data from the kernel to an application the getifaddrs(8) function had been changed to pass ifam_data with each address. [1] The new implementation definitely closes all PRs related to carp(4) being an interface, and may close several others. It also allows to run a single redundant IP per interface. Big thanks to Bjoern Zeeb for his help with inet6 part of patch, for idea on using ifam_data and for several rounds of reviewing! PR: kern/117000, kern/126945, kern/126714, kern/120130, kern/117448 Reviewed by: bz Submitted by: bz [1]
2011-12-16 12:16:56 +00:00
#endif
Introduce network device nodes. Network devices will now automatically appear in /dev. Interface hardware ioctls (not protocol or routing) can be performed on the descriptor. The SIOCGIFCONF ioctl may be performed on the special /dev/network node.
2001-09-29 05:55:04 +00:00
}
Add ifunit_ref(), a version of ifunit(), that returns not just an interface pointer, but also a reference to it. Modify ifioctl() to use ifunit_ref(), holding the reference until all ioctls, etc, have completed. This closes a class of reader-writer races in which interfaces could be removed during long-running ioctls, leading to crashes. Many other consumers of ifunit() should now use ifunit_ref() to avoid similar races. MFC after: 3 weeks
2009-04-23 13:08:47 +00:00
ifp = ifunit_ref(ifr->ifr_name);
Mfp4 CH=177274,177280,177284-177285,177297,177324-177325 VNET socket push back: try to minimize the number of places where we have to switch vnets and narrow down the time we stay switched. Add assertions to the socket code to catch possibly unset vnets as seen in r204147. While this reduces the number of vnet recursion in some places like NFS, POSIX local sockets and some netgraph, .. recursions are impossible to fix. The current expectations are documented at the beginning of uipc_socket.c along with the other information there. Sponsored by: The FreeBSD Foundation Sponsored by: CK Software GmbH Reviewed by: jhb Tested by: zec Tested by: Mikolaj Golub (to.my.trociny gmail.com) MFC after: 2 weeks
2011-02-16 21:29:13 +00:00
if (ifp == NULL) {
Translate 32-bit ifmedia requests into native ones. We use transformation rather than accessors as virtually ever driver implements SIOCGIFMEDIA and all would have to be touched. Keep the code readable by always performing copies and (possiably no-op) transforms. Reviewed by: jhb, kib Obtained from: CheriBSD MFC after: 1 week Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D14996
2018-04-25 15:30:42 +00:00
error = ENXIO;
goto out_noref;
Mfp4 CH=177274,177280,177284-177285,177297,177324-177325 VNET socket push back: try to minimize the number of places where we have to switch vnets and narrow down the time we stay switched. Add assertions to the socket code to catch possibly unset vnets as seen in r204147. While this reduces the number of vnet recursion in some places like NFS, POSIX local sockets and some netgraph, .. recursions are impossible to fix. The current expectations are documented at the beginning of uipc_socket.c along with the other information there. Sponsored by: The FreeBSD Foundation Sponsored by: CK Software GmbH Reviewed by: jhb Tested by: zec Tested by: Mikolaj Golub (to.my.trociny gmail.com) MFC after: 2 weeks
2011-02-16 21:29:13 +00:00
}
Introduce network device nodes. Network devices will now automatically appear in /dev. Interface hardware ioctls (not protocol or routing) can be performed on the descriptor. The SIOCGIFCONF ioctl may be performed on the special /dev/network node.
2001-09-29 05:55:04 +00:00
error = ifhwioctl(cmd, ifp, data, td);
Translate 32-bit ifmedia requests into native ones. We use transformation rather than accessors as virtually ever driver implements SIOCGIFMEDIA and all would have to be touched. Keep the code readable by always performing copies and (possiably no-op) transforms. Reviewed by: jhb, kib Obtained from: CheriBSD MFC after: 1 week Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D14996
2018-04-25 15:30:42 +00:00
if (error != ENOIOCTL)
goto out_ref;
Introduce network device nodes. Network devices will now automatically appear in /dev. Interface hardware ioctls (not protocol or routing) can be performed on the descriptor. The SIOCGIFCONF ioctl may be performed on the special /dev/network node.
2001-09-29 05:55:04 +00:00
oif_flags = ifp->if_flags;
Add ifunit_ref(), a version of ifunit(), that returns not just an interface pointer, but also a reference to it. Modify ifioctl() to use ifunit_ref(), holding the reference until all ioctls, etc, have completed. This closes a class of reader-writer races in which interfaces could be removed during long-running ioctls, leading to crashes. Many other consumers of ifunit() should now use ifunit_ref() to avoid similar races. MFC after: 3 weeks
2009-04-23 13:08:47 +00:00
if (so->so_proto == NULL) {
Translate 32-bit ifmedia requests into native ones. We use transformation rather than accessors as virtually ever driver implements SIOCGIFMEDIA and all would have to be touched. Keep the code readable by always performing copies and (possiably no-op) transforms. Reviewed by: jhb, kib Obtained from: CheriBSD MFC after: 1 week Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D14996
2018-04-25 15:30:42 +00:00
error = EOPNOTSUPP;
goto out_ref;
Add ifunit_ref(), a version of ifunit(), that returns not just an interface pointer, but also a reference to it. Modify ifioctl() to use ifunit_ref(), holding the reference until all ioctls, etc, have completed. This closes a class of reader-writer races in which interfaces could be removed during long-running ioctls, leading to crashes. Many other consumers of ifunit() should now use ifunit_ref() to avoid similar races. MFC after: 3 weeks
2009-04-23 13:08:47 +00:00
}
Fix the length calculation for the final block of a sendfile(2) transmission which could be tricked into rounding up to the nearest page size, leaking up to a page of kernel memory. [13:11] In IPv6 and NetATM, stop SIOCSIFADDR, SIOCSIFBRDADDR, SIOCSIFDSTADDR and SIOCSIFNETMASK at the socket layer rather than pass them on to the link layer without validation or credential checks. [SA-13:12] Prevent cross-mount hardlinks between different nullfs mounts of the same underlying filesystem. [SA-13:13] Security: CVE-2013-5666 Security: FreeBSD-SA-13:11.sendfile Security: CVE-2013-5691 Security: FreeBSD-SA-13:12.ifioctl Security: CVE-2013-5710 Security: FreeBSD-SA-13:13.nullfs Approved by: re
2013-09-10 10:05:59 +00:00
/*
* Pass the request on to the socket control method, and if the
* latter returns EOPNOTSUPP, directly to the interface.
*
* Make an exception for the legacy SIOCSIF* requests. Drivers
* trust SIOCSIFADDR et al to come from an already privileged
* layer, and do not perform any credentials checks or input
* validation.
*/
Drop support for historic ioctls and also undefine them, so that code that checks their presence via ifdef, won't use them. Bump __FreeBSD_version as safety measure.
2013-11-05 10:29:47 +00:00
error = ((*so->so_proto->pr_usrreqs->pru_control)(so, cmd, data,
ifp, td));
Fix the length calculation for the final block of a sendfile(2) transmission which could be tricked into rounding up to the nearest page size, leaking up to a page of kernel memory. [13:11] In IPv6 and NetATM, stop SIOCSIFADDR, SIOCSIFBRDADDR, SIOCSIFDSTADDR and SIOCSIFNETMASK at the socket layer rather than pass them on to the link layer without validation or credential checks. [SA-13:12] Prevent cross-mount hardlinks between different nullfs mounts of the same underlying filesystem. [SA-13:13] Security: CVE-2013-5666 Security: FreeBSD-SA-13:11.sendfile Security: CVE-2013-5691 Security: FreeBSD-SA-13:12.ifioctl Security: CVE-2013-5710 Security: FreeBSD-SA-13:13.nullfs Approved by: re
2013-09-10 10:05:59 +00:00
if (error == EOPNOTSUPP && ifp != NULL && ifp->if_ioctl != NULL &&
cmd != SIOCSIFADDR && cmd != SIOCSIFBRDADDR &&
cmd != SIOCSIFDSTADDR && cmd != SIOCSIFNETMASK)
Call the interface's if_ioctl from ifioctl(), if the protocol didn't handle the ioctl. There are other paths that already call it, but this allows for a non-interface socket (like AF_LOCAL which ifconfig now uses) to use a broader class of interface ioctls. Approved by: bz (mentor), rwatson
2009-03-20 13:41:23 +00:00
error = (*ifp->if_ioctl)(ifp, cmd, data);
KAME netinet6 basic part(no IPsec,no V6 Multicast Forwarding, no UDP/TCP for IPv6 yet) With this patch, you can assigne IPv6 addr automatically, and can reply to IPv6 ping. Reviewed by: freebsd-arch, cvs-committers Obtained from: KAME project
1999-11-22 02:45:11 +00:00
Introduce network device nodes. Network devices will now automatically appear in /dev. Interface hardware ioctls (not protocol or routing) can be performed on the descriptor. The SIOCGIFCONF ioctl may be performed on the special /dev/network node.
2001-09-29 05:55:04 +00:00
if ((oif_flags ^ ifp->if_flags) & IFF_UP) {
KAME netinet6 basic part(no IPsec,no V6 Multicast Forwarding, no UDP/TCP for IPv6 yet) With this patch, you can assigne IPv6 addr automatically, and can reply to IPv6 ping. Reviewed by: freebsd-arch, cvs-committers Obtained from: KAME project
1999-11-22 02:45:11 +00:00
#ifdef INET6
Mechanically remove the last stray remains of spl* calls from net*/*. They have been Noop's for a long time now.
2012-10-18 13:57:24 +00:00
if (ifp->if_flags & IFF_UP)
Introduce network device nodes. Network devices will now automatically appear in /dev. Interface hardware ioctls (not protocol or routing) can be performed on the descriptor. The SIOCGIFCONF ioctl may be performed on the special /dev/network node.
2001-09-29 05:55:04 +00:00
in6_if_up(ifp);
#endif
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
}
Translate 32-bit ifmedia requests into native ones. We use transformation rather than accessors as virtually ever driver implements SIOCGIFMEDIA and all would have to be touched. Keep the code readable by always performing copies and (possiably no-op) transforms. Reviewed by: jhb, kib Obtained from: CheriBSD MFC after: 1 week Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D14996
2018-04-25 15:30:42 +00:00
out_ref:
Add ifunit_ref(), a version of ifunit(), that returns not just an interface pointer, but also a reference to it. Modify ifioctl() to use ifunit_ref(), holding the reference until all ioctls, etc, have completed. This closes a class of reader-writer races in which interfaces could be removed during long-running ioctls, leading to crashes. Many other consumers of ifunit() should now use ifunit_ref() to avoid similar races. MFC after: 3 weeks
2009-04-23 13:08:47 +00:00
if_rele(ifp);
Translate 32-bit ifmedia requests into native ones. We use transformation rather than accessors as virtually ever driver implements SIOCGIFMEDIA and all would have to be touched. Keep the code readable by always performing copies and (possiably no-op) transforms. Reviewed by: jhb, kib Obtained from: CheriBSD MFC after: 1 week Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D14996
2018-04-25 15:30:42 +00:00
out_noref:
#ifdef COMPAT_FREEBSD32
if (ifmrp != NULL) {
KASSERT((cmd == SIOCGIFMEDIA || cmd == SIOCGIFXMEDIA),
("ifmrp non-NULL, but cmd is not an ifmedia req 0x%lx",
cmd));
data = saved_data;
ifmr_update(ifmrp, data);
}
#endif
Mfp4 CH=177274,177280,177284-177285,177297,177324-177325 VNET socket push back: try to minimize the number of places where we have to switch vnets and narrow down the time we stay switched. Add assertions to the socket code to catch possibly unset vnets as seen in r204147. While this reduces the number of vnet recursion in some places like NFS, POSIX local sockets and some netgraph, .. recursions are impossible to fix. The current expectations are documented at the beginning of uipc_socket.c along with the other information there. Sponsored by: The FreeBSD Foundation Sponsored by: CK Software GmbH Reviewed by: jhb Tested by: zec Tested by: Mikolaj Golub (to.my.trociny gmail.com) MFC after: 2 weeks
2011-02-16 21:29:13 +00:00
CURVNET_RESTORE();
Introduce network device nodes. Network devices will now automatically appear in /dev. Interface hardware ioctls (not protocol or routing) can be performed on the descriptor. The SIOCGIFCONF ioctl may be performed on the special /dev/network node.
2001-09-29 05:55:04 +00:00
return (error);
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
}
Fix BPf to generate a header mbuf for writes. Fix loopback and discard interfaces to understand BPF writes. (These two from Bill Fenner to fix PR 512.) Move ifpromisc() from bpf.c to if.c as suggested by comment in BPF. Send a notice to the log when promiscuous mode is enabled.
1995-09-22 17:57:48 +00:00
/*
Rename IFF_RUNNING to IFF_DRV_RUNNING, IFF_OACTIVE to IFF_DRV_OACTIVE, and move both flags from ifnet.if_flags to ifnet.if_drv_flags, making and documenting the locking of these flags the responsibility of the device driver, not the network stack. The flags for these two fields will be mutually exclusive so that they can be exposed to user space as though they were stored in the same variable. Provide #defines to provide the old names #ifndef _KERNEL, so that user applications (such as ifconfig) can use the old flag names. Using the old names in a device driver will result in a compile error in order to help device driver writers adopt the new model. When exposing the interface flags to user space, via interface ioctls or routing sockets, or the two fields together. Since the driver flags cannot currently be set for user space, no new logic is currently required to handle this case. Add some assertions that general purpose network stack routines, such as if_setflags(), are not improperly used on driver-owned flags. With this change, a large number of very minor network stack races are closed, subject to correct device driver locking. Most were likely never triggered. Driver sweep to follow; many thanks to pjd and bz for the line-by-line review they gave this patch. Reviewed by: pjd, bz MFC after: 7 days
2005-08-09 10:16:17 +00:00
* The code common to handling reference counted flags,
MFp4: - Introduce a helper function if_setflag() containing the code common to ifpromisc() and if_allmulti() instead of duplicating the code poorly, with different bugs. - Call ifp->if_ioctl() in a consistent way: always use more compatible C syntax and check whether ifp->if_ioctl is not NULL prior to the call. MFC after: 1 month
2005-07-14 13:56:51 +00:00
* e.g., in ifpromisc() and if_allmulti().
Clean up consistency checks in if_setflag(): . use KASSERT for all checks so that the source of an error can be detected; . use __func__ instead of spelling function name each time; . fix a typo.
2005-10-03 02:14:51 +00:00
* The "pflag" argument can specify a permanent mode flag to check,
MFp4: - Introduce a helper function if_setflag() containing the code common to ifpromisc() and if_allmulti() instead of duplicating the code poorly, with different bugs. - Call ifp->if_ioctl() in a consistent way: always use more compatible C syntax and check whether ifp->if_ioctl is not NULL prior to the call. MFC after: 1 month
2005-07-14 13:56:51 +00:00
* such as IFF_PPROMISC for promiscuous mode; should be 0 if none.
Rename IFF_RUNNING to IFF_DRV_RUNNING, IFF_OACTIVE to IFF_DRV_OACTIVE, and move both flags from ifnet.if_flags to ifnet.if_drv_flags, making and documenting the locking of these flags the responsibility of the device driver, not the network stack. The flags for these two fields will be mutually exclusive so that they can be exposed to user space as though they were stored in the same variable. Provide #defines to provide the old names #ifndef _KERNEL, so that user applications (such as ifconfig) can use the old flag names. Using the old names in a device driver will result in a compile error in order to help device driver writers adopt the new model. When exposing the interface flags to user space, via interface ioctls or routing sockets, or the two fields together. Since the driver flags cannot currently be set for user space, no new logic is currently required to handle this case. Add some assertions that general purpose network stack routines, such as if_setflags(), are not improperly used on driver-owned flags. With this change, a large number of very minor network stack races are closed, subject to correct device driver locking. Most were likely never triggered. Driver sweep to follow; many thanks to pjd and bz for the line-by-line review they gave this patch. Reviewed by: pjd, bz MFC after: 7 days
2005-08-09 10:16:17 +00:00
*
* Only to be used on stack-owned flags, not driver-owned flags.
Fix BPf to generate a header mbuf for writes. Fix loopback and discard interfaces to understand BPF writes. (These two from Bill Fenner to fix PR 512.) Move ifpromisc() from bpf.c to if.c as suggested by comment in BPF. Send a notice to the log when promiscuous mode is enabled.
1995-09-22 17:57:48 +00:00
*/
MFp4: - Introduce a helper function if_setflag() containing the code common to ifpromisc() and if_allmulti() instead of duplicating the code poorly, with different bugs. - Call ifp->if_ioctl() in a consistent way: always use more compatible C syntax and check whether ifp->if_ioctl is not NULL prior to the call. MFC after: 1 month
2005-07-14 13:56:51 +00:00
static int
if_setflag(struct ifnet *ifp, int flag, int pflag, int *refcount, int onswitch)
Fix BPf to generate a header mbuf for writes. Fix loopback and discard interfaces to understand BPF writes. (These two from Bill Fenner to fix PR 512.) Move ifpromisc() from bpf.c to if.c as suggested by comment in BPF. Send a notice to the log when promiscuous mode is enabled.
1995-09-22 17:57:48 +00:00
{
struct ifreq ifr;
Send RTM_IFINFO messages whenever promiscuous and all-multicast modes are enabled or disabled.
1997-02-14 15:30:54 +00:00
int error;
MFp4: - Introduce a helper function if_setflag() containing the code common to ifpromisc() and if_allmulti() instead of duplicating the code poorly, with different bugs. - Call ifp->if_ioctl() in a consistent way: always use more compatible C syntax and check whether ifp->if_ioctl is not NULL prior to the call. MFC after: 1 month
2005-07-14 13:56:51 +00:00
int oldflags, oldcount;
Fix BPf to generate a header mbuf for writes. Fix loopback and discard interfaces to understand BPF writes. (These two from Bill Fenner to fix PR 512.) Move ifpromisc() from bpf.c to if.c as suggested by comment in BPF. Send a notice to the log when promiscuous mode is enabled.
1995-09-22 17:57:48 +00:00
Clean up consistency checks in if_setflag(): . use KASSERT for all checks so that the source of an error can be detected; . use __func__ instead of spelling function name each time; . fix a typo.
2005-10-03 02:14:51 +00:00
/* Sanity checks to catch programming errors */
Rename IFF_RUNNING to IFF_DRV_RUNNING, IFF_OACTIVE to IFF_DRV_OACTIVE, and move both flags from ifnet.if_flags to ifnet.if_drv_flags, making and documenting the locking of these flags the responsibility of the device driver, not the network stack. The flags for these two fields will be mutually exclusive so that they can be exposed to user space as though they were stored in the same variable. Provide #defines to provide the old names #ifndef _KERNEL, so that user applications (such as ifconfig) can use the old flag names. Using the old names in a device driver will result in a compile error in order to help device driver writers adopt the new model. When exposing the interface flags to user space, via interface ioctls or routing sockets, or the two fields together. Since the driver flags cannot currently be set for user space, no new logic is currently required to handle this case. Add some assertions that general purpose network stack routines, such as if_setflags(), are not improperly used on driver-owned flags. With this change, a large number of very minor network stack races are closed, subject to correct device driver locking. Most were likely never triggered. Driver sweep to follow; many thanks to pjd and bz for the line-by-line review they gave this patch. Reviewed by: pjd, bz MFC after: 7 days
2005-08-09 10:16:17 +00:00
KASSERT((flag & (IFF_DRV_OACTIVE|IFF_DRV_RUNNING)) == 0,
Clean up consistency checks in if_setflag(): . use KASSERT for all checks so that the source of an error can be detected; . use __func__ instead of spelling function name each time; . fix a typo.
2005-10-03 02:14:51 +00:00
("%s: setting driver-owned flag %d", __func__, flag));
Rename IFF_RUNNING to IFF_DRV_RUNNING, IFF_OACTIVE to IFF_DRV_OACTIVE, and move both flags from ifnet.if_flags to ifnet.if_drv_flags, making and documenting the locking of these flags the responsibility of the device driver, not the network stack. The flags for these two fields will be mutually exclusive so that they can be exposed to user space as though they were stored in the same variable. Provide #defines to provide the old names #ifndef _KERNEL, so that user applications (such as ifconfig) can use the old flag names. Using the old names in a device driver will result in a compile error in order to help device driver writers adopt the new model. When exposing the interface flags to user space, via interface ioctls or routing sockets, or the two fields together. Since the driver flags cannot currently be set for user space, no new logic is currently required to handle this case. Add some assertions that general purpose network stack routines, such as if_setflags(), are not improperly used on driver-owned flags. With this change, a large number of very minor network stack races are closed, subject to correct device driver locking. Most were likely never triggered. Driver sweep to follow; many thanks to pjd and bz for the line-by-line review they gave this patch. Reviewed by: pjd, bz MFC after: 7 days
2005-08-09 10:16:17 +00:00
Clean up consistency checks in if_setflag(): . use KASSERT for all checks so that the source of an error can be detected; . use __func__ instead of spelling function name each time; . fix a typo.
2005-10-03 02:14:51 +00:00
if (onswitch)
KASSERT(*refcount >= 0,
("%s: increment negative refcount %d for flag %d",
__func__, *refcount, flag));
else
KASSERT(*refcount > 0,
("%s: decrement non-positive refcount %d for flag %d",
__func__, *refcount, flag));
MFp4: - Introduce a helper function if_setflag() containing the code common to ifpromisc() and if_allmulti() instead of duplicating the code poorly, with different bugs. - Call ifp->if_ioctl() in a consistent way: always use more compatible C syntax and check whether ifp->if_ioctl is not NULL prior to the call. MFC after: 1 month
2005-07-14 13:56:51 +00:00
/* In case this mode is permanent, just touch refcount */
if (ifp->if_flags & pflag) {
*refcount += onswitch ? 1 : -1;
Implement user-setable promiscuous mode (a new `promisc' flag for ifconfig(8)). Also, for all interfaces in this mode pass all ethernet frames to upper layer, even those not addressed to our own MAC, which allows packets encapsulated in those frames be processed with packet filters (ipfw(8) et al). Emphatically requested by: Anton Turygin <pa3op@ukr-link.net> Valuable suggestions by: fenner
2002-08-19 15:16:38 +00:00
return (0);
}
MFp4: - Introduce a helper function if_setflag() containing the code common to ifpromisc() and if_allmulti() instead of duplicating the code poorly, with different bugs. - Call ifp->if_ioctl() in a consistent way: always use more compatible C syntax and check whether ifp->if_ioctl is not NULL prior to the call. MFC after: 1 month
2005-07-14 13:56:51 +00:00
/* Save ifnet parameters for if_ioctl() may fail */
oldcount = *refcount;
oldflags = ifp->if_flags;
/*
* See if we aren't the only and touching refcount is enough.
* Actually toggle interface flag if we are the first or last.
*/
if (onswitch) {
if ((*refcount)++)
Fix BPf to generate a header mbuf for writes. Fix loopback and discard interfaces to understand BPF writes. (These two from Bill Fenner to fix PR 512.) Move ifpromisc() from bpf.c to if.c as suggested by comment in BPF. Send a notice to the log when promiscuous mode is enabled.
1995-09-22 17:57:48 +00:00
return (0);
MFp4: - Introduce a helper function if_setflag() containing the code common to ifpromisc() and if_allmulti() instead of duplicating the code poorly, with different bugs. - Call ifp->if_ioctl() in a consistent way: always use more compatible C syntax and check whether ifp->if_ioctl is not NULL prior to the call. MFC after: 1 month
2005-07-14 13:56:51 +00:00
ifp->if_flags |= flag;
Fix BPf to generate a header mbuf for writes. Fix loopback and discard interfaces to understand BPF writes. (These two from Bill Fenner to fix PR 512.) Move ifpromisc() from bpf.c to if.c as suggested by comment in BPF. Send a notice to the log when promiscuous mode is enabled.
1995-09-22 17:57:48 +00:00
} else {
MFp4: - Introduce a helper function if_setflag() containing the code common to ifpromisc() and if_allmulti() instead of duplicating the code poorly, with different bugs. - Call ifp->if_ioctl() in a consistent way: always use more compatible C syntax and check whether ifp->if_ioctl is not NULL prior to the call. MFC after: 1 month
2005-07-14 13:56:51 +00:00
if (--(*refcount))
Fix BPf to generate a header mbuf for writes. Fix loopback and discard interfaces to understand BPF writes. (These two from Bill Fenner to fix PR 512.) Move ifpromisc() from bpf.c to if.c as suggested by comment in BPF. Send a notice to the log when promiscuous mode is enabled.
1995-09-22 17:57:48 +00:00
return (0);
MFp4: - Introduce a helper function if_setflag() containing the code common to ifpromisc() and if_allmulti() instead of duplicating the code poorly, with different bugs. - Call ifp->if_ioctl() in a consistent way: always use more compatible C syntax and check whether ifp->if_ioctl is not NULL prior to the call. MFC after: 1 month
2005-07-14 13:56:51 +00:00
ifp->if_flags &= ~flag;
}
/* Call down the driver since we've changed interface flags */
if (ifp->if_ioctl == NULL) {
error = EOPNOTSUPP;
goto recover;
Fix BPf to generate a header mbuf for writes. Fix loopback and discard interfaces to understand BPF writes. (These two from Bill Fenner to fix PR 512.) Move ifpromisc() from bpf.c to if.c as suggested by comment in BPF. Send a notice to the log when promiscuous mode is enabled.
1995-09-22 17:57:48 +00:00
}
Increase size of ifnet.if_flags from 16 bits (short) to 32 bits (int). To avoid breaking application ABI use unused ifreq.ifru_flags[1] for upper 16 bits in SIOCSIFFLAGS and SIOCGIFFLAGS ioctl's. Reviewed by: -hackers, -net
2002-08-18 07:05:00 +00:00
ifr.ifr_flags = ifp->if_flags & 0xffff;
ifr.ifr_flagshigh = ifp->if_flags >> 16;
Send RTM_IFINFO messages whenever promiscuous and all-multicast modes are enabled or disabled.
1997-02-14 15:30:54 +00:00
error = (*ifp->if_ioctl)(ifp, SIOCSIFFLAGS, (caddr_t)&ifr);
MFp4: - Introduce a helper function if_setflag() containing the code common to ifpromisc() and if_allmulti() instead of duplicating the code poorly, with different bugs. - Call ifp->if_ioctl() in a consistent way: always use more compatible C syntax and check whether ifp->if_ioctl is not NULL prior to the call. MFC after: 1 month
2005-07-14 13:56:51 +00:00
if (error)
goto recover;
/* Notify userland that interface flags have changed */
rt_ifmsg(ifp);
return (0);
recover:
/* Recover after driver error */
*refcount = oldcount;
ifp->if_flags = oldflags;
return (error);
}
/*
* Set/clear promiscuous mode on interface ifp based on the truth value
* of pswitch. The calls are reference counted so that only the first
* "on" request actually has an effect, as does the final "off" request.
* Results are undefined if the "off" and "on" requests are not matched.
*/
int
ifpromisc(struct ifnet *ifp, int pswitch)
{
int error;
int oldflags = ifp->if_flags;
error = if_setflag(ifp, IFF_PROMISC, IFF_PPROMISC,
&ifp->if_pcount, pswitch);
/* If promiscuous mode status has changed, log a message */
Allow silencing of 'promiscuous mode enabled/disabled' messages. PR: 166255 Submitted by: eugen.grosbein.net Obtained from: eugen.grosbein.net MFC after: 1 week
2016-05-12 19:42:13 +00:00
if (error == 0 && ((ifp->if_flags ^ oldflags) & IFF_PROMISC) &&
log_promisc_mode_change)
Slight cleanup of interface event logging. Make if_printf() use vlog() instead of vprintf(). This means it can no longer return the number of characters printed, as it used to, but every single call to if_printf() in the entire kernel ignores the return value anyway; just return 0 so we don't have to change the prototype. Consistently use if_printf() throughout sys/net/if.c, instead of a mixture of if_printf() and log(). In ifa_maintain_loopback_route(), don't needlessly log an error if we either failed to add a route because it already existed or failed to remove one because it did not. We still return an error code, though. MFC after: 1 week
2018-05-11 00:19:49 +00:00
if_printf(ifp, "promiscuous mode %s\n",
Better handling of ioctl(SIOCSIFFLAGS) failing in ifpromisc(): - Don't print the "promiscuous mode (enabled|disabled)" on failure - Restore the reference count on failure
2001-04-27 22:20:22 +00:00
(ifp->if_flags & IFF_PROMISC) ? "enabled" : "disabled");
MFp4: - Introduce a helper function if_setflag() containing the code common to ifpromisc() and if_allmulti() instead of duplicating the code poorly, with different bugs. - Call ifp->if_ioctl() in a consistent way: always use more compatible C syntax and check whether ifp->if_ioctl is not NULL prior to the call. MFC after: 1 month
2005-07-14 13:56:51 +00:00
return (error);
Fix BPf to generate a header mbuf for writes. Fix loopback and discard interfaces to understand BPF writes. (These two from Bill Fenner to fix PR 512.) Move ifpromisc() from bpf.c to if.c as suggested by comment in BPF. Send a notice to the log when promiscuous mode is enabled.
1995-09-22 17:57:48 +00:00
}
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
/*
* Return interface configuration
* of system. List may be used
* in later ioctl's (above) to get
* other information.
*/
/*ARGSUSED*/
Staticize, clean lint.
1995-12-09 20:47:15 +00:00
static int
Clean up whitespace, remove "register" keyword, ANSIfy. No functional changes.
2003-10-23 13:49:10 +00:00
ifconf(u_long cmd, caddr_t data)
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
{
Cosmetic cleanups and rearrangement for code to come. There should be no functional change in this commit.
2001-09-06 00:44:45 +00:00
struct ifconf *ifc = (struct ifconf *)data;
struct ifnet *ifp;
struct ifaddr *ifa;
Fix a LOR where ifconf() used copyout while holding a mutex. This LOR was seen when configuring addresses on interfaces using ifconfig. This patch has been verified to work with over eight thousand addresses assigned to an interface. LOR id: 031
2004-09-22 08:59:41 +00:00
struct ifreq ifr;
struct sbuf *sb;
int error, full = 0, valid_len, max_len;
/* Limit initial buffer size to MAXPHYS to avoid DoS from userspace. */
max_len = MAXPHYS - 1;
Validate ifc->ifc_len before submitting its incarnation to sbuf_new, which will finally lead to kernel panic. Security: This prevents a local (root-launched) DoS Submitted by: Wojciech A. Koszek [dunstan at freebsd czest pl] PR: 77421 MFC After: 1 week
2005-02-12 17:51:12 +00:00
/* Prevent hostile input from being able to crash the system */
if (ifc->ifc_len <= 0)
return (EINVAL);
Fix a LOR where ifconf() used copyout while holding a mutex. This LOR was seen when configuring addresses on interfaces using ifconfig. This patch has been verified to work with over eight thousand addresses assigned to an interface. LOR id: 031
2004-09-22 08:59:41 +00:00
again:
if (ifc->ifc_len <= max_len) {
max_len = ifc->ifc_len;
full = 1;
}
sb = sbuf_new(NULL, NULL, max_len + 1, SBUF_FIXEDLEN);
max_len = 0;
valid_len = 0;
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
Rework global locks for interface list and index management, correcting several critical bugs, including race conditions and lock order issues: Replace the single rwlock, ifnet_lock, with two locks, an rwlock and an sxlock. Either can be held to stablize the lists and indexes, but both are required to write. This allows the list to be held stable in both network interrupt contexts and sleepable user threads across sleeping memory allocations or device driver interactions. As before, writes to the interface list must occur from sleepable contexts. Reviewed by: bz, julian MFC after: 3 days
2009-08-23 20:40:19 +00:00
IFNET_RLOCK();
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
CK_STAILQ_FOREACH(ifp, &V_ifnet, if_link) {
Mechanical cleanup of epoch(9) usage in network stack. - Remove macros that covertly create epoch_tracker on thread stack. Such macros a quite unsafe, e.g. will produce a buggy code if same macro is used in embedded scopes. Explicitly declare epoch_tracker always. - Unmask interface list IFNET_RLOCK_NOSLEEP(), interface address list IF_ADDR_RLOCK() and interface AF specific data IF_AFDATA_RLOCK() read locking macros to what they actually are - the net_epoch. Keeping them as is is very misleading. They all are named FOO_RLOCK(), while they no longer have lock semantics. Now they allow recursion and what's more important they now no longer guarantee protection against their companion WLOCK macros. Note: INP_HASH_RLOCK() has same problems, but not touched by this commit. This is non functional mechanical change. The only functionally changed functions are ni6_addrs() and ni6_store_addrs(), where we no longer enter epoch recursively. Discussed with: jtl, gallatin
2019-01-09 01:11:19 +00:00
struct epoch_tracker et;
Replace the if_name and if_unit members of struct ifnet with new members if_xname, if_dname, and if_dunit. if_xname is the name of the interface and if_dname/unit are the driver name and instance. This change paves the way for interface renaming and enhanced pseudo device creation and configuration symantics. Approved By: re (in principle) Reviewed By: njl, imp Tested On: i386, amd64, sparc64 Obtained From: NetBSD (if_xname)
2003-10-31 18:32:15 +00:00
int addrs;
Remove trailing whitespace.
1995-05-30 08:16:23 +00:00
Zero the ifr.ifr_name buffer in ifconf() in order to avoid accidental disclosure of kernel memory to userland. Security: FreeBSD-SA-05:04.ifconf
2005-04-15 01:52:40 +00:00
/*
ifconf(): Always zero the whole struct ifreq. The previous split of zeroing ifr_name and ifr_addr seperately is safe on current architectures, but would be unsafe if pointers were larger than 8 bytes. Combining the zeroing adds no real cost (a few instructions) and makes the security property easier to verify. Reviewed by: kib, emaste Obtained from: CheriBSD MFC after: 3 days Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D14912
2018-04-05 21:58:28 +00:00
* Zero the ifr to make sure we don't disclose the contents
* of the stack.
Zero the ifr.ifr_name buffer in ifconf() in order to avoid accidental disclosure of kernel memory to userland. Security: FreeBSD-SA-05:04.ifconf
2005-04-15 01:52:40 +00:00
*/
ifconf(): Always zero the whole struct ifreq. The previous split of zeroing ifr_name and ifr_addr seperately is safe on current architectures, but would be unsafe if pointers were larger than 8 bytes. Combining the zeroing adds no real cost (a few instructions) and makes the security property easier to verify. Reviewed by: kib, emaste Obtained from: CheriBSD MFC after: 3 days Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D14912
2018-04-05 21:58:28 +00:00
memset(&ifr, 0, sizeof(ifr));
Zero the ifr.ifr_name buffer in ifconf() in order to avoid accidental disclosure of kernel memory to userland. Security: FreeBSD-SA-05:04.ifconf
2005-04-15 01:52:40 +00:00
Replace the if_name and if_unit members of struct ifnet with new members if_xname, if_dname, and if_dunit. if_xname is the name of the interface and if_dname/unit are the driver name and instance. This change paves the way for interface renaming and enhanced pseudo device creation and configuration symantics. Approved By: re (in principle) Reviewed By: njl, imp Tested On: i386, amd64, sparc64 Obtained From: NetBSD (if_xname)
2003-10-31 18:32:15 +00:00
if (strlcpy(ifr.ifr_name, ifp->if_xname, sizeof(ifr.ifr_name))
reclaim sbuf and clear lock on error in ifconf Submitted by: Ted Unangst Reviewed by: rwatson MFC after: 3 days
2005-09-04 17:32:47 +00:00
>= sizeof(ifr.ifr_name)) {
sbuf_delete(sb);
IFNET_RUNLOCK();
Fix a LOR where ifconf() used copyout while holding a mutex. This LOR was seen when configuring addresses on interfaces using ifconfig. This patch has been verified to work with over eight thousand addresses assigned to an interface. LOR id: 031
2004-09-22 08:59:41 +00:00
return (ENAMETOOLONG);
reclaim sbuf and clear lock on error in ifconf Submitted by: Ted Unangst Reviewed by: rwatson MFC after: 3 days
2005-09-04 17:32:47 +00:00
}
A number of bug-fixes inspired by Mark Treacy: - Allow PPP to run multicasts natively. - Deal properly with lots of similarly-named interfaces. - Don't sign-extend if_flags. NB: the last fix (to rtsock.c) must be reversed when we expand if_flags to a reasonable size. Submitted by: Mark Treacy
1994-10-05 20:11:28 +00:00
This Implements the mumbled about "Jail" feature. This is a seriously beefed up chroot kind of thing. The process is jailed along the same lines as a chroot does it, but with additional tough restrictions imposed on what the superuser can do. For all I know, it is safe to hand over the root bit inside a prison to the customer living in that prison, this is what it was developed for in fact: "real virtual servers". Each prison has an ip number associated with it, which all IP communications will be coerced to use and each prison has its own hostname. Needless to say, you need more RAM this way, but the advantage is that each customer can run their own particular version of apache and not stomp on the toes of their neighbors. It generally does what one would expect, but setting up a jail still takes a little knowledge. A few notes: I have no scripts for setting up a jail, don't ask me for them. The IP number should be an alias on one of the interfaces. mount a /proc in each jail, it will make ps more useable. /proc/<pid>/status tells the hostname of the prison for jailed processes. Quotas are only sensible if you have a mountpoint per prison. There are no privisions for stopping resource-hogging. Some "#ifdef INET" and similar may be missing (send patches!) If somebody wants to take it from here and develop it into more of a "virtual machine" they should be most welcome! Tools, comments, patches & documentation most welcome. Have fun... Sponsored by: http://www.rndassociates.com/ Run for almost a year by: http://www.servetheweb.com/
1999-04-28 11:38:52 +00:00
addrs = 0;
Mechanical cleanup of epoch(9) usage in network stack. - Remove macros that covertly create epoch_tracker on thread stack. Such macros a quite unsafe, e.g. will produce a buggy code if same macro is used in embedded scopes. Explicitly declare epoch_tracker always. - Unmask interface list IFNET_RLOCK_NOSLEEP(), interface address list IF_ADDR_RLOCK() and interface AF specific data IF_AFDATA_RLOCK() read locking macros to what they actually are - the net_epoch. Keeping them as is is very misleading. They all are named FOO_RLOCK(), while they no longer have lock semantics. Now they allow recursion and what's more important they now no longer guarantee protection against their companion WLOCK macros. Note: INP_HASH_RLOCK() has same problems, but not touched by this commit. This is non functional mechanical change. The only functionally changed functions are ni6_addrs() and ni6_store_addrs(), where we no longer enter epoch recursively. Discussed with: jtl, gallatin
2019-01-09 01:11:19 +00:00
NET_EPOCH_ENTER(et);
ifnet: Replace if_addr_lock rwlock with epoch + mutex Run on LLNW canaries and tested by pho@ gallatin: Using a 14-core, 28-HTT single socket E5-2697 v3 with a 40GbE MLX5 based ConnectX 4-LX NIC, I see an almost 12% improvement in received packet rate, and a larger improvement in bytes delivered all the way to userspace. When the host receiving 64 streams of netperf -H $DUT -t UDP_STREAM -- -m 1, I see, using nstat -I mce0 1 before the patch: InMpps OMpps InGbs OGbs err TCP Est %CPU syscalls csw irq GBfree 4.98 0.00 4.42 0.00 4235592 33 83.80 4720653 2149771 1235 247.32 4.73 0.00 4.20 0.00 4025260 33 82.99 4724900 2139833 1204 247.32 4.72 0.00 4.20 0.00 4035252 33 82.14 4719162 2132023 1264 247.32 4.71 0.00 4.21 0.00 4073206 33 83.68 4744973 2123317 1347 247.32 4.72 0.00 4.21 0.00 4061118 33 80.82 4713615 2188091 1490 247.32 4.72 0.00 4.21 0.00 4051675 33 85.29 4727399 2109011 1205 247.32 4.73 0.00 4.21 0.00 4039056 33 84.65 4724735 2102603 1053 247.32 After the patch InMpps OMpps InGbs OGbs err TCP Est %CPU syscalls csw irq GBfree 5.43 0.00 4.20 0.00 3313143 33 84.96 5434214 1900162 2656 245.51 5.43 0.00 4.20 0.00 3308527 33 85.24 5439695 1809382 2521 245.51 5.42 0.00 4.19 0.00 3316778 33 87.54 5416028 1805835 2256 245.51 5.42 0.00 4.19 0.00 3317673 33 90.44 5426044 1763056 2332 245.51 5.42 0.00 4.19 0.00 3314839 33 88.11 5435732 1792218 2499 245.52 5.44 0.00 4.19 0.00 3293228 33 91.84 5426301 1668597 2121 245.52 Similarly, netperf reports 230Mb/s before the patch, and 270Mb/s after the patch Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15366
2018-05-18 20:13:34 +00:00
CK_STAILQ_FOREACH(ifa, &ifp->if_addrhead, ifa_link) {
Fix sense of comparison in space test. Also eliminate a compile warning and remove a previously existing off-by-one error.
2001-09-07 05:32:54 +00:00
struct sockaddr *sa = ifa->ifa_addr;
Standardize the various prison_foo_ip[46] functions and prison_if to return zero on success and an error code otherwise. The possible errors are EADDRNOTAVAIL if an address being checked for doesn't match the prison, and EAFNOSUPPORT if the prison doesn't have any addresses in that address family. For most callers of these functions, use the returned error code instead of e.g. a hard-coded EADDRNOTAVAIL or EINVAL. Always include a jailed() check in these functions, where a non-jailed cred always returns success (and makes no changes). Remove the explicit jailed() checks that preceded many of the function calls. Approved by: bz (mentor)
2009-02-05 14:06:09 +00:00
if (prison_if(curthread->td_ucred, sa) != 0)
This Implements the mumbled about "Jail" feature. This is a seriously beefed up chroot kind of thing. The process is jailed along the same lines as a chroot does it, but with additional tough restrictions imposed on what the superuser can do. For all I know, it is safe to hand over the root bit inside a prison to the customer living in that prison, this is what it was developed for in fact: "real virtual servers". Each prison has an ip number associated with it, which all IP communications will be coerced to use and each prison has its own hostname. Needless to say, you need more RAM this way, but the advantage is that each customer can run their own particular version of apache and not stomp on the toes of their neighbors. It generally does what one would expect, but setting up a jail still takes a little knowledge. A few notes: I have no scripts for setting up a jail, don't ask me for them. The IP number should be an alias on one of the interfaces. mount a /proc in each jail, it will make ps more useable. /proc/<pid>/status tells the hostname of the prison for jailed processes. Quotas are only sensible if you have a mountpoint per prison. There are no privisions for stopping resource-hogging. Some "#ifdef INET" and similar may be missing (send patches!) If somebody wants to take it from here and develop it into more of a "virtual machine" they should be most welcome! Tools, comments, patches & documentation most welcome. Have fun... Sponsored by: http://www.rndassociates.com/ Run for almost a year by: http://www.servetheweb.com/
1999-04-28 11:38:52 +00:00
continue;
addrs++;
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
if (sa->sa_len <= sizeof(*sa)) {
ifconf(): correct handling of sockaddrs smaller than struct sockaddr. Portable programs that use SIOCGIFCONF (e.g. traceroute) assume that each pseudo ifreq is of length MAX(sizeof(struct ifreq), sizeof(ifr_name) + ifr_addr.sa_len). For short sockaddrs we copied too much from the source sockaddr resulting in a heap leak. I believe only one such sockaddr exists (struct sockaddr_sco which is 8 bytes) and it is unclear if such sockaddrs end up on interfaces in practice. If it did, the result would be an 8 byte heap leak on current architectures. admbugs: 869 Reviewed by: kib Obtained from: CheriBSD MFC after: 3 days Security: kernel heap leak Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D14981
2018-04-06 20:26:56 +00:00
if (sa->sa_len < sizeof(*sa)) {
memset(&ifr.ifr_ifru.ifru_addr, 0,
sizeof(ifr.ifr_ifru.ifru_addr));
memcpy(&ifr.ifr_ifru.ifru_addr, sa,
sa->sa_len);
} else
ifr.ifr_ifru.ifru_addr = *sa;
Fix a LOR where ifconf() used copyout while holding a mutex. This LOR was seen when configuring addresses on interfaces using ifconfig. This patch has been verified to work with over eight thousand addresses assigned to an interface. LOR id: 031
2004-09-22 08:59:41 +00:00
sbuf_bcat(sb, &ifr, sizeof(ifr));
max_len += sizeof(ifr);
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
} else {
Fix a LOR where ifconf() used copyout while holding a mutex. This LOR was seen when configuring addresses on interfaces using ifconfig. This patch has been verified to work with over eight thousand addresses assigned to an interface. LOR id: 031
2004-09-22 08:59:41 +00:00
sbuf_bcat(sb, &ifr,
offsetof(struct ifreq, ifr_addr));
max_len += offsetof(struct ifreq, ifr_addr);
sbuf_bcat(sb, sa, sa->sa_len);
max_len += sa->sa_len;
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
}
Fix a LOR where ifconf() used copyout while holding a mutex. This LOR was seen when configuring addresses on interfaces using ifconfig. This patch has been verified to work with over eight thousand addresses assigned to an interface. LOR id: 031
2004-09-22 08:59:41 +00:00
Replace sbuf_overflowed() with sbuf_error(), which returns any error code associated with overflow or with the drain function. While this function is not expected to be used often, it produces more information in the form of an errno that sbuf_overflowed() did.
2010-09-10 16:42:16 +00:00
if (sbuf_error(sb) == 0)
Fix a LOR where ifconf() used copyout while holding a mutex. This LOR was seen when configuring addresses on interfaces using ifconfig. This patch has been verified to work with over eight thousand addresses assigned to an interface. LOR id: 031
2004-09-22 08:59:41 +00:00
valid_len = sbuf_len(sb);
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
}
Mechanical cleanup of epoch(9) usage in network stack. - Remove macros that covertly create epoch_tracker on thread stack. Such macros a quite unsafe, e.g. will produce a buggy code if same macro is used in embedded scopes. Explicitly declare epoch_tracker always. - Unmask interface list IFNET_RLOCK_NOSLEEP(), interface address list IF_ADDR_RLOCK() and interface AF specific data IF_AFDATA_RLOCK() read locking macros to what they actually are - the net_epoch. Keeping them as is is very misleading. They all are named FOO_RLOCK(), while they no longer have lock semantics. Now they allow recursion and what's more important they now no longer guarantee protection against their companion WLOCK macros. Note: INP_HASH_RLOCK() has same problems, but not touched by this commit. This is non functional mechanical change. The only functionally changed functions are ni6_addrs() and ni6_store_addrs(), where we no longer enter epoch recursively. Discussed with: jtl, gallatin
2019-01-09 01:11:19 +00:00
NET_EPOCH_EXIT(et);
Fix a LOR where ifconf() used copyout while holding a mutex. This LOR was seen when configuring addresses on interfaces using ifconfig. This patch has been verified to work with over eight thousand addresses assigned to an interface. LOR id: 031
2004-09-22 08:59:41 +00:00
if (addrs == 0) {
sbuf_bcat(sb, &ifr, sizeof(ifr));
max_len += sizeof(ifr);
Replace sbuf_overflowed() with sbuf_error(), which returns any error code associated with overflow or with the drain function. While this function is not expected to be used often, it produces more information in the form of an errno that sbuf_overflowed() did.
2010-09-10 16:42:16 +00:00
if (sbuf_error(sb) == 0)
Fix a LOR where ifconf() used copyout while holding a mutex. This LOR was seen when configuring addresses on interfaces using ifconfig. This patch has been verified to work with over eight thousand addresses assigned to an interface. LOR id: 031
2004-09-22 08:59:41 +00:00
valid_len = sbuf_len(sb);
This Implements the mumbled about "Jail" feature. This is a seriously beefed up chroot kind of thing. The process is jailed along the same lines as a chroot does it, but with additional tough restrictions imposed on what the superuser can do. For all I know, it is safe to hand over the root bit inside a prison to the customer living in that prison, this is what it was developed for in fact: "real virtual servers". Each prison has an ip number associated with it, which all IP communications will be coerced to use and each prison has its own hostname. Needless to say, you need more RAM this way, but the advantage is that each customer can run their own particular version of apache and not stomp on the toes of their neighbors. It generally does what one would expect, but setting up a jail still takes a little knowledge. A few notes: I have no scripts for setting up a jail, don't ask me for them. The IP number should be an alias on one of the interfaces. mount a /proc in each jail, it will make ps more useable. /proc/<pid>/status tells the hostname of the prison for jailed processes. Quotas are only sensible if you have a mountpoint per prison. There are no privisions for stopping resource-hogging. Some "#ifdef INET" and similar may be missing (send patches!) If somebody wants to take it from here and develop it into more of a "virtual machine" they should be most welcome! Tools, comments, patches & documentation most welcome. Have fun... Sponsored by: http://www.rndassociates.com/ Run for almost a year by: http://www.servetheweb.com/
1999-04-28 11:38:52 +00:00
}
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
}
SMP locking for ifnet list.
2002-12-22 05:35:03 +00:00
IFNET_RUNLOCK();
Fix a LOR where ifconf() used copyout while holding a mutex. This LOR was seen when configuring addresses on interfaces using ifconfig. This patch has been verified to work with over eight thousand addresses assigned to an interface. LOR id: 031
2004-09-22 08:59:41 +00:00
/*
* If we didn't allocate enough space (uncommon), try again. If
* we have already allocated as much space as we are allowed,
* return what we've got.
*/
if (valid_len != max_len && !full) {
sbuf_delete(sb);
goto again;
}
ifc->ifc_len = valid_len;
Call sbuf_finish() before sbuf_data() so as to not panic the system.
2004-09-22 12:53:27 +00:00
sbuf_finish(sb);
Fix a LOR where ifconf() used copyout while holding a mutex. This LOR was seen when configuring addresses on interfaces using ifconfig. This patch has been verified to work with over eight thousand addresses assigned to an interface. LOR id: 031
2004-09-22 08:59:41 +00:00
error = copyout(sbuf_data(sb), ifc->ifc_req, ifc->ifc_len);
sbuf_delete(sb);
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
return (error);
}
Checkpoint the beginnings of the new kernel interface for multicast group memberships. This is not actually operative at the moment (a lot of other code still needs to be changed), but this seemed like a useful reference point to check in so that others (i.e. Bill Fenner) have fair warning of where we are going.
1997-01-07 19:15:32 +00:00
/*
Typo in comment.
2005-02-22 15:29:29 +00:00
* Just like ifpromisc(), but for all-multicast-reception mode.
Checkpoint the beginnings of the new kernel interface for multicast group memberships. This is not actually operative at the moment (a lot of other code still needs to be changed), but this seemed like a useful reference point to check in so that others (i.e. Bill Fenner) have fair warning of where we are going.
1997-01-07 19:15:32 +00:00
*/
int
Clean up whitespace, remove "register" keyword, ANSIfy. No functional changes.
2003-10-23 13:49:10 +00:00
if_allmulti(struct ifnet *ifp, int onswitch)
Checkpoint the beginnings of the new kernel interface for multicast group memberships. This is not actually operative at the moment (a lot of other code still needs to be changed), but this seemed like a useful reference point to check in so that others (i.e. Bill Fenner) have fair warning of where we are going.
1997-01-07 19:15:32 +00:00
{
Send RTM_IFINFO messages whenever promiscuous and all-multicast modes are enabled or disabled.
1997-02-14 15:30:54 +00:00
MFp4: - Introduce a helper function if_setflag() containing the code common to ifpromisc() and if_allmulti() instead of duplicating the code poorly, with different bugs. - Call ifp->if_ioctl() in a consistent way: always use more compatible C syntax and check whether ifp->if_ioctl is not NULL prior to the call. MFC after: 1 month
2005-07-14 13:56:51 +00:00
return (if_setflag(ifp, IFF_ALLMULTI, 0, &ifp->if_amcount, onswitch));
Checkpoint the beginnings of the new kernel interface for multicast group memberships. This is not actually operative at the moment (a lot of other code still needs to be changed), but this seemed like a useful reference point to check in so that others (i.e. Bill Fenner) have fair warning of where we are going.
1997-01-07 19:15:32 +00:00
}
Fix tinderbox; ng_ether needs to see if_findmulti().
2007-03-20 03:15:43 +00:00
struct ifmultiaddr *
Constantify lookup key in ifa_ifwith* functions. Some places in our network stack already have const arguments (like if_output() routines and LLE functions). Code using ifa_ifwith (and similar functins) along with LLE/_output functions is currently bound to use tricks like __DECONST(). Provide a cleaner way by making sockaddr lookup key really constant. MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D3464
2015-09-05 05:33:20 +00:00
if_findmulti(struct ifnet *ifp, const struct sockaddr *sa)
Protect link layer network interface multicast address list manipulation using ifp->if_addr_mtx: - Initialize if_addr_mtx when ifnet is initialized. - Destroy if_addr_mtx when ifnet is torn down. - Rename ifmaof_ifpforaddr() to if_findmulti(); assert if_addr_mtx. Staticize. - Extract ifmultiaddr allocation and initialization into if_allocmulti(); accept a 'mflags' argument to indicate whether or not sleeping is permitted. This centralizes error handling and address duplication. - Extract ifmultiaddr tear-down and deallocation in if_freemulti(). - Re-structure if_addmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Make use of non-sleeping allocations. Annotate the fact that we only generate routing socket events for explicit address addition, not implicit link layer address addition. - Re-structure if_delmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Annotate the lack of a routing socket event for implicit link layer address removal. - De-spl all and sundry. Problem reported by: Ed Maste <emaste at phaedrus dot sandvine dot ca> MFC after: 1 week
2005-08-02 23:23:26 +00:00
{
struct ifmultiaddr *ifma;
IF_ADDR_LOCK_ASSERT(ifp);
ifnet: Replace if_addr_lock rwlock with epoch + mutex Run on LLNW canaries and tested by pho@ gallatin: Using a 14-core, 28-HTT single socket E5-2697 v3 with a 40GbE MLX5 based ConnectX 4-LX NIC, I see an almost 12% improvement in received packet rate, and a larger improvement in bytes delivered all the way to userspace. When the host receiving 64 streams of netperf -H $DUT -t UDP_STREAM -- -m 1, I see, using nstat -I mce0 1 before the patch: InMpps OMpps InGbs OGbs err TCP Est %CPU syscalls csw irq GBfree 4.98 0.00 4.42 0.00 4235592 33 83.80 4720653 2149771 1235 247.32 4.73 0.00 4.20 0.00 4025260 33 82.99 4724900 2139833 1204 247.32 4.72 0.00 4.20 0.00 4035252 33 82.14 4719162 2132023 1264 247.32 4.71 0.00 4.21 0.00 4073206 33 83.68 4744973 2123317 1347 247.32 4.72 0.00 4.21 0.00 4061118 33 80.82 4713615 2188091 1490 247.32 4.72 0.00 4.21 0.00 4051675 33 85.29 4727399 2109011 1205 247.32 4.73 0.00 4.21 0.00 4039056 33 84.65 4724735 2102603 1053 247.32 After the patch InMpps OMpps InGbs OGbs err TCP Est %CPU syscalls csw irq GBfree 5.43 0.00 4.20 0.00 3313143 33 84.96 5434214 1900162 2656 245.51 5.43 0.00 4.20 0.00 3308527 33 85.24 5439695 1809382 2521 245.51 5.42 0.00 4.19 0.00 3316778 33 87.54 5416028 1805835 2256 245.51 5.42 0.00 4.19 0.00 3317673 33 90.44 5426044 1763056 2332 245.51 5.42 0.00 4.19 0.00 3314839 33 88.11 5435732 1792218 2499 245.52 5.44 0.00 4.19 0.00 3293228 33 91.84 5426301 1668597 2121 245.52 Similarly, netperf reports 230Mb/s before the patch, and 270Mb/s after the patch Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15366
2018-05-18 20:13:34 +00:00
CK_STAILQ_FOREACH(ifma, &ifp->if_multiaddrs, ifma_link) {
Fix a bug in if_findmulti(), whereby it would not find (and thus delete) a link-layer multicast group membership. Such memberships are needed in order to support protocols such as IS-IS without putting the interface into PROMISC or ALLMULTI modes. sa_equal() is not OK for comparing sockaddr_dl as it has deeper structure than a simple byte array, so add sa_dl_equal() and use that instead. Reviewed by: rwatson Verified with: /usr/sbin/mtest Bug found by: Jouke Witteveen MFC after: 2 weeks
2007-02-22 00:14:02 +00:00
if (sa->sa_family == AF_LINK) {
if (sa_dl_equal(ifma->ifma_addr, sa))
break;
} else {
if (sa_equal(ifma->ifma_addr, sa))
break;
}
Protect link layer network interface multicast address list manipulation using ifp->if_addr_mtx: - Initialize if_addr_mtx when ifnet is initialized. - Destroy if_addr_mtx when ifnet is torn down. - Rename ifmaof_ifpforaddr() to if_findmulti(); assert if_addr_mtx. Staticize. - Extract ifmultiaddr allocation and initialization into if_allocmulti(); accept a 'mflags' argument to indicate whether or not sleeping is permitted. This centralizes error handling and address duplication. - Extract ifmultiaddr tear-down and deallocation in if_freemulti(). - Re-structure if_addmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Make use of non-sleeping allocations. Annotate the fact that we only generate routing socket events for explicit address addition, not implicit link layer address addition. - Re-structure if_delmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Annotate the lack of a routing socket event for implicit link layer address removal. - De-spl all and sundry. Problem reported by: Ed Maste <emaste at phaedrus dot sandvine dot ca> MFC after: 1 week
2005-08-02 23:23:26 +00:00
}
return ifma;
}
Checkpoint the beginnings of the new kernel interface for multicast group memberships. This is not actually operative at the moment (a lot of other code still needs to be changed), but this seemed like a useful reference point to check in so that others (i.e. Bill Fenner) have fair warning of where we are going.
1997-01-07 19:15:32 +00:00
/*
Protect link layer network interface multicast address list manipulation using ifp->if_addr_mtx: - Initialize if_addr_mtx when ifnet is initialized. - Destroy if_addr_mtx when ifnet is torn down. - Rename ifmaof_ifpforaddr() to if_findmulti(); assert if_addr_mtx. Staticize. - Extract ifmultiaddr allocation and initialization into if_allocmulti(); accept a 'mflags' argument to indicate whether or not sleeping is permitted. This centralizes error handling and address duplication. - Extract ifmultiaddr tear-down and deallocation in if_freemulti(). - Re-structure if_addmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Make use of non-sleeping allocations. Annotate the fact that we only generate routing socket events for explicit address addition, not implicit link layer address addition. - Re-structure if_delmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Annotate the lack of a routing socket event for implicit link layer address removal. - De-spl all and sundry. Problem reported by: Ed Maste <emaste at phaedrus dot sandvine dot ca> MFC after: 1 week
2005-08-02 23:23:26 +00:00
* Allocate a new ifmultiaddr and initialize based on passed arguments. We
* make copies of passed sockaddrs. The ifmultiaddr will not be added to
* the ifnet multicast address list here, so the caller must do that and
* other setup work (such as notifying the device driver). The reference
* count is initialized to 1.
Checkpoint the beginnings of the new kernel interface for multicast group memberships. This is not actually operative at the moment (a lot of other code still needs to be changed), but this seemed like a useful reference point to check in so that others (i.e. Bill Fenner) have fair warning of where we are going.
1997-01-07 19:15:32 +00:00
*/
Protect link layer network interface multicast address list manipulation using ifp->if_addr_mtx: - Initialize if_addr_mtx when ifnet is initialized. - Destroy if_addr_mtx when ifnet is torn down. - Rename ifmaof_ifpforaddr() to if_findmulti(); assert if_addr_mtx. Staticize. - Extract ifmultiaddr allocation and initialization into if_allocmulti(); accept a 'mflags' argument to indicate whether or not sleeping is permitted. This centralizes error handling and address duplication. - Extract ifmultiaddr tear-down and deallocation in if_freemulti(). - Re-structure if_addmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Make use of non-sleeping allocations. Annotate the fact that we only generate routing socket events for explicit address addition, not implicit link layer address addition. - Re-structure if_delmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Annotate the lack of a routing socket event for implicit link layer address removal. - De-spl all and sundry. Problem reported by: Ed Maste <emaste at phaedrus dot sandvine dot ca> MFC after: 1 week
2005-08-02 23:23:26 +00:00
static struct ifmultiaddr *
if_allocmulti(struct ifnet *ifp, struct sockaddr *sa, struct sockaddr *llsa,
int mflags)
Checkpoint the beginnings of the new kernel interface for multicast group memberships. This is not actually operative at the moment (a lot of other code still needs to be changed), but this seemed like a useful reference point to check in so that others (i.e. Bill Fenner) have fair warning of where we are going.
1997-01-07 19:15:32 +00:00
{
struct ifmultiaddr *ifma;
Protect link layer network interface multicast address list manipulation using ifp->if_addr_mtx: - Initialize if_addr_mtx when ifnet is initialized. - Destroy if_addr_mtx when ifnet is torn down. - Rename ifmaof_ifpforaddr() to if_findmulti(); assert if_addr_mtx. Staticize. - Extract ifmultiaddr allocation and initialization into if_allocmulti(); accept a 'mflags' argument to indicate whether or not sleeping is permitted. This centralizes error handling and address duplication. - Extract ifmultiaddr tear-down and deallocation in if_freemulti(). - Re-structure if_addmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Make use of non-sleeping allocations. Annotate the fact that we only generate routing socket events for explicit address addition, not implicit link layer address addition. - Re-structure if_delmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Annotate the lack of a routing socket event for implicit link layer address removal. - De-spl all and sundry. Problem reported by: Ed Maste <emaste at phaedrus dot sandvine dot ca> MFC after: 1 week
2005-08-02 23:23:26 +00:00
struct sockaddr *dupsa;
Retire the MALLOC and FREE macros. They are an abomination unto style(9). MFC after: 3 months
2008-10-23 15:53:51 +00:00
ifma = malloc(sizeof *ifma, M_IFMADDR, mflags |
Protect link layer network interface multicast address list manipulation using ifp->if_addr_mtx: - Initialize if_addr_mtx when ifnet is initialized. - Destroy if_addr_mtx when ifnet is torn down. - Rename ifmaof_ifpforaddr() to if_findmulti(); assert if_addr_mtx. Staticize. - Extract ifmultiaddr allocation and initialization into if_allocmulti(); accept a 'mflags' argument to indicate whether or not sleeping is permitted. This centralizes error handling and address duplication. - Extract ifmultiaddr tear-down and deallocation in if_freemulti(). - Re-structure if_addmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Make use of non-sleeping allocations. Annotate the fact that we only generate routing socket events for explicit address addition, not implicit link layer address addition. - Re-structure if_delmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Annotate the lack of a routing socket event for implicit link layer address removal. - De-spl all and sundry. Problem reported by: Ed Maste <emaste at phaedrus dot sandvine dot ca> MFC after: 1 week
2005-08-02 23:23:26 +00:00
M_ZERO);
if (ifma == NULL)
return (NULL);
Retire the MALLOC and FREE macros. They are an abomination unto style(9). MFC after: 3 months
2008-10-23 15:53:51 +00:00
dupsa = malloc(sa->sa_len, M_IFMADDR, mflags);
Protect link layer network interface multicast address list manipulation using ifp->if_addr_mtx: - Initialize if_addr_mtx when ifnet is initialized. - Destroy if_addr_mtx when ifnet is torn down. - Rename ifmaof_ifpforaddr() to if_findmulti(); assert if_addr_mtx. Staticize. - Extract ifmultiaddr allocation and initialization into if_allocmulti(); accept a 'mflags' argument to indicate whether or not sleeping is permitted. This centralizes error handling and address duplication. - Extract ifmultiaddr tear-down and deallocation in if_freemulti(). - Re-structure if_addmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Make use of non-sleeping allocations. Annotate the fact that we only generate routing socket events for explicit address addition, not implicit link layer address addition. - Re-structure if_delmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Annotate the lack of a routing socket event for implicit link layer address removal. - De-spl all and sundry. Problem reported by: Ed Maste <emaste at phaedrus dot sandvine dot ca> MFC after: 1 week
2005-08-02 23:23:26 +00:00
if (dupsa == NULL) {
Retire the MALLOC and FREE macros. They are an abomination unto style(9). MFC after: 3 months
2008-10-23 15:53:51 +00:00
free(ifma, M_IFMADDR);
Protect link layer network interface multicast address list manipulation using ifp->if_addr_mtx: - Initialize if_addr_mtx when ifnet is initialized. - Destroy if_addr_mtx when ifnet is torn down. - Rename ifmaof_ifpforaddr() to if_findmulti(); assert if_addr_mtx. Staticize. - Extract ifmultiaddr allocation and initialization into if_allocmulti(); accept a 'mflags' argument to indicate whether or not sleeping is permitted. This centralizes error handling and address duplication. - Extract ifmultiaddr tear-down and deallocation in if_freemulti(). - Re-structure if_addmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Make use of non-sleeping allocations. Annotate the fact that we only generate routing socket events for explicit address addition, not implicit link layer address addition. - Re-structure if_delmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Annotate the lack of a routing socket event for implicit link layer address removal. - De-spl all and sundry. Problem reported by: Ed Maste <emaste at phaedrus dot sandvine dot ca> MFC after: 1 week
2005-08-02 23:23:26 +00:00
return (NULL);
}
bcopy(sa, dupsa, sa->sa_len);
ifma->ifma_addr = dupsa;
ifma->ifma_ifp = ifp;
ifma->ifma_refcount = 1;
ifma->ifma_protospec = NULL;
if (llsa == NULL) {
ifma->ifma_lladdr = NULL;
return (ifma);
}
Retire the MALLOC and FREE macros. They are an abomination unto style(9). MFC after: 3 months
2008-10-23 15:53:51 +00:00
dupsa = malloc(llsa->sa_len, M_IFMADDR, mflags);
Protect link layer network interface multicast address list manipulation using ifp->if_addr_mtx: - Initialize if_addr_mtx when ifnet is initialized. - Destroy if_addr_mtx when ifnet is torn down. - Rename ifmaof_ifpforaddr() to if_findmulti(); assert if_addr_mtx. Staticize. - Extract ifmultiaddr allocation and initialization into if_allocmulti(); accept a 'mflags' argument to indicate whether or not sleeping is permitted. This centralizes error handling and address duplication. - Extract ifmultiaddr tear-down and deallocation in if_freemulti(). - Re-structure if_addmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Make use of non-sleeping allocations. Annotate the fact that we only generate routing socket events for explicit address addition, not implicit link layer address addition. - Re-structure if_delmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Annotate the lack of a routing socket event for implicit link layer address removal. - De-spl all and sundry. Problem reported by: Ed Maste <emaste at phaedrus dot sandvine dot ca> MFC after: 1 week
2005-08-02 23:23:26 +00:00
if (dupsa == NULL) {
Retire the MALLOC and FREE macros. They are an abomination unto style(9). MFC after: 3 months
2008-10-23 15:53:51 +00:00
free(ifma->ifma_addr, M_IFMADDR);
free(ifma, M_IFMADDR);
Protect link layer network interface multicast address list manipulation using ifp->if_addr_mtx: - Initialize if_addr_mtx when ifnet is initialized. - Destroy if_addr_mtx when ifnet is torn down. - Rename ifmaof_ifpforaddr() to if_findmulti(); assert if_addr_mtx. Staticize. - Extract ifmultiaddr allocation and initialization into if_allocmulti(); accept a 'mflags' argument to indicate whether or not sleeping is permitted. This centralizes error handling and address duplication. - Extract ifmultiaddr tear-down and deallocation in if_freemulti(). - Re-structure if_addmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Make use of non-sleeping allocations. Annotate the fact that we only generate routing socket events for explicit address addition, not implicit link layer address addition. - Re-structure if_delmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Annotate the lack of a routing socket event for implicit link layer address removal. - De-spl all and sundry. Problem reported by: Ed Maste <emaste at phaedrus dot sandvine dot ca> MFC after: 1 week
2005-08-02 23:23:26 +00:00
return (NULL);
}
bcopy(llsa, dupsa, llsa->sa_len);
ifma->ifma_lladdr = dupsa;
return (ifma);
}
/*
* if_freemulti: free ifmultiaddr structure and possibly attached related
* addresses. The caller is responsible for implementing reference
* counting, notifying the driver, handling routing messages, and releasing
* any dependent link layer state.
*/
r333175 introduced deferred deletion of multicast addresses in order to permit the driver ioctl to sleep on commands to the NIC when updating multicast filters. More generally this permitted driver's to use an sx as a softc lock. Unfortunately this change introduced a race whereby a a multicast update would still be queued for deletion when ifconfig deleted the interface thus calling down in to _purgemaddrs and synchronously deleting _all_ of the multicast addresses on the interface. Synchronously remove all external references to a multicast address before enqueueing for delete. Reported by: lwhsu Approved by: sbruno
2018-05-06 20:34:13 +00:00
#ifdef MCAST_VERBOSE
extern void kdb_backtrace(void);
#endif
ifnet: Replace if_addr_lock rwlock with epoch + mutex Run on LLNW canaries and tested by pho@ gallatin: Using a 14-core, 28-HTT single socket E5-2697 v3 with a 40GbE MLX5 based ConnectX 4-LX NIC, I see an almost 12% improvement in received packet rate, and a larger improvement in bytes delivered all the way to userspace. When the host receiving 64 streams of netperf -H $DUT -t UDP_STREAM -- -m 1, I see, using nstat -I mce0 1 before the patch: InMpps OMpps InGbs OGbs err TCP Est %CPU syscalls csw irq GBfree 4.98 0.00 4.42 0.00 4235592 33 83.80 4720653 2149771 1235 247.32 4.73 0.00 4.20 0.00 4025260 33 82.99 4724900 2139833 1204 247.32 4.72 0.00 4.20 0.00 4035252 33 82.14 4719162 2132023 1264 247.32 4.71 0.00 4.21 0.00 4073206 33 83.68 4744973 2123317 1347 247.32 4.72 0.00 4.21 0.00 4061118 33 80.82 4713615 2188091 1490 247.32 4.72 0.00 4.21 0.00 4051675 33 85.29 4727399 2109011 1205 247.32 4.73 0.00 4.21 0.00 4039056 33 84.65 4724735 2102603 1053 247.32 After the patch InMpps OMpps InGbs OGbs err TCP Est %CPU syscalls csw irq GBfree 5.43 0.00 4.20 0.00 3313143 33 84.96 5434214 1900162 2656 245.51 5.43 0.00 4.20 0.00 3308527 33 85.24 5439695 1809382 2521 245.51 5.42 0.00 4.19 0.00 3316778 33 87.54 5416028 1805835 2256 245.51 5.42 0.00 4.19 0.00 3317673 33 90.44 5426044 1763056 2332 245.51 5.42 0.00 4.19 0.00 3314839 33 88.11 5435732 1792218 2499 245.52 5.44 0.00 4.19 0.00 3293228 33 91.84 5426301 1668597 2121 245.52 Similarly, netperf reports 230Mb/s before the patch, and 270Mb/s after the patch Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15366
2018-05-18 20:13:34 +00:00
static void
if_freemulti_internal(struct ifmultiaddr *ifma)
Protect link layer network interface multicast address list manipulation using ifp->if_addr_mtx: - Initialize if_addr_mtx when ifnet is initialized. - Destroy if_addr_mtx when ifnet is torn down. - Rename ifmaof_ifpforaddr() to if_findmulti(); assert if_addr_mtx. Staticize. - Extract ifmultiaddr allocation and initialization into if_allocmulti(); accept a 'mflags' argument to indicate whether or not sleeping is permitted. This centralizes error handling and address duplication. - Extract ifmultiaddr tear-down and deallocation in if_freemulti(). - Re-structure if_addmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Make use of non-sleeping allocations. Annotate the fact that we only generate routing socket events for explicit address addition, not implicit link layer address addition. - Re-structure if_delmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Annotate the lack of a routing socket event for implicit link layer address removal. - De-spl all and sundry. Problem reported by: Ed Maste <emaste at phaedrus dot sandvine dot ca> MFC after: 1 week
2005-08-02 23:23:26 +00:00
{
Implement reference counting for ifmultiaddr, in_multi, and in6_multi structures. Detect when ifnet instances are detached from the network stack and perform appropriate cleanup to prevent memory leaks. This has been implemented in such a way as to be backwards ABI compatible. Kernel consumers are changed to use if_delmulti_ifma(); in_delmulti() is unable to detect interface removal by design, as it performs searches on structures which are removed with the interface. With this architectural change, the panics FreeBSD users have experienced with carp and pfsync should be resolved. Obtained from: p4 branch bms_netdev Reviewed by: andre Sponsored by: Garance A Drosehn Idea from: NetBSD MFC after: 1 month
2007-03-20 00:36:10 +00:00
KASSERT(ifma->ifma_refcount == 0, ("if_freemulti: refcount %d",
Protect link layer network interface multicast address list manipulation using ifp->if_addr_mtx: - Initialize if_addr_mtx when ifnet is initialized. - Destroy if_addr_mtx when ifnet is torn down. - Rename ifmaof_ifpforaddr() to if_findmulti(); assert if_addr_mtx. Staticize. - Extract ifmultiaddr allocation and initialization into if_allocmulti(); accept a 'mflags' argument to indicate whether or not sleeping is permitted. This centralizes error handling and address duplication. - Extract ifmultiaddr tear-down and deallocation in if_freemulti(). - Re-structure if_addmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Make use of non-sleeping allocations. Annotate the fact that we only generate routing socket events for explicit address addition, not implicit link layer address addition. - Re-structure if_delmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Annotate the lack of a routing socket event for implicit link layer address removal. - De-spl all and sundry. Problem reported by: Ed Maste <emaste at phaedrus dot sandvine dot ca> MFC after: 1 week
2005-08-02 23:23:26 +00:00
ifma->ifma_refcount));
if (ifma->ifma_lladdr != NULL)
Retire the MALLOC and FREE macros. They are an abomination unto style(9). MFC after: 3 months
2008-10-23 15:53:51 +00:00
free(ifma->ifma_lladdr, M_IFMADDR);
r333175 introduced deferred deletion of multicast addresses in order to permit the driver ioctl to sleep on commands to the NIC when updating multicast filters. More generally this permitted driver's to use an sx as a softc lock. Unfortunately this change introduced a race whereby a a multicast update would still be queued for deletion when ifconfig deleted the interface thus calling down in to _purgemaddrs and synchronously deleting _all_ of the multicast addresses on the interface. Synchronously remove all external references to a multicast address before enqueueing for delete. Reported by: lwhsu Approved by: sbruno
2018-05-06 20:34:13 +00:00
#ifdef MCAST_VERBOSE
kdb_backtrace();
printf("%s freeing ifma: %p\n", __func__, ifma);
#endif
Retire the MALLOC and FREE macros. They are an abomination unto style(9). MFC after: 3 months
2008-10-23 15:53:51 +00:00
free(ifma->ifma_addr, M_IFMADDR);
free(ifma, M_IFMADDR);
Protect link layer network interface multicast address list manipulation using ifp->if_addr_mtx: - Initialize if_addr_mtx when ifnet is initialized. - Destroy if_addr_mtx when ifnet is torn down. - Rename ifmaof_ifpforaddr() to if_findmulti(); assert if_addr_mtx. Staticize. - Extract ifmultiaddr allocation and initialization into if_allocmulti(); accept a 'mflags' argument to indicate whether or not sleeping is permitted. This centralizes error handling and address duplication. - Extract ifmultiaddr tear-down and deallocation in if_freemulti(). - Re-structure if_addmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Make use of non-sleeping allocations. Annotate the fact that we only generate routing socket events for explicit address addition, not implicit link layer address addition. - Re-structure if_delmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Annotate the lack of a routing socket event for implicit link layer address removal. - De-spl all and sundry. Problem reported by: Ed Maste <emaste at phaedrus dot sandvine dot ca> MFC after: 1 week
2005-08-02 23:23:26 +00:00
}
ifnet: Replace if_addr_lock rwlock with epoch + mutex Run on LLNW canaries and tested by pho@ gallatin: Using a 14-core, 28-HTT single socket E5-2697 v3 with a 40GbE MLX5 based ConnectX 4-LX NIC, I see an almost 12% improvement in received packet rate, and a larger improvement in bytes delivered all the way to userspace. When the host receiving 64 streams of netperf -H $DUT -t UDP_STREAM -- -m 1, I see, using nstat -I mce0 1 before the patch: InMpps OMpps InGbs OGbs err TCP Est %CPU syscalls csw irq GBfree 4.98 0.00 4.42 0.00 4235592 33 83.80 4720653 2149771 1235 247.32 4.73 0.00 4.20 0.00 4025260 33 82.99 4724900 2139833 1204 247.32 4.72 0.00 4.20 0.00 4035252 33 82.14 4719162 2132023 1264 247.32 4.71 0.00 4.21 0.00 4073206 33 83.68 4744973 2123317 1347 247.32 4.72 0.00 4.21 0.00 4061118 33 80.82 4713615 2188091 1490 247.32 4.72 0.00 4.21 0.00 4051675 33 85.29 4727399 2109011 1205 247.32 4.73 0.00 4.21 0.00 4039056 33 84.65 4724735 2102603 1053 247.32 After the patch InMpps OMpps InGbs OGbs err TCP Est %CPU syscalls csw irq GBfree 5.43 0.00 4.20 0.00 3313143 33 84.96 5434214 1900162 2656 245.51 5.43 0.00 4.20 0.00 3308527 33 85.24 5439695 1809382 2521 245.51 5.42 0.00 4.19 0.00 3316778 33 87.54 5416028 1805835 2256 245.51 5.42 0.00 4.19 0.00 3317673 33 90.44 5426044 1763056 2332 245.51 5.42 0.00 4.19 0.00 3314839 33 88.11 5435732 1792218 2499 245.52 5.44 0.00 4.19 0.00 3293228 33 91.84 5426301 1668597 2121 245.52 Similarly, netperf reports 230Mb/s before the patch, and 270Mb/s after the patch Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15366
2018-05-18 20:13:34 +00:00
static void
if_destroymulti(epoch_context_t ctx)
{
struct ifmultiaddr *ifma;
ifma = __containerof(ctx, struct ifmultiaddr, ifma_epoch_ctx);
if_freemulti_internal(ifma);
}
void
if_freemulti(struct ifmultiaddr *ifma)
{
KASSERT(ifma->ifma_refcount == 0, ("if_freemulti_epoch: refcount %d",
ifma->ifma_refcount));
epoch_call(net_epoch_preempt, &ifma->ifma_epoch_ctx, if_destroymulti);
}
Protect link layer network interface multicast address list manipulation using ifp->if_addr_mtx: - Initialize if_addr_mtx when ifnet is initialized. - Destroy if_addr_mtx when ifnet is torn down. - Rename ifmaof_ifpforaddr() to if_findmulti(); assert if_addr_mtx. Staticize. - Extract ifmultiaddr allocation and initialization into if_allocmulti(); accept a 'mflags' argument to indicate whether or not sleeping is permitted. This centralizes error handling and address duplication. - Extract ifmultiaddr tear-down and deallocation in if_freemulti(). - Re-structure if_addmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Make use of non-sleeping allocations. Annotate the fact that we only generate routing socket events for explicit address addition, not implicit link layer address addition. - Re-structure if_delmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Annotate the lack of a routing socket event for implicit link layer address removal. - De-spl all and sundry. Problem reported by: Ed Maste <emaste at phaedrus dot sandvine dot ca> MFC after: 1 week
2005-08-02 23:23:26 +00:00
/*
* Register an additional multicast address with a network interface.
*
* - If the address is already present, bump the reference count on the
* address and return.
* - If the address is not link-layer, look up a link layer address.
* - Allocate address structures for one or both addresses, and attach to the
* multicast address list on the interface. If automatically adding a link
* layer address, the protocol address will own a reference to the link
* layer address, to be freed when it is freed.
* - Notify the network device driver of an addition to the multicast address
* list.
*
* 'sa' points to caller-owned memory with the desired multicast address.
*
* 'retifma' will be used to return a pointer to the resulting multicast
* address reference, if desired.
*/
int
if_addmulti(struct ifnet *ifp, struct sockaddr *sa,
struct ifmultiaddr **retifma)
{
struct ifmultiaddr *ifma, *ll_ifma;
struct sockaddr *llsa;
Simplify filling sockaddr_dl structure for if_resolvemulti() callback providers. link_init_sdl() function can be used to fill most of the parameters. Use caller stack instead of allocation / freing memory for each request. Do not drop support for extra-long (probably non-existing) link-layer protocols by introducing link_alloc_sdl() (used by if_resolvemulti() callback) and link_free_sdl() (used by caller). Since this change breaks KBI, MFC requires slightly different approach (link_init_sdl() auto-allocating buffer if necessary to handle cases with unmodified if_resolvemulti() callers). MFC after: 2 weeks
2014-01-18 23:24:51 +00:00
struct sockaddr_dl sdl;
Protect link layer network interface multicast address list manipulation using ifp->if_addr_mtx: - Initialize if_addr_mtx when ifnet is initialized. - Destroy if_addr_mtx when ifnet is torn down. - Rename ifmaof_ifpforaddr() to if_findmulti(); assert if_addr_mtx. Staticize. - Extract ifmultiaddr allocation and initialization into if_allocmulti(); accept a 'mflags' argument to indicate whether or not sleeping is permitted. This centralizes error handling and address duplication. - Extract ifmultiaddr tear-down and deallocation in if_freemulti(). - Re-structure if_addmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Make use of non-sleeping allocations. Annotate the fact that we only generate routing socket events for explicit address addition, not implicit link layer address addition. - Re-structure if_delmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Annotate the lack of a routing socket event for implicit link layer address removal. - De-spl all and sundry. Problem reported by: Ed Maste <emaste at phaedrus dot sandvine dot ca> MFC after: 1 week
2005-08-02 23:23:26 +00:00
int error;
Checkpoint the beginnings of the new kernel interface for multicast group memberships. This is not actually operative at the moment (a lot of other code still needs to be changed), but this seemed like a useful reference point to check in so that others (i.e. Bill Fenner) have fair warning of where we are going.
1997-01-07 19:15:32 +00:00
Separate list manipulation locking from state change in multicast Multicast incorrectly calls in to drivers with a mutex held causing drivers to have to go through all manner of contortions to use a non sleepable lock. Serialize multicast updates instead. Submitted by: mmacy <mmacy@mattmacy.io> Reviewed by: shurd, sbruno Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D14969
2018-05-02 19:36:29 +00:00
#ifdef INET
IN_MULTI_LIST_UNLOCK_ASSERT();
#endif
#ifdef INET6
IN6_MULTI_LIST_UNLOCK_ASSERT();
#endif
Don't add an item to the multicast linked list if it's already on the list.
1997-07-07 17:36:06 +00:00
/*
Protect link layer network interface multicast address list manipulation using ifp->if_addr_mtx: - Initialize if_addr_mtx when ifnet is initialized. - Destroy if_addr_mtx when ifnet is torn down. - Rename ifmaof_ifpforaddr() to if_findmulti(); assert if_addr_mtx. Staticize. - Extract ifmultiaddr allocation and initialization into if_allocmulti(); accept a 'mflags' argument to indicate whether or not sleeping is permitted. This centralizes error handling and address duplication. - Extract ifmultiaddr tear-down and deallocation in if_freemulti(). - Re-structure if_addmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Make use of non-sleeping allocations. Annotate the fact that we only generate routing socket events for explicit address addition, not implicit link layer address addition. - Re-structure if_delmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Annotate the lack of a routing socket event for implicit link layer address removal. - De-spl all and sundry. Problem reported by: Ed Maste <emaste at phaedrus dot sandvine dot ca> MFC after: 1 week
2005-08-02 23:23:26 +00:00
* If the address is already present, return a new reference to it;
* otherwise, allocate storage and set up a new address.
Don't add an item to the multicast linked list if it's already on the list.
1997-07-07 17:36:06 +00:00
*/
Convert all users of IF_ADDR_LOCK to use new locking macros that specify either a read lock or write lock. Reviewed by: bz MFC after: 2 weeks
2012-01-05 19:00:36 +00:00
IF_ADDR_WLOCK(ifp);
Protect link layer network interface multicast address list manipulation using ifp->if_addr_mtx: - Initialize if_addr_mtx when ifnet is initialized. - Destroy if_addr_mtx when ifnet is torn down. - Rename ifmaof_ifpforaddr() to if_findmulti(); assert if_addr_mtx. Staticize. - Extract ifmultiaddr allocation and initialization into if_allocmulti(); accept a 'mflags' argument to indicate whether or not sleeping is permitted. This centralizes error handling and address duplication. - Extract ifmultiaddr tear-down and deallocation in if_freemulti(). - Re-structure if_addmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Make use of non-sleeping allocations. Annotate the fact that we only generate routing socket events for explicit address addition, not implicit link layer address addition. - Re-structure if_delmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Annotate the lack of a routing socket event for implicit link layer address removal. - De-spl all and sundry. Problem reported by: Ed Maste <emaste at phaedrus dot sandvine dot ca> MFC after: 1 week
2005-08-02 23:23:26 +00:00
ifma = if_findmulti(ifp, sa);
if (ifma != NULL) {
ifma->ifma_refcount++;
if (retifma != NULL)
*retifma = ifma;
Convert all users of IF_ADDR_LOCK to use new locking macros that specify either a read lock or write lock. Reviewed by: bz MFC after: 2 weeks
2012-01-05 19:00:36 +00:00
IF_ADDR_WUNLOCK(ifp);
Protect link layer network interface multicast address list manipulation using ifp->if_addr_mtx: - Initialize if_addr_mtx when ifnet is initialized. - Destroy if_addr_mtx when ifnet is torn down. - Rename ifmaof_ifpforaddr() to if_findmulti(); assert if_addr_mtx. Staticize. - Extract ifmultiaddr allocation and initialization into if_allocmulti(); accept a 'mflags' argument to indicate whether or not sleeping is permitted. This centralizes error handling and address duplication. - Extract ifmultiaddr tear-down and deallocation in if_freemulti(). - Re-structure if_addmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Make use of non-sleeping allocations. Annotate the fact that we only generate routing socket events for explicit address addition, not implicit link layer address addition. - Re-structure if_delmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Annotate the lack of a routing socket event for implicit link layer address removal. - De-spl all and sundry. Problem reported by: Ed Maste <emaste at phaedrus dot sandvine dot ca> MFC after: 1 week
2005-08-02 23:23:26 +00:00
return (0);
Checkpoint the beginnings of the new kernel interface for multicast group memberships. This is not actually operative at the moment (a lot of other code still needs to be changed), but this seemed like a useful reference point to check in so that others (i.e. Bill Fenner) have fair warning of where we are going.
1997-01-07 19:15:32 +00:00
}
/*
Protect link layer network interface multicast address list manipulation using ifp->if_addr_mtx: - Initialize if_addr_mtx when ifnet is initialized. - Destroy if_addr_mtx when ifnet is torn down. - Rename ifmaof_ifpforaddr() to if_findmulti(); assert if_addr_mtx. Staticize. - Extract ifmultiaddr allocation and initialization into if_allocmulti(); accept a 'mflags' argument to indicate whether or not sleeping is permitted. This centralizes error handling and address duplication. - Extract ifmultiaddr tear-down and deallocation in if_freemulti(). - Re-structure if_addmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Make use of non-sleeping allocations. Annotate the fact that we only generate routing socket events for explicit address addition, not implicit link layer address addition. - Re-structure if_delmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Annotate the lack of a routing socket event for implicit link layer address removal. - De-spl all and sundry. Problem reported by: Ed Maste <emaste at phaedrus dot sandvine dot ca> MFC after: 1 week
2005-08-02 23:23:26 +00:00
* The address isn't already present; resolve the protocol address
* into a link layer address, and then look that up, bump its
Simplify filling sockaddr_dl structure for if_resolvemulti() callback providers. link_init_sdl() function can be used to fill most of the parameters. Use caller stack instead of allocation / freing memory for each request. Do not drop support for extra-long (probably non-existing) link-layer protocols by introducing link_alloc_sdl() (used by if_resolvemulti() callback) and link_free_sdl() (used by caller). Since this change breaks KBI, MFC requires slightly different approach (link_init_sdl() auto-allocating buffer if necessary to handle cases with unmodified if_resolvemulti() callers). MFC after: 2 weeks
2014-01-18 23:24:51 +00:00
* refcount or allocate an ifma for that also.
* Most link layer resolving functions returns address data which
* fits inside default sockaddr_dl structure. However callback
* can allocate another sockaddr structure, in that case we need to
* free it later.
Checkpoint the beginnings of the new kernel interface for multicast group memberships. This is not actually operative at the moment (a lot of other code still needs to be changed), but this seemed like a useful reference point to check in so that others (i.e. Bill Fenner) have fair warning of where we are going.
1997-01-07 19:15:32 +00:00
*/
Protect link layer network interface multicast address list manipulation using ifp->if_addr_mtx: - Initialize if_addr_mtx when ifnet is initialized. - Destroy if_addr_mtx when ifnet is torn down. - Rename ifmaof_ifpforaddr() to if_findmulti(); assert if_addr_mtx. Staticize. - Extract ifmultiaddr allocation and initialization into if_allocmulti(); accept a 'mflags' argument to indicate whether or not sleeping is permitted. This centralizes error handling and address duplication. - Extract ifmultiaddr tear-down and deallocation in if_freemulti(). - Re-structure if_addmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Make use of non-sleeping allocations. Annotate the fact that we only generate routing socket events for explicit address addition, not implicit link layer address addition. - Re-structure if_delmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Annotate the lack of a routing socket event for implicit link layer address removal. - De-spl all and sundry. Problem reported by: Ed Maste <emaste at phaedrus dot sandvine dot ca> MFC after: 1 week
2005-08-02 23:23:26 +00:00
llsa = NULL;
ll_ifma = NULL;
In multicast routines: Compare pointers with NULL rather than treating them as booleans. Compare pointers with NULL rather than 0 to make it more clear they are pointers. Assign pointers value of NULL rather than 0 to make it more clear they are pointers. MFC after: 3 days
2005-07-19 10:12:58 +00:00
if (ifp->if_resolvemulti != NULL) {
Simplify filling sockaddr_dl structure for if_resolvemulti() callback providers. link_init_sdl() function can be used to fill most of the parameters. Use caller stack instead of allocation / freing memory for each request. Do not drop support for extra-long (probably non-existing) link-layer protocols by introducing link_alloc_sdl() (used by if_resolvemulti() callback) and link_free_sdl() (used by caller). Since this change breaks KBI, MFC requires slightly different approach (link_init_sdl() auto-allocating buffer if necessary to handle cases with unmodified if_resolvemulti() callers). MFC after: 2 weeks
2014-01-18 23:24:51 +00:00
/* Provide called function with buffer size information */
sdl.sdl_len = sizeof(sdl);
llsa = (struct sockaddr *)&sdl;
Checkpoint the beginnings of the new kernel interface for multicast group memberships. This is not actually operative at the moment (a lot of other code still needs to be changed), but this seemed like a useful reference point to check in so that others (i.e. Bill Fenner) have fair warning of where we are going.
1997-01-07 19:15:32 +00:00
error = ifp->if_resolvemulti(ifp, &llsa, sa);
Protect link layer network interface multicast address list manipulation using ifp->if_addr_mtx: - Initialize if_addr_mtx when ifnet is initialized. - Destroy if_addr_mtx when ifnet is torn down. - Rename ifmaof_ifpforaddr() to if_findmulti(); assert if_addr_mtx. Staticize. - Extract ifmultiaddr allocation and initialization into if_allocmulti(); accept a 'mflags' argument to indicate whether or not sleeping is permitted. This centralizes error handling and address duplication. - Extract ifmultiaddr tear-down and deallocation in if_freemulti(). - Re-structure if_addmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Make use of non-sleeping allocations. Annotate the fact that we only generate routing socket events for explicit address addition, not implicit link layer address addition. - Re-structure if_delmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Annotate the lack of a routing socket event for implicit link layer address removal. - De-spl all and sundry. Problem reported by: Ed Maste <emaste at phaedrus dot sandvine dot ca> MFC after: 1 week
2005-08-02 23:23:26 +00:00
if (error)
goto unlock_out;
Checkpoint the beginnings of the new kernel interface for multicast group memberships. This is not actually operative at the moment (a lot of other code still needs to be changed), but this seemed like a useful reference point to check in so that others (i.e. Bill Fenner) have fair warning of where we are going.
1997-01-07 19:15:32 +00:00
}
Protect link layer network interface multicast address list manipulation using ifp->if_addr_mtx: - Initialize if_addr_mtx when ifnet is initialized. - Destroy if_addr_mtx when ifnet is torn down. - Rename ifmaof_ifpforaddr() to if_findmulti(); assert if_addr_mtx. Staticize. - Extract ifmultiaddr allocation and initialization into if_allocmulti(); accept a 'mflags' argument to indicate whether or not sleeping is permitted. This centralizes error handling and address duplication. - Extract ifmultiaddr tear-down and deallocation in if_freemulti(). - Re-structure if_addmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Make use of non-sleeping allocations. Annotate the fact that we only generate routing socket events for explicit address addition, not implicit link layer address addition. - Re-structure if_delmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Annotate the lack of a routing socket event for implicit link layer address removal. - De-spl all and sundry. Problem reported by: Ed Maste <emaste at phaedrus dot sandvine dot ca> MFC after: 1 week
2005-08-02 23:23:26 +00:00
/*
* Allocate the new address. Don't hook it up yet, as we may also
* need to allocate a link layer multicast address.
*/
ifma = if_allocmulti(ifp, sa, llsa, M_NOWAIT);
if (ifma == NULL) {
error = ENOMEM;
goto free_llsa_out;
}
Checkpoint the beginnings of the new kernel interface for multicast group memberships. This is not actually operative at the moment (a lot of other code still needs to be changed), but this seemed like a useful reference point to check in so that others (i.e. Bill Fenner) have fair warning of where we are going.
1997-01-07 19:15:32 +00:00
Protect link layer network interface multicast address list manipulation using ifp->if_addr_mtx: - Initialize if_addr_mtx when ifnet is initialized. - Destroy if_addr_mtx when ifnet is torn down. - Rename ifmaof_ifpforaddr() to if_findmulti(); assert if_addr_mtx. Staticize. - Extract ifmultiaddr allocation and initialization into if_allocmulti(); accept a 'mflags' argument to indicate whether or not sleeping is permitted. This centralizes error handling and address duplication. - Extract ifmultiaddr tear-down and deallocation in if_freemulti(). - Re-structure if_addmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Make use of non-sleeping allocations. Annotate the fact that we only generate routing socket events for explicit address addition, not implicit link layer address addition. - Re-structure if_delmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Annotate the lack of a routing socket event for implicit link layer address removal. - De-spl all and sundry. Problem reported by: Ed Maste <emaste at phaedrus dot sandvine dot ca> MFC after: 1 week
2005-08-02 23:23:26 +00:00
/*
* If a link layer address is found, we'll need to see if it's
* already present in the address list, or allocate is as well.
* When this block finishes, the link layer address will be on the
* list.
*/
if (llsa != NULL) {
ll_ifma = if_findmulti(ifp, llsa);
if (ll_ifma == NULL) {
ll_ifma = if_allocmulti(ifp, llsa, NULL, M_NOWAIT);
if (ll_ifma == NULL) {
Implement reference counting for ifmultiaddr, in_multi, and in6_multi structures. Detect when ifnet instances are detached from the network stack and perform appropriate cleanup to prevent memory leaks. This has been implemented in such a way as to be backwards ABI compatible. Kernel consumers are changed to use if_delmulti_ifma(); in_delmulti() is unable to detect interface removal by design, as it performs searches on structures which are removed with the interface. With this architectural change, the panics FreeBSD users have experienced with carp and pfsync should be resolved. Obtained from: p4 branch bms_netdev Reviewed by: andre Sponsored by: Garance A Drosehn Idea from: NetBSD MFC after: 1 month
2007-03-20 00:36:10 +00:00
--ifma->ifma_refcount;
Protect link layer network interface multicast address list manipulation using ifp->if_addr_mtx: - Initialize if_addr_mtx when ifnet is initialized. - Destroy if_addr_mtx when ifnet is torn down. - Rename ifmaof_ifpforaddr() to if_findmulti(); assert if_addr_mtx. Staticize. - Extract ifmultiaddr allocation and initialization into if_allocmulti(); accept a 'mflags' argument to indicate whether or not sleeping is permitted. This centralizes error handling and address duplication. - Extract ifmultiaddr tear-down and deallocation in if_freemulti(). - Re-structure if_addmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Make use of non-sleeping allocations. Annotate the fact that we only generate routing socket events for explicit address addition, not implicit link layer address addition. - Re-structure if_delmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Annotate the lack of a routing socket event for implicit link layer address removal. - De-spl all and sundry. Problem reported by: Ed Maste <emaste at phaedrus dot sandvine dot ca> MFC after: 1 week
2005-08-02 23:23:26 +00:00
if_freemulti(ifma);
error = ENOMEM;
goto free_llsa_out;
}
Fix in6_multi double free This is actually several different bugs: - The code is not designed to handle inpcb deletion after interface deletion - add reference for inpcb membership - The multicast address has to be removed from interface lists when the refcount goes to zero OR when the interface goes away - decouple list disconnect from refcount (v6 only for now) - ifmultiaddr can exist past being on interface lists - add flag for tracking whether or not it's enqueued - deferring freeing moptions makes the incpb cleanup code simpler but opens the door wider still to races - call inp_gcmoptions synchronously after dropping the the inpcb lock Fundamentally multicast needs a rewrite - but keep applying band-aids for now. Tested by: kp Reported by: novel, kp, lwhsu
2018-08-15 20:23:08 +00:00
ll_ifma->ifma_flags |= IFMA_F_ENQUEUED;
ifnet: Replace if_addr_lock rwlock with epoch + mutex Run on LLNW canaries and tested by pho@ gallatin: Using a 14-core, 28-HTT single socket E5-2697 v3 with a 40GbE MLX5 based ConnectX 4-LX NIC, I see an almost 12% improvement in received packet rate, and a larger improvement in bytes delivered all the way to userspace. When the host receiving 64 streams of netperf -H $DUT -t UDP_STREAM -- -m 1, I see, using nstat -I mce0 1 before the patch: InMpps OMpps InGbs OGbs err TCP Est %CPU syscalls csw irq GBfree 4.98 0.00 4.42 0.00 4235592 33 83.80 4720653 2149771 1235 247.32 4.73 0.00 4.20 0.00 4025260 33 82.99 4724900 2139833 1204 247.32 4.72 0.00 4.20 0.00 4035252 33 82.14 4719162 2132023 1264 247.32 4.71 0.00 4.21 0.00 4073206 33 83.68 4744973 2123317 1347 247.32 4.72 0.00 4.21 0.00 4061118 33 80.82 4713615 2188091 1490 247.32 4.72 0.00 4.21 0.00 4051675 33 85.29 4727399 2109011 1205 247.32 4.73 0.00 4.21 0.00 4039056 33 84.65 4724735 2102603 1053 247.32 After the patch InMpps OMpps InGbs OGbs err TCP Est %CPU syscalls csw irq GBfree 5.43 0.00 4.20 0.00 3313143 33 84.96 5434214 1900162 2656 245.51 5.43 0.00 4.20 0.00 3308527 33 85.24 5439695 1809382 2521 245.51 5.42 0.00 4.19 0.00 3316778 33 87.54 5416028 1805835 2256 245.51 5.42 0.00 4.19 0.00 3317673 33 90.44 5426044 1763056 2332 245.51 5.42 0.00 4.19 0.00 3314839 33 88.11 5435732 1792218 2499 245.52 5.44 0.00 4.19 0.00 3293228 33 91.84 5426301 1668597 2121 245.52 Similarly, netperf reports 230Mb/s before the patch, and 270Mb/s after the patch Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15366
2018-05-18 20:13:34 +00:00
CK_STAILQ_INSERT_HEAD(&ifp->if_multiaddrs, ll_ifma,
Protect link layer network interface multicast address list manipulation using ifp->if_addr_mtx: - Initialize if_addr_mtx when ifnet is initialized. - Destroy if_addr_mtx when ifnet is torn down. - Rename ifmaof_ifpforaddr() to if_findmulti(); assert if_addr_mtx. Staticize. - Extract ifmultiaddr allocation and initialization into if_allocmulti(); accept a 'mflags' argument to indicate whether or not sleeping is permitted. This centralizes error handling and address duplication. - Extract ifmultiaddr tear-down and deallocation in if_freemulti(). - Re-structure if_addmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Make use of non-sleeping allocations. Annotate the fact that we only generate routing socket events for explicit address addition, not implicit link layer address addition. - Re-structure if_delmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Annotate the lack of a routing socket event for implicit link layer address removal. - De-spl all and sundry. Problem reported by: Ed Maste <emaste at phaedrus dot sandvine dot ca> MFC after: 1 week
2005-08-02 23:23:26 +00:00
ifma_link);
} else
ll_ifma->ifma_refcount++;
Implement reference counting for ifmultiaddr, in_multi, and in6_multi structures. Detect when ifnet instances are detached from the network stack and perform appropriate cleanup to prevent memory leaks. This has been implemented in such a way as to be backwards ABI compatible. Kernel consumers are changed to use if_delmulti_ifma(); in_delmulti() is unable to detect interface removal by design, as it performs searches on structures which are removed with the interface. With this architectural change, the panics FreeBSD users have experienced with carp and pfsync should be resolved. Obtained from: p4 branch bms_netdev Reviewed by: andre Sponsored by: Garance A Drosehn Idea from: NetBSD MFC after: 1 month
2007-03-20 00:36:10 +00:00
ifma->ifma_llifma = ll_ifma;
Protect link layer network interface multicast address list manipulation using ifp->if_addr_mtx: - Initialize if_addr_mtx when ifnet is initialized. - Destroy if_addr_mtx when ifnet is torn down. - Rename ifmaof_ifpforaddr() to if_findmulti(); assert if_addr_mtx. Staticize. - Extract ifmultiaddr allocation and initialization into if_allocmulti(); accept a 'mflags' argument to indicate whether or not sleeping is permitted. This centralizes error handling and address duplication. - Extract ifmultiaddr tear-down and deallocation in if_freemulti(). - Re-structure if_addmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Make use of non-sleeping allocations. Annotate the fact that we only generate routing socket events for explicit address addition, not implicit link layer address addition. - Re-structure if_delmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Annotate the lack of a routing socket event for implicit link layer address removal. - De-spl all and sundry. Problem reported by: Ed Maste <emaste at phaedrus dot sandvine dot ca> MFC after: 1 week
2005-08-02 23:23:26 +00:00
}
Fix a few oversights in the new multicast membership interface.
1997-01-08 13:20:25 +00:00
Checkpoint the beginnings of the new kernel interface for multicast group memberships. This is not actually operative at the moment (a lot of other code still needs to be changed), but this seemed like a useful reference point to check in so that others (i.e. Bill Fenner) have fair warning of where we are going.
1997-01-07 19:15:32 +00:00
/*
Protect link layer network interface multicast address list manipulation using ifp->if_addr_mtx: - Initialize if_addr_mtx when ifnet is initialized. - Destroy if_addr_mtx when ifnet is torn down. - Rename ifmaof_ifpforaddr() to if_findmulti(); assert if_addr_mtx. Staticize. - Extract ifmultiaddr allocation and initialization into if_allocmulti(); accept a 'mflags' argument to indicate whether or not sleeping is permitted. This centralizes error handling and address duplication. - Extract ifmultiaddr tear-down and deallocation in if_freemulti(). - Re-structure if_addmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Make use of non-sleeping allocations. Annotate the fact that we only generate routing socket events for explicit address addition, not implicit link layer address addition. - Re-structure if_delmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Annotate the lack of a routing socket event for implicit link layer address removal. - De-spl all and sundry. Problem reported by: Ed Maste <emaste at phaedrus dot sandvine dot ca> MFC after: 1 week
2005-08-02 23:23:26 +00:00
* We now have a new multicast address, ifma, and possibly a new or
* referenced link layer address. Add the primary address to the
* ifnet address list.
Checkpoint the beginnings of the new kernel interface for multicast group memberships. This is not actually operative at the moment (a lot of other code still needs to be changed), but this seemed like a useful reference point to check in so that others (i.e. Bill Fenner) have fair warning of where we are going.
1997-01-07 19:15:32 +00:00
*/
Fix in6_multi double free This is actually several different bugs: - The code is not designed to handle inpcb deletion after interface deletion - add reference for inpcb membership - The multicast address has to be removed from interface lists when the refcount goes to zero OR when the interface goes away - decouple list disconnect from refcount (v6 only for now) - ifmultiaddr can exist past being on interface lists - add flag for tracking whether or not it's enqueued - deferring freeing moptions makes the incpb cleanup code simpler but opens the door wider still to races - call inp_gcmoptions synchronously after dropping the the inpcb lock Fundamentally multicast needs a rewrite - but keep applying band-aids for now. Tested by: kp Reported by: novel, kp, lwhsu
2018-08-15 20:23:08 +00:00
ifma->ifma_flags |= IFMA_F_ENQUEUED;
ifnet: Replace if_addr_lock rwlock with epoch + mutex Run on LLNW canaries and tested by pho@ gallatin: Using a 14-core, 28-HTT single socket E5-2697 v3 with a 40GbE MLX5 based ConnectX 4-LX NIC, I see an almost 12% improvement in received packet rate, and a larger improvement in bytes delivered all the way to userspace. When the host receiving 64 streams of netperf -H $DUT -t UDP_STREAM -- -m 1, I see, using nstat -I mce0 1 before the patch: InMpps OMpps InGbs OGbs err TCP Est %CPU syscalls csw irq GBfree 4.98 0.00 4.42 0.00 4235592 33 83.80 4720653 2149771 1235 247.32 4.73 0.00 4.20 0.00 4025260 33 82.99 4724900 2139833 1204 247.32 4.72 0.00 4.20 0.00 4035252 33 82.14 4719162 2132023 1264 247.32 4.71 0.00 4.21 0.00 4073206 33 83.68 4744973 2123317 1347 247.32 4.72 0.00 4.21 0.00 4061118 33 80.82 4713615 2188091 1490 247.32 4.72 0.00 4.21 0.00 4051675 33 85.29 4727399 2109011 1205 247.32 4.73 0.00 4.21 0.00 4039056 33 84.65 4724735 2102603 1053 247.32 After the patch InMpps OMpps InGbs OGbs err TCP Est %CPU syscalls csw irq GBfree 5.43 0.00 4.20 0.00 3313143 33 84.96 5434214 1900162 2656 245.51 5.43 0.00 4.20 0.00 3308527 33 85.24 5439695 1809382 2521 245.51 5.42 0.00 4.19 0.00 3316778 33 87.54 5416028 1805835 2256 245.51 5.42 0.00 4.19 0.00 3317673 33 90.44 5426044 1763056 2332 245.51 5.42 0.00 4.19 0.00 3314839 33 88.11 5435732 1792218 2499 245.52 5.44 0.00 4.19 0.00 3293228 33 91.84 5426301 1668597 2121 245.52 Similarly, netperf reports 230Mb/s before the patch, and 270Mb/s after the patch Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15366
2018-05-18 20:13:34 +00:00
CK_STAILQ_INSERT_HEAD(&ifp->if_multiaddrs, ifma, ifma_link);
Protect link layer network interface multicast address list manipulation using ifp->if_addr_mtx: - Initialize if_addr_mtx when ifnet is initialized. - Destroy if_addr_mtx when ifnet is torn down. - Rename ifmaof_ifpforaddr() to if_findmulti(); assert if_addr_mtx. Staticize. - Extract ifmultiaddr allocation and initialization into if_allocmulti(); accept a 'mflags' argument to indicate whether or not sleeping is permitted. This centralizes error handling and address duplication. - Extract ifmultiaddr tear-down and deallocation in if_freemulti(). - Re-structure if_addmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Make use of non-sleeping allocations. Annotate the fact that we only generate routing socket events for explicit address addition, not implicit link layer address addition. - Re-structure if_delmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Annotate the lack of a routing socket event for implicit link layer address removal. - De-spl all and sundry. Problem reported by: Ed Maste <emaste at phaedrus dot sandvine dot ca> MFC after: 1 week
2005-08-02 23:23:26 +00:00
Check retifma for NULL before using it. PR: kern/9391 Submitted by: Assar Westerlund <assar@sics.se> MFC after: 3 days
2002-07-02 08:23:00 +00:00
if (retifma != NULL)
*retifma = ifma;
Checkpoint the beginnings of the new kernel interface for multicast group memberships. This is not actually operative at the moment (a lot of other code still needs to be changed), but this seemed like a useful reference point to check in so that others (i.e. Bill Fenner) have fair warning of where we are going.
1997-01-07 19:15:32 +00:00
Protect link layer network interface multicast address list manipulation using ifp->if_addr_mtx: - Initialize if_addr_mtx when ifnet is initialized. - Destroy if_addr_mtx when ifnet is torn down. - Rename ifmaof_ifpforaddr() to if_findmulti(); assert if_addr_mtx. Staticize. - Extract ifmultiaddr allocation and initialization into if_allocmulti(); accept a 'mflags' argument to indicate whether or not sleeping is permitted. This centralizes error handling and address duplication. - Extract ifmultiaddr tear-down and deallocation in if_freemulti(). - Re-structure if_addmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Make use of non-sleeping allocations. Annotate the fact that we only generate routing socket events for explicit address addition, not implicit link layer address addition. - Re-structure if_delmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Annotate the lack of a routing socket event for implicit link layer address removal. - De-spl all and sundry. Problem reported by: Ed Maste <emaste at phaedrus dot sandvine dot ca> MFC after: 1 week
2005-08-02 23:23:26 +00:00
/*
* Must generate the message while holding the lock so that 'ifma'
* pointer is still valid.
*/
rt_newmaddrmsg(RTM_NEWMADDR, ifma);
Convert all users of IF_ADDR_LOCK to use new locking macros that specify either a read lock or write lock. Reviewed by: bz MFC after: 2 weeks
2012-01-05 19:00:36 +00:00
IF_ADDR_WUNLOCK(ifp);
Protect link layer network interface multicast address list manipulation using ifp->if_addr_mtx: - Initialize if_addr_mtx when ifnet is initialized. - Destroy if_addr_mtx when ifnet is torn down. - Rename ifmaof_ifpforaddr() to if_findmulti(); assert if_addr_mtx. Staticize. - Extract ifmultiaddr allocation and initialization into if_allocmulti(); accept a 'mflags' argument to indicate whether or not sleeping is permitted. This centralizes error handling and address duplication. - Extract ifmultiaddr tear-down and deallocation in if_freemulti(). - Re-structure if_addmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Make use of non-sleeping allocations. Annotate the fact that we only generate routing socket events for explicit address addition, not implicit link layer address addition. - Re-structure if_delmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Annotate the lack of a routing socket event for implicit link layer address removal. - De-spl all and sundry. Problem reported by: Ed Maste <emaste at phaedrus dot sandvine dot ca> MFC after: 1 week
2005-08-02 23:23:26 +00:00
Checkpoint the beginnings of the new kernel interface for multicast group memberships. This is not actually operative at the moment (a lot of other code still needs to be changed), but this seemed like a useful reference point to check in so that others (i.e. Bill Fenner) have fair warning of where we are going.
1997-01-07 19:15:32 +00:00
/*
* We are certain we have added something, so call down to the
* interface to let them know about it.
*/
In multicast routines: Compare pointers with NULL rather than treating them as booleans. Compare pointers with NULL rather than 0 to make it more clear they are pointers. Assign pointers value of NULL rather than 0 to make it more clear they are pointers. MFC after: 3 days
2005-07-19 10:12:58 +00:00
if (ifp->if_ioctl != NULL) {
MFp4: - Introduce a helper function if_setflag() containing the code common to ifpromisc() and if_allmulti() instead of duplicating the code poorly, with different bugs. - Call ifp->if_ioctl() in a consistent way: always use more compatible C syntax and check whether ifp->if_ioctl is not NULL prior to the call. MFC after: 1 month
2005-07-14 13:56:51 +00:00
(void) (*ifp->if_ioctl)(ifp, SIOCADDMULTI, 0);
}
Checkpoint the beginnings of the new kernel interface for multicast group memberships. This is not actually operative at the moment (a lot of other code still needs to be changed), but this seemed like a useful reference point to check in so that others (i.e. Bill Fenner) have fair warning of where we are going.
1997-01-07 19:15:32 +00:00
Simplify filling sockaddr_dl structure for if_resolvemulti() callback providers. link_init_sdl() function can be used to fill most of the parameters. Use caller stack instead of allocation / freing memory for each request. Do not drop support for extra-long (probably non-existing) link-layer protocols by introducing link_alloc_sdl() (used by if_resolvemulti() callback) and link_free_sdl() (used by caller). Since this change breaks KBI, MFC requires slightly different approach (link_init_sdl() auto-allocating buffer if necessary to handle cases with unmodified if_resolvemulti() callers). MFC after: 2 weeks
2014-01-18 23:24:51 +00:00
if ((llsa != NULL) && (llsa != (struct sockaddr *)&sdl))
link_free_sdl(llsa);
Protect link layer network interface multicast address list manipulation using ifp->if_addr_mtx: - Initialize if_addr_mtx when ifnet is initialized. - Destroy if_addr_mtx when ifnet is torn down. - Rename ifmaof_ifpforaddr() to if_findmulti(); assert if_addr_mtx. Staticize. - Extract ifmultiaddr allocation and initialization into if_allocmulti(); accept a 'mflags' argument to indicate whether or not sleeping is permitted. This centralizes error handling and address duplication. - Extract ifmultiaddr tear-down and deallocation in if_freemulti(). - Re-structure if_addmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Make use of non-sleeping allocations. Annotate the fact that we only generate routing socket events for explicit address addition, not implicit link layer address addition. - Re-structure if_delmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Annotate the lack of a routing socket event for implicit link layer address removal. - De-spl all and sundry. Problem reported by: Ed Maste <emaste at phaedrus dot sandvine dot ca> MFC after: 1 week
2005-08-02 23:23:26 +00:00
return (0);
free_llsa_out:
Simplify filling sockaddr_dl structure for if_resolvemulti() callback providers. link_init_sdl() function can be used to fill most of the parameters. Use caller stack instead of allocation / freing memory for each request. Do not drop support for extra-long (probably non-existing) link-layer protocols by introducing link_alloc_sdl() (used by if_resolvemulti() callback) and link_free_sdl() (used by caller). Since this change breaks KBI, MFC requires slightly different approach (link_init_sdl() auto-allocating buffer if necessary to handle cases with unmodified if_resolvemulti() callers). MFC after: 2 weeks
2014-01-18 23:24:51 +00:00
if ((llsa != NULL) && (llsa != (struct sockaddr *)&sdl))
link_free_sdl(llsa);
Protect link layer network interface multicast address list manipulation using ifp->if_addr_mtx: - Initialize if_addr_mtx when ifnet is initialized. - Destroy if_addr_mtx when ifnet is torn down. - Rename ifmaof_ifpforaddr() to if_findmulti(); assert if_addr_mtx. Staticize. - Extract ifmultiaddr allocation and initialization into if_allocmulti(); accept a 'mflags' argument to indicate whether or not sleeping is permitted. This centralizes error handling and address duplication. - Extract ifmultiaddr tear-down and deallocation in if_freemulti(). - Re-structure if_addmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Make use of non-sleeping allocations. Annotate the fact that we only generate routing socket events for explicit address addition, not implicit link layer address addition. - Re-structure if_delmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Annotate the lack of a routing socket event for implicit link layer address removal. - De-spl all and sundry. Problem reported by: Ed Maste <emaste at phaedrus dot sandvine dot ca> MFC after: 1 week
2005-08-02 23:23:26 +00:00
unlock_out:
Convert all users of IF_ADDR_LOCK to use new locking macros that specify either a read lock or write lock. Reviewed by: bz MFC after: 2 weeks
2012-01-05 19:00:36 +00:00
IF_ADDR_WUNLOCK(ifp);
Protect link layer network interface multicast address list manipulation using ifp->if_addr_mtx: - Initialize if_addr_mtx when ifnet is initialized. - Destroy if_addr_mtx when ifnet is torn down. - Rename ifmaof_ifpforaddr() to if_findmulti(); assert if_addr_mtx. Staticize. - Extract ifmultiaddr allocation and initialization into if_allocmulti(); accept a 'mflags' argument to indicate whether or not sleeping is permitted. This centralizes error handling and address duplication. - Extract ifmultiaddr tear-down and deallocation in if_freemulti(). - Re-structure if_addmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Make use of non-sleeping allocations. Annotate the fact that we only generate routing socket events for explicit address addition, not implicit link layer address addition. - Re-structure if_delmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Annotate the lack of a routing socket event for implicit link layer address removal. - De-spl all and sundry. Problem reported by: Ed Maste <emaste at phaedrus dot sandvine dot ca> MFC after: 1 week
2005-08-02 23:23:26 +00:00
return (error);
Checkpoint the beginnings of the new kernel interface for multicast group memberships. This is not actually operative at the moment (a lot of other code still needs to be changed), but this seemed like a useful reference point to check in so that others (i.e. Bill Fenner) have fair warning of where we are going.
1997-01-07 19:15:32 +00:00
}
/*
Implement reference counting for ifmultiaddr, in_multi, and in6_multi structures. Detect when ifnet instances are detached from the network stack and perform appropriate cleanup to prevent memory leaks. This has been implemented in such a way as to be backwards ABI compatible. Kernel consumers are changed to use if_delmulti_ifma(); in_delmulti() is unable to detect interface removal by design, as it performs searches on structures which are removed with the interface. With this architectural change, the panics FreeBSD users have experienced with carp and pfsync should be resolved. Obtained from: p4 branch bms_netdev Reviewed by: andre Sponsored by: Garance A Drosehn Idea from: NetBSD MFC after: 1 month
2007-03-20 00:36:10 +00:00
* Delete a multicast group membership by network-layer group address.
*
* Returns ENOENT if the entry could not be found. If ifp no longer
* exists, results are undefined. This entry point should only be used
* from subsystems which do appropriate locking to hold ifp for the
* duration of the call.
* Network-layer protocol domains must use if_delmulti_ifma().
Checkpoint the beginnings of the new kernel interface for multicast group memberships. This is not actually operative at the moment (a lot of other code still needs to be changed), but this seemed like a useful reference point to check in so that others (i.e. Bill Fenner) have fair warning of where we are going.
1997-01-07 19:15:32 +00:00
*/
int
Clean up whitespace, remove "register" keyword, ANSIfy. No functional changes.
2003-10-23 13:49:10 +00:00
if_delmulti(struct ifnet *ifp, struct sockaddr *sa)
Checkpoint the beginnings of the new kernel interface for multicast group memberships. This is not actually operative at the moment (a lot of other code still needs to be changed), but this seemed like a useful reference point to check in so that others (i.e. Bill Fenner) have fair warning of where we are going.
1997-01-07 19:15:32 +00:00
{
Implement reference counting for ifmultiaddr, in_multi, and in6_multi structures. Detect when ifnet instances are detached from the network stack and perform appropriate cleanup to prevent memory leaks. This has been implemented in such a way as to be backwards ABI compatible. Kernel consumers are changed to use if_delmulti_ifma(); in_delmulti() is unable to detect interface removal by design, as it performs searches on structures which are removed with the interface. With this architectural change, the panics FreeBSD users have experienced with carp and pfsync should be resolved. Obtained from: p4 branch bms_netdev Reviewed by: andre Sponsored by: Garance A Drosehn Idea from: NetBSD MFC after: 1 month
2007-03-20 00:36:10 +00:00
struct ifmultiaddr *ifma;
int lastref;
if_delmulti() is never called without ifp argument, assert this instead of doing a useless search through interfaces.
2019-10-14 21:18:37 +00:00
KASSERT(ifp, ("%s: NULL ifp", __func__));
Checkpoint the beginnings of the new kernel interface for multicast group memberships. This is not actually operative at the moment (a lot of other code still needs to be changed), but this seemed like a useful reference point to check in so that others (i.e. Bill Fenner) have fair warning of where we are going.
1997-01-07 19:15:32 +00:00
Convert all users of IF_ADDR_LOCK to use new locking macros that specify either a read lock or write lock. Reviewed by: bz MFC after: 2 weeks
2012-01-05 19:00:36 +00:00
IF_ADDR_WLOCK(ifp);
Implement reference counting for ifmultiaddr, in_multi, and in6_multi structures. Detect when ifnet instances are detached from the network stack and perform appropriate cleanup to prevent memory leaks. This has been implemented in such a way as to be backwards ABI compatible. Kernel consumers are changed to use if_delmulti_ifma(); in_delmulti() is unable to detect interface removal by design, as it performs searches on structures which are removed with the interface. With this architectural change, the panics FreeBSD users have experienced with carp and pfsync should be resolved. Obtained from: p4 branch bms_netdev Reviewed by: andre Sponsored by: Garance A Drosehn Idea from: NetBSD MFC after: 1 month
2007-03-20 00:36:10 +00:00
lastref = 0;
Protect link layer network interface multicast address list manipulation using ifp->if_addr_mtx: - Initialize if_addr_mtx when ifnet is initialized. - Destroy if_addr_mtx when ifnet is torn down. - Rename ifmaof_ifpforaddr() to if_findmulti(); assert if_addr_mtx. Staticize. - Extract ifmultiaddr allocation and initialization into if_allocmulti(); accept a 'mflags' argument to indicate whether or not sleeping is permitted. This centralizes error handling and address duplication. - Extract ifmultiaddr tear-down and deallocation in if_freemulti(). - Re-structure if_addmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Make use of non-sleeping allocations. Annotate the fact that we only generate routing socket events for explicit address addition, not implicit link layer address addition. - Re-structure if_delmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Annotate the lack of a routing socket event for implicit link layer address removal. - De-spl all and sundry. Problem reported by: Ed Maste <emaste at phaedrus dot sandvine dot ca> MFC after: 1 week
2005-08-02 23:23:26 +00:00
ifma = if_findmulti(ifp, sa);
Implement reference counting for ifmultiaddr, in_multi, and in6_multi structures. Detect when ifnet instances are detached from the network stack and perform appropriate cleanup to prevent memory leaks. This has been implemented in such a way as to be backwards ABI compatible. Kernel consumers are changed to use if_delmulti_ifma(); in_delmulti() is unable to detect interface removal by design, as it performs searches on structures which are removed with the interface. With this architectural change, the panics FreeBSD users have experienced with carp and pfsync should be resolved. Obtained from: p4 branch bms_netdev Reviewed by: andre Sponsored by: Garance A Drosehn Idea from: NetBSD MFC after: 1 month
2007-03-20 00:36:10 +00:00
if (ifma != NULL)
lastref = if_delmulti_locked(ifp, ifma, 0);
Convert all users of IF_ADDR_LOCK to use new locking macros that specify either a read lock or write lock. Reviewed by: bz MFC after: 2 weeks
2012-01-05 19:00:36 +00:00
IF_ADDR_WUNLOCK(ifp);
Implement reference counting for ifmultiaddr, in_multi, and in6_multi structures. Detect when ifnet instances are detached from the network stack and perform appropriate cleanup to prevent memory leaks. This has been implemented in such a way as to be backwards ABI compatible. Kernel consumers are changed to use if_delmulti_ifma(); in_delmulti() is unable to detect interface removal by design, as it performs searches on structures which are removed with the interface. With this architectural change, the panics FreeBSD users have experienced with carp and pfsync should be resolved. Obtained from: p4 branch bms_netdev Reviewed by: andre Sponsored by: Garance A Drosehn Idea from: NetBSD MFC after: 1 month
2007-03-20 00:36:10 +00:00
if (ifma == NULL)
return (ENOENT);
if (lastref && ifp->if_ioctl != NULL) {
(void)(*ifp->if_ioctl)(ifp, SIOCDELMULTI, 0);
Protect link layer network interface multicast address list manipulation using ifp->if_addr_mtx: - Initialize if_addr_mtx when ifnet is initialized. - Destroy if_addr_mtx when ifnet is torn down. - Rename ifmaof_ifpforaddr() to if_findmulti(); assert if_addr_mtx. Staticize. - Extract ifmultiaddr allocation and initialization into if_allocmulti(); accept a 'mflags' argument to indicate whether or not sleeping is permitted. This centralizes error handling and address duplication. - Extract ifmultiaddr tear-down and deallocation in if_freemulti(). - Re-structure if_addmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Make use of non-sleeping allocations. Annotate the fact that we only generate routing socket events for explicit address addition, not implicit link layer address addition. - Re-structure if_delmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Annotate the lack of a routing socket event for implicit link layer address removal. - De-spl all and sundry. Problem reported by: Ed Maste <emaste at phaedrus dot sandvine dot ca> MFC after: 1 week
2005-08-02 23:23:26 +00:00
}
Checkpoint the beginnings of the new kernel interface for multicast group memberships. This is not actually operative at the moment (a lot of other code still needs to be changed), but this seemed like a useful reference point to check in so that others (i.e. Bill Fenner) have fair warning of where we are going.
1997-01-07 19:15:32 +00:00
Implement reference counting for ifmultiaddr, in_multi, and in6_multi structures. Detect when ifnet instances are detached from the network stack and perform appropriate cleanup to prevent memory leaks. This has been implemented in such a way as to be backwards ABI compatible. Kernel consumers are changed to use if_delmulti_ifma(); in_delmulti() is unable to detect interface removal by design, as it performs searches on structures which are removed with the interface. With this architectural change, the panics FreeBSD users have experienced with carp and pfsync should be resolved. Obtained from: p4 branch bms_netdev Reviewed by: andre Sponsored by: Garance A Drosehn Idea from: NetBSD MFC after: 1 month
2007-03-20 00:36:10 +00:00
return (0);
}
While flushing the multicast filter of an interface, do not zero the relevant ifmultiaddr structures' reference to the parent interface, unless the parent interface is really detaching. While here, program only link layer multicast filters to a wlan's hardware parent interface. PR: kern/142391, kern/142392 Reviewed by: sam, rpaolo, bms MFC after: 1 week
2010-01-24 16:17:58 +00:00
/*
* Delete all multicast group membership for an interface.
* Should be used to quickly flush all multicast filters.
*/
void
if_delallmulti(struct ifnet *ifp)
{
struct ifmultiaddr *ifma;
struct ifmultiaddr *next;
Convert all users of IF_ADDR_LOCK to use new locking macros that specify either a read lock or write lock. Reviewed by: bz MFC after: 2 weeks
2012-01-05 19:00:36 +00:00
IF_ADDR_WLOCK(ifp);
ifnet: Replace if_addr_lock rwlock with epoch + mutex Run on LLNW canaries and tested by pho@ gallatin: Using a 14-core, 28-HTT single socket E5-2697 v3 with a 40GbE MLX5 based ConnectX 4-LX NIC, I see an almost 12% improvement in received packet rate, and a larger improvement in bytes delivered all the way to userspace. When the host receiving 64 streams of netperf -H $DUT -t UDP_STREAM -- -m 1, I see, using nstat -I mce0 1 before the patch: InMpps OMpps InGbs OGbs err TCP Est %CPU syscalls csw irq GBfree 4.98 0.00 4.42 0.00 4235592 33 83.80 4720653 2149771 1235 247.32 4.73 0.00 4.20 0.00 4025260 33 82.99 4724900 2139833 1204 247.32 4.72 0.00 4.20 0.00 4035252 33 82.14 4719162 2132023 1264 247.32 4.71 0.00 4.21 0.00 4073206 33 83.68 4744973 2123317 1347 247.32 4.72 0.00 4.21 0.00 4061118 33 80.82 4713615 2188091 1490 247.32 4.72 0.00 4.21 0.00 4051675 33 85.29 4727399 2109011 1205 247.32 4.73 0.00 4.21 0.00 4039056 33 84.65 4724735 2102603 1053 247.32 After the patch InMpps OMpps InGbs OGbs err TCP Est %CPU syscalls csw irq GBfree 5.43 0.00 4.20 0.00 3313143 33 84.96 5434214 1900162 2656 245.51 5.43 0.00 4.20 0.00 3308527 33 85.24 5439695 1809382 2521 245.51 5.42 0.00 4.19 0.00 3316778 33 87.54 5416028 1805835 2256 245.51 5.42 0.00 4.19 0.00 3317673 33 90.44 5426044 1763056 2332 245.51 5.42 0.00 4.19 0.00 3314839 33 88.11 5435732 1792218 2499 245.52 5.44 0.00 4.19 0.00 3293228 33 91.84 5426301 1668597 2121 245.52 Similarly, netperf reports 230Mb/s before the patch, and 270Mb/s after the patch Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15366
2018-05-18 20:13:34 +00:00
CK_STAILQ_FOREACH_SAFE(ifma, &ifp->if_multiaddrs, ifma_link, next)
While flushing the multicast filter of an interface, do not zero the relevant ifmultiaddr structures' reference to the parent interface, unless the parent interface is really detaching. While here, program only link layer multicast filters to a wlan's hardware parent interface. PR: kern/142391, kern/142392 Reviewed by: sam, rpaolo, bms MFC after: 1 week
2010-01-24 16:17:58 +00:00
if_delmulti_locked(ifp, ifma, 0);
Convert all users of IF_ADDR_LOCK to use new locking macros that specify either a read lock or write lock. Reviewed by: bz MFC after: 2 weeks
2012-01-05 19:00:36 +00:00
IF_ADDR_WUNLOCK(ifp);
While flushing the multicast filter of an interface, do not zero the relevant ifmultiaddr structures' reference to the parent interface, unless the parent interface is really detaching. While here, program only link layer multicast filters to a wlan's hardware parent interface. PR: kern/142391, kern/142392 Reviewed by: sam, rpaolo, bms MFC after: 1 week
2010-01-24 16:17:58 +00:00
}
r333175 introduced deferred deletion of multicast addresses in order to permit the driver ioctl to sleep on commands to the NIC when updating multicast filters. More generally this permitted driver's to use an sx as a softc lock. Unfortunately this change introduced a race whereby a a multicast update would still be queued for deletion when ifconfig deleted the interface thus calling down in to _purgemaddrs and synchronously deleting _all_ of the multicast addresses on the interface. Synchronously remove all external references to a multicast address before enqueueing for delete. Reported by: lwhsu Approved by: sbruno
2018-05-06 20:34:13 +00:00
void
if_delmulti_ifma(struct ifmultiaddr *ifma)
{
if_delmulti_ifma_flags(ifma, 0);
}
Implement reference counting for ifmultiaddr, in_multi, and in6_multi structures. Detect when ifnet instances are detached from the network stack and perform appropriate cleanup to prevent memory leaks. This has been implemented in such a way as to be backwards ABI compatible. Kernel consumers are changed to use if_delmulti_ifma(); in_delmulti() is unable to detect interface removal by design, as it performs searches on structures which are removed with the interface. With this architectural change, the panics FreeBSD users have experienced with carp and pfsync should be resolved. Obtained from: p4 branch bms_netdev Reviewed by: andre Sponsored by: Garance A Drosehn Idea from: NetBSD MFC after: 1 month
2007-03-20 00:36:10 +00:00
/*
* Delete a multicast group membership by group membership pointer.
* Network-layer protocol domains must use this routine.
*
Remove IFF_NEEDSGIANT, a compatibility infrastructure introduced in FreeBSD 5.x to allow network device drivers to run with Giant despite the network stack being Giant-free. This significantly simplifies calls into ioctl() on network interfaces, especially in the multicast code, as well as eliminates deferred invocation of interface if_start routines. Disable the build on device drivers still depending on IFF_NEEDSGIANT as they no longer compile. They will be removed in a few weeks if they haven't been made MPSAFE in that time. Disabled drivers: if_ar if_axe if_aue if_cdce if_cue if_kue if_ray if_rue if_rum if_sr if_udav if_ural if_zyd Drivers that were already disabled because of tty changes: if_ppp if_sl Discussed on: arch@
2009-03-15 14:21:05 +00:00
* It is safe to call this routine if the ifp disappeared.
Implement reference counting for ifmultiaddr, in_multi, and in6_multi structures. Detect when ifnet instances are detached from the network stack and perform appropriate cleanup to prevent memory leaks. This has been implemented in such a way as to be backwards ABI compatible. Kernel consumers are changed to use if_delmulti_ifma(); in_delmulti() is unable to detect interface removal by design, as it performs searches on structures which are removed with the interface. With this architectural change, the panics FreeBSD users have experienced with carp and pfsync should be resolved. Obtained from: p4 branch bms_netdev Reviewed by: andre Sponsored by: Garance A Drosehn Idea from: NetBSD MFC after: 1 month
2007-03-20 00:36:10 +00:00
*/
void
r333175 introduced deferred deletion of multicast addresses in order to permit the driver ioctl to sleep on commands to the NIC when updating multicast filters. More generally this permitted driver's to use an sx as a softc lock. Unfortunately this change introduced a race whereby a a multicast update would still be queued for deletion when ifconfig deleted the interface thus calling down in to _purgemaddrs and synchronously deleting _all_ of the multicast addresses on the interface. Synchronously remove all external references to a multicast address before enqueueing for delete. Reported by: lwhsu Approved by: sbruno
2018-05-06 20:34:13 +00:00
if_delmulti_ifma_flags(struct ifmultiaddr *ifma, int flags)
Implement reference counting for ifmultiaddr, in_multi, and in6_multi structures. Detect when ifnet instances are detached from the network stack and perform appropriate cleanup to prevent memory leaks. This has been implemented in such a way as to be backwards ABI compatible. Kernel consumers are changed to use if_delmulti_ifma(); in_delmulti() is unable to detect interface removal by design, as it performs searches on structures which are removed with the interface. With this architectural change, the panics FreeBSD users have experienced with carp and pfsync should be resolved. Obtained from: p4 branch bms_netdev Reviewed by: andre Sponsored by: Garance A Drosehn Idea from: NetBSD MFC after: 1 month
2007-03-20 00:36:10 +00:00
{
struct ifnet *ifp;
int lastref;
r333175 introduced deferred deletion of multicast addresses in order to permit the driver ioctl to sleep on commands to the NIC when updating multicast filters. More generally this permitted driver's to use an sx as a softc lock. Unfortunately this change introduced a race whereby a a multicast update would still be queued for deletion when ifconfig deleted the interface thus calling down in to _purgemaddrs and synchronously deleting _all_ of the multicast addresses on the interface. Synchronously remove all external references to a multicast address before enqueueing for delete. Reported by: lwhsu Approved by: sbruno
2018-05-06 20:34:13 +00:00
MCDPRINTF("%s freeing ifma: %p\n", __func__, ifma);
Separate list manipulation locking from state change in multicast Multicast incorrectly calls in to drivers with a mutex held causing drivers to have to go through all manner of contortions to use a non sleepable lock. Serialize multicast updates instead. Submitted by: mmacy <mmacy@mattmacy.io> Reviewed by: shurd, sbruno Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D14969
2018-05-02 19:36:29 +00:00
#ifdef INET
IN_MULTI_LIST_UNLOCK_ASSERT();
#endif
Implement reference counting for ifmultiaddr, in_multi, and in6_multi structures. Detect when ifnet instances are detached from the network stack and perform appropriate cleanup to prevent memory leaks. This has been implemented in such a way as to be backwards ABI compatible. Kernel consumers are changed to use if_delmulti_ifma(); in_delmulti() is unable to detect interface removal by design, as it performs searches on structures which are removed with the interface. With this architectural change, the panics FreeBSD users have experienced with carp and pfsync should be resolved. Obtained from: p4 branch bms_netdev Reviewed by: andre Sponsored by: Garance A Drosehn Idea from: NetBSD MFC after: 1 month
2007-03-20 00:36:10 +00:00
ifp = ifma->ifma_ifp;
#ifdef DIAGNOSTIC
if (ifp == NULL) {
printf("%s: ifma_ifp seems to be detached\n", __func__);
} else {
In DIAGNOSTIC block of if_delmulti_ifma_flags() enter the network epoch. This quickly plugs the regression from r353292. The locking of multicast definitely needs a broader review today... Reported by: pho, dhw
2019-10-08 16:45:56 +00:00
struct epoch_tracker et;
Implement reference counting for ifmultiaddr, in_multi, and in6_multi structures. Detect when ifnet instances are detached from the network stack and perform appropriate cleanup to prevent memory leaks. This has been implemented in such a way as to be backwards ABI compatible. Kernel consumers are changed to use if_delmulti_ifma(); in_delmulti() is unable to detect interface removal by design, as it performs searches on structures which are removed with the interface. With this architectural change, the panics FreeBSD users have experienced with carp and pfsync should be resolved. Obtained from: p4 branch bms_netdev Reviewed by: andre Sponsored by: Garance A Drosehn Idea from: NetBSD MFC after: 1 month
2007-03-20 00:36:10 +00:00
struct ifnet *oifp;
In DIAGNOSTIC block of if_delmulti_ifma_flags() enter the network epoch. This quickly plugs the regression from r353292. The locking of multicast definitely needs a broader review today... Reported by: pho, dhw
2019-10-08 16:45:56 +00:00
NET_EPOCH_ENTER(et);
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
CK_STAILQ_FOREACH(oifp, &V_ifnet, if_link)
Implement reference counting for ifmultiaddr, in_multi, and in6_multi structures. Detect when ifnet instances are detached from the network stack and perform appropriate cleanup to prevent memory leaks. This has been implemented in such a way as to be backwards ABI compatible. Kernel consumers are changed to use if_delmulti_ifma(); in_delmulti() is unable to detect interface removal by design, as it performs searches on structures which are removed with the interface. With this architectural change, the panics FreeBSD users have experienced with carp and pfsync should be resolved. Obtained from: p4 branch bms_netdev Reviewed by: andre Sponsored by: Garance A Drosehn Idea from: NetBSD MFC after: 1 month
2007-03-20 00:36:10 +00:00
if (ifp == oifp)
break;
In DIAGNOSTIC block of if_delmulti_ifma_flags() enter the network epoch. This quickly plugs the regression from r353292. The locking of multicast definitely needs a broader review today... Reported by: pho, dhw
2019-10-08 16:45:56 +00:00
NET_EPOCH_EXIT(et);
route: fix missed ref adds - ensure that we bump the ifa ref whenever we add a reference - defer freeing epoch protected references until after the if_purgaddrs loop
2018-05-29 00:53:53 +00:00
if (ifp != oifp)
Implement reference counting for ifmultiaddr, in_multi, and in6_multi structures. Detect when ifnet instances are detached from the network stack and perform appropriate cleanup to prevent memory leaks. This has been implemented in such a way as to be backwards ABI compatible. Kernel consumers are changed to use if_delmulti_ifma(); in_delmulti() is unable to detect interface removal by design, as it performs searches on structures which are removed with the interface. With this architectural change, the panics FreeBSD users have experienced with carp and pfsync should be resolved. Obtained from: p4 branch bms_netdev Reviewed by: andre Sponsored by: Garance A Drosehn Idea from: NetBSD MFC after: 1 month
2007-03-20 00:36:10 +00:00
ifp = NULL;
}
#endif
/*
* If and only if the ifnet instance exists: Acquire the address lock.
*/
if (ifp != NULL)
Convert all users of IF_ADDR_LOCK to use new locking macros that specify either a read lock or write lock. Reviewed by: bz MFC after: 2 weeks
2012-01-05 19:00:36 +00:00
IF_ADDR_WLOCK(ifp);
Implement reference counting for ifmultiaddr, in_multi, and in6_multi structures. Detect when ifnet instances are detached from the network stack and perform appropriate cleanup to prevent memory leaks. This has been implemented in such a way as to be backwards ABI compatible. Kernel consumers are changed to use if_delmulti_ifma(); in_delmulti() is unable to detect interface removal by design, as it performs searches on structures which are removed with the interface. With this architectural change, the panics FreeBSD users have experienced with carp and pfsync should be resolved. Obtained from: p4 branch bms_netdev Reviewed by: andre Sponsored by: Garance A Drosehn Idea from: NetBSD MFC after: 1 month
2007-03-20 00:36:10 +00:00
r333175 introduced deferred deletion of multicast addresses in order to permit the driver ioctl to sleep on commands to the NIC when updating multicast filters. More generally this permitted driver's to use an sx as a softc lock. Unfortunately this change introduced a race whereby a a multicast update would still be queued for deletion when ifconfig deleted the interface thus calling down in to _purgemaddrs and synchronously deleting _all_ of the multicast addresses on the interface. Synchronously remove all external references to a multicast address before enqueueing for delete. Reported by: lwhsu Approved by: sbruno
2018-05-06 20:34:13 +00:00
lastref = if_delmulti_locked(ifp, ifma, flags);
Implement reference counting for ifmultiaddr, in_multi, and in6_multi structures. Detect when ifnet instances are detached from the network stack and perform appropriate cleanup to prevent memory leaks. This has been implemented in such a way as to be backwards ABI compatible. Kernel consumers are changed to use if_delmulti_ifma(); in_delmulti() is unable to detect interface removal by design, as it performs searches on structures which are removed with the interface. With this architectural change, the panics FreeBSD users have experienced with carp and pfsync should be resolved. Obtained from: p4 branch bms_netdev Reviewed by: andre Sponsored by: Garance A Drosehn Idea from: NetBSD MFC after: 1 month
2007-03-20 00:36:10 +00:00
if (ifp != NULL) {
/*
* If and only if the ifnet instance exists:
* Release the address lock.
* If the group was left: update the hardware hash filter.
*/
Convert all users of IF_ADDR_LOCK to use new locking macros that specify either a read lock or write lock. Reviewed by: bz MFC after: 2 weeks
2012-01-05 19:00:36 +00:00
IF_ADDR_WUNLOCK(ifp);
Implement reference counting for ifmultiaddr, in_multi, and in6_multi structures. Detect when ifnet instances are detached from the network stack and perform appropriate cleanup to prevent memory leaks. This has been implemented in such a way as to be backwards ABI compatible. Kernel consumers are changed to use if_delmulti_ifma(); in_delmulti() is unable to detect interface removal by design, as it performs searches on structures which are removed with the interface. With this architectural change, the panics FreeBSD users have experienced with carp and pfsync should be resolved. Obtained from: p4 branch bms_netdev Reviewed by: andre Sponsored by: Garance A Drosehn Idea from: NetBSD MFC after: 1 month
2007-03-20 00:36:10 +00:00
if (lastref && ifp->if_ioctl != NULL) {
(void)(*ifp->if_ioctl)(ifp, SIOCDELMULTI, 0);
}
Checkpoint the beginnings of the new kernel interface for multicast group memberships. This is not actually operative at the moment (a lot of other code still needs to be changed), but this seemed like a useful reference point to check in so that others (i.e. Bill Fenner) have fair warning of where we are going.
1997-01-07 19:15:32 +00:00
}
Implement reference counting for ifmultiaddr, in_multi, and in6_multi structures. Detect when ifnet instances are detached from the network stack and perform appropriate cleanup to prevent memory leaks. This has been implemented in such a way as to be backwards ABI compatible. Kernel consumers are changed to use if_delmulti_ifma(); in_delmulti() is unable to detect interface removal by design, as it performs searches on structures which are removed with the interface. With this architectural change, the panics FreeBSD users have experienced with carp and pfsync should be resolved. Obtained from: p4 branch bms_netdev Reviewed by: andre Sponsored by: Garance A Drosehn Idea from: NetBSD MFC after: 1 month
2007-03-20 00:36:10 +00:00
}
Checkpoint the beginnings of the new kernel interface for multicast group memberships. This is not actually operative at the moment (a lot of other code still needs to be changed), but this seemed like a useful reference point to check in so that others (i.e. Bill Fenner) have fair warning of where we are going.
1997-01-07 19:15:32 +00:00
Implement reference counting for ifmultiaddr, in_multi, and in6_multi structures. Detect when ifnet instances are detached from the network stack and perform appropriate cleanup to prevent memory leaks. This has been implemented in such a way as to be backwards ABI compatible. Kernel consumers are changed to use if_delmulti_ifma(); in_delmulti() is unable to detect interface removal by design, as it performs searches on structures which are removed with the interface. With this architectural change, the panics FreeBSD users have experienced with carp and pfsync should be resolved. Obtained from: p4 branch bms_netdev Reviewed by: andre Sponsored by: Garance A Drosehn Idea from: NetBSD MFC after: 1 month
2007-03-20 00:36:10 +00:00
/*
* Perform deletion of network-layer and/or link-layer multicast address.
*
* Return 0 if the reference count was decremented.
* Return 1 if the final reference was released, indicating that the
* hardware hash filter should be reprogrammed.
*/
static int
if_delmulti_locked(struct ifnet *ifp, struct ifmultiaddr *ifma, int detaching)
{
struct ifmultiaddr *ll_ifma;
if (ifp != NULL && ifma->ifma_ifp != NULL) {
KASSERT(ifma->ifma_ifp == ifp,
("%s: inconsistent ifp %p", __func__, ifp));
Convert all users of IF_ADDR_LOCK to use new locking macros that specify either a read lock or write lock. Reviewed by: bz MFC after: 2 weeks
2012-01-05 19:00:36 +00:00
IF_ADDR_WLOCK_ASSERT(ifp);
Implement reference counting for ifmultiaddr, in_multi, and in6_multi structures. Detect when ifnet instances are detached from the network stack and perform appropriate cleanup to prevent memory leaks. This has been implemented in such a way as to be backwards ABI compatible. Kernel consumers are changed to use if_delmulti_ifma(); in_delmulti() is unable to detect interface removal by design, as it performs searches on structures which are removed with the interface. With this architectural change, the panics FreeBSD users have experienced with carp and pfsync should be resolved. Obtained from: p4 branch bms_netdev Reviewed by: andre Sponsored by: Garance A Drosehn Idea from: NetBSD MFC after: 1 month
2007-03-20 00:36:10 +00:00
}
ifp = ifma->ifma_ifp;
r333175 introduced deferred deletion of multicast addresses in order to permit the driver ioctl to sleep on commands to the NIC when updating multicast filters. More generally this permitted driver's to use an sx as a softc lock. Unfortunately this change introduced a race whereby a a multicast update would still be queued for deletion when ifconfig deleted the interface thus calling down in to _purgemaddrs and synchronously deleting _all_ of the multicast addresses on the interface. Synchronously remove all external references to a multicast address before enqueueing for delete. Reported by: lwhsu Approved by: sbruno
2018-05-06 20:34:13 +00:00
MCDPRINTF("%s freeing %p from %s \n", __func__, ifma, ifp ? ifp->if_xname : "");
Checkpoint the beginnings of the new kernel interface for multicast group memberships. This is not actually operative at the moment (a lot of other code still needs to be changed), but this seemed like a useful reference point to check in so that others (i.e. Bill Fenner) have fair warning of where we are going.
1997-01-07 19:15:32 +00:00
/*
Implement reference counting for ifmultiaddr, in_multi, and in6_multi structures. Detect when ifnet instances are detached from the network stack and perform appropriate cleanup to prevent memory leaks. This has been implemented in such a way as to be backwards ABI compatible. Kernel consumers are changed to use if_delmulti_ifma(); in_delmulti() is unable to detect interface removal by design, as it performs searches on structures which are removed with the interface. With this architectural change, the panics FreeBSD users have experienced with carp and pfsync should be resolved. Obtained from: p4 branch bms_netdev Reviewed by: andre Sponsored by: Garance A Drosehn Idea from: NetBSD MFC after: 1 month
2007-03-20 00:36:10 +00:00
* If the ifnet is detaching, null out references to ifnet,
* so that upper protocol layers will notice, and not attempt
Fix a case where hardware removal of an interface caused an attempt to announce an ll_ifma which has gone away. Add a KASSERT to catch regressions. Bug found by: Tom Uffner
2007-03-27 16:11:28 +00:00
* to obtain locks for an ifnet which no longer exists. The
* routing socket announcement must happen before the ifnet
* instance is detached from the system.
Checkpoint the beginnings of the new kernel interface for multicast group memberships. This is not actually operative at the moment (a lot of other code still needs to be changed), but this seemed like a useful reference point to check in so that others (i.e. Bill Fenner) have fair warning of where we are going.
1997-01-07 19:15:32 +00:00
*/
Implement reference counting for ifmultiaddr, in_multi, and in6_multi structures. Detect when ifnet instances are detached from the network stack and perform appropriate cleanup to prevent memory leaks. This has been implemented in such a way as to be backwards ABI compatible. Kernel consumers are changed to use if_delmulti_ifma(); in_delmulti() is unable to detect interface removal by design, as it performs searches on structures which are removed with the interface. With this architectural change, the panics FreeBSD users have experienced with carp and pfsync should be resolved. Obtained from: p4 branch bms_netdev Reviewed by: andre Sponsored by: Garance A Drosehn Idea from: NetBSD MFC after: 1 month
2007-03-20 00:36:10 +00:00
if (detaching) {
#ifdef DIAGNOSTIC
printf("%s: detaching ifnet instance %p\n", __func__, ifp);
#endif
Fix a case where hardware removal of an interface caused an attempt to announce an ll_ifma which has gone away. Add a KASSERT to catch regressions. Bug found by: Tom Uffner
2007-03-27 16:11:28 +00:00
/*
* ifp may already be nulled out if we are being reentered
* to delete the ll_ifma.
*/
if (ifp != NULL) {
rt_newmaddrmsg(RTM_DELMADDR, ifma);
ifma->ifma_ifp = NULL;
}
Implement reference counting for ifmultiaddr, in_multi, and in6_multi structures. Detect when ifnet instances are detached from the network stack and perform appropriate cleanup to prevent memory leaks. This has been implemented in such a way as to be backwards ABI compatible. Kernel consumers are changed to use if_delmulti_ifma(); in_delmulti() is unable to detect interface removal by design, as it performs searches on structures which are removed with the interface. With this architectural change, the panics FreeBSD users have experienced with carp and pfsync should be resolved. Obtained from: p4 branch bms_netdev Reviewed by: andre Sponsored by: Garance A Drosehn Idea from: NetBSD MFC after: 1 month
2007-03-20 00:36:10 +00:00
}
Checkpoint the beginnings of the new kernel interface for multicast group memberships. This is not actually operative at the moment (a lot of other code still needs to be changed), but this seemed like a useful reference point to check in so that others (i.e. Bill Fenner) have fair warning of where we are going.
1997-01-07 19:15:32 +00:00
Implement reference counting for ifmultiaddr, in_multi, and in6_multi structures. Detect when ifnet instances are detached from the network stack and perform appropriate cleanup to prevent memory leaks. This has been implemented in such a way as to be backwards ABI compatible. Kernel consumers are changed to use if_delmulti_ifma(); in_delmulti() is unable to detect interface removal by design, as it performs searches on structures which are removed with the interface. With this architectural change, the panics FreeBSD users have experienced with carp and pfsync should be resolved. Obtained from: p4 branch bms_netdev Reviewed by: andre Sponsored by: Garance A Drosehn Idea from: NetBSD MFC after: 1 month
2007-03-20 00:36:10 +00:00
if (--ifma->ifma_refcount > 0)
return 0;
Fix in6_multi double free This is actually several different bugs: - The code is not designed to handle inpcb deletion after interface deletion - add reference for inpcb membership - The multicast address has to be removed from interface lists when the refcount goes to zero OR when the interface goes away - decouple list disconnect from refcount (v6 only for now) - ifmultiaddr can exist past being on interface lists - add flag for tracking whether or not it's enqueued - deferring freeing moptions makes the incpb cleanup code simpler but opens the door wider still to races - call inp_gcmoptions synchronously after dropping the the inpcb lock Fundamentally multicast needs a rewrite - but keep applying band-aids for now. Tested by: kp Reported by: novel, kp, lwhsu
2018-08-15 20:23:08 +00:00
if (ifp != NULL && detaching == 0 && (ifma->ifma_flags & IFMA_F_ENQUEUED)) {
ifnet: Replace if_addr_lock rwlock with epoch + mutex Run on LLNW canaries and tested by pho@ gallatin: Using a 14-core, 28-HTT single socket E5-2697 v3 with a 40GbE MLX5 based ConnectX 4-LX NIC, I see an almost 12% improvement in received packet rate, and a larger improvement in bytes delivered all the way to userspace. When the host receiving 64 streams of netperf -H $DUT -t UDP_STREAM -- -m 1, I see, using nstat -I mce0 1 before the patch: InMpps OMpps InGbs OGbs err TCP Est %CPU syscalls csw irq GBfree 4.98 0.00 4.42 0.00 4235592 33 83.80 4720653 2149771 1235 247.32 4.73 0.00 4.20 0.00 4025260 33 82.99 4724900 2139833 1204 247.32 4.72 0.00 4.20 0.00 4035252 33 82.14 4719162 2132023 1264 247.32 4.71 0.00 4.21 0.00 4073206 33 83.68 4744973 2123317 1347 247.32 4.72 0.00 4.21 0.00 4061118 33 80.82 4713615 2188091 1490 247.32 4.72 0.00 4.21 0.00 4051675 33 85.29 4727399 2109011 1205 247.32 4.73 0.00 4.21 0.00 4039056 33 84.65 4724735 2102603 1053 247.32 After the patch InMpps OMpps InGbs OGbs err TCP Est %CPU syscalls csw irq GBfree 5.43 0.00 4.20 0.00 3313143 33 84.96 5434214 1900162 2656 245.51 5.43 0.00 4.20 0.00 3308527 33 85.24 5439695 1809382 2521 245.51 5.42 0.00 4.19 0.00 3316778 33 87.54 5416028 1805835 2256 245.51 5.42 0.00 4.19 0.00 3317673 33 90.44 5426044 1763056 2332 245.51 5.42 0.00 4.19 0.00 3314839 33 88.11 5435732 1792218 2499 245.52 5.44 0.00 4.19 0.00 3293228 33 91.84 5426301 1668597 2121 245.52 Similarly, netperf reports 230Mb/s before the patch, and 270Mb/s after the patch Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15366
2018-05-18 20:13:34 +00:00
CK_STAILQ_REMOVE(&ifp->if_multiaddrs, ifma, ifmultiaddr, ifma_link);
Fix in6_multi double free This is actually several different bugs: - The code is not designed to handle inpcb deletion after interface deletion - add reference for inpcb membership - The multicast address has to be removed from interface lists when the refcount goes to zero OR when the interface goes away - decouple list disconnect from refcount (v6 only for now) - ifmultiaddr can exist past being on interface lists - add flag for tracking whether or not it's enqueued - deferring freeing moptions makes the incpb cleanup code simpler but opens the door wider still to races - call inp_gcmoptions synchronously after dropping the the inpcb lock Fundamentally multicast needs a rewrite - but keep applying band-aids for now. Tested by: kp Reported by: novel, kp, lwhsu
2018-08-15 20:23:08 +00:00
ifma->ifma_flags &= ~IFMA_F_ENQUEUED;
}
Implement reference counting for ifmultiaddr, in_multi, and in6_multi structures. Detect when ifnet instances are detached from the network stack and perform appropriate cleanup to prevent memory leaks. This has been implemented in such a way as to be backwards ABI compatible. Kernel consumers are changed to use if_delmulti_ifma(); in_delmulti() is unable to detect interface removal by design, as it performs searches on structures which are removed with the interface. With this architectural change, the panics FreeBSD users have experienced with carp and pfsync should be resolved. Obtained from: p4 branch bms_netdev Reviewed by: andre Sponsored by: Garance A Drosehn Idea from: NetBSD MFC after: 1 month
2007-03-20 00:36:10 +00:00
/*
* If this ifma is a network-layer ifma, a link-layer ifma may
* have been associated with it. Release it first if so.
*/
ll_ifma = ifma->ifma_llifma;
Protect link layer network interface multicast address list manipulation using ifp->if_addr_mtx: - Initialize if_addr_mtx when ifnet is initialized. - Destroy if_addr_mtx when ifnet is torn down. - Rename ifmaof_ifpforaddr() to if_findmulti(); assert if_addr_mtx. Staticize. - Extract ifmultiaddr allocation and initialization into if_allocmulti(); accept a 'mflags' argument to indicate whether or not sleeping is permitted. This centralizes error handling and address duplication. - Extract ifmultiaddr tear-down and deallocation in if_freemulti(). - Re-structure if_addmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Make use of non-sleeping allocations. Annotate the fact that we only generate routing socket events for explicit address addition, not implicit link layer address addition. - Re-structure if_delmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Annotate the lack of a routing socket event for implicit link layer address removal. - De-spl all and sundry. Problem reported by: Ed Maste <emaste at phaedrus dot sandvine dot ca> MFC after: 1 week
2005-08-02 23:23:26 +00:00
if (ll_ifma != NULL) {
Implement reference counting for ifmultiaddr, in_multi, and in6_multi structures. Detect when ifnet instances are detached from the network stack and perform appropriate cleanup to prevent memory leaks. This has been implemented in such a way as to be backwards ABI compatible. Kernel consumers are changed to use if_delmulti_ifma(); in_delmulti() is unable to detect interface removal by design, as it performs searches on structures which are removed with the interface. With this architectural change, the panics FreeBSD users have experienced with carp and pfsync should be resolved. Obtained from: p4 branch bms_netdev Reviewed by: andre Sponsored by: Garance A Drosehn Idea from: NetBSD MFC after: 1 month
2007-03-20 00:36:10 +00:00
KASSERT(ifma->ifma_lladdr != NULL,
("%s: llifma w/o lladdr", __func__));
if (detaching)
ll_ifma->ifma_ifp = NULL; /* XXX */
if (--ll_ifma->ifma_refcount == 0) {
if (ifp != NULL) {
Fix in6_multi double free This is actually several different bugs: - The code is not designed to handle inpcb deletion after interface deletion - add reference for inpcb membership - The multicast address has to be removed from interface lists when the refcount goes to zero OR when the interface goes away - decouple list disconnect from refcount (v6 only for now) - ifmultiaddr can exist past being on interface lists - add flag for tracking whether or not it's enqueued - deferring freeing moptions makes the incpb cleanup code simpler but opens the door wider still to races - call inp_gcmoptions synchronously after dropping the the inpcb lock Fundamentally multicast needs a rewrite - but keep applying band-aids for now. Tested by: kp Reported by: novel, kp, lwhsu
2018-08-15 20:23:08 +00:00
if (ll_ifma->ifma_flags & IFMA_F_ENQUEUED) {
CK_STAILQ_REMOVE(&ifp->if_multiaddrs, ll_ifma, ifmultiaddr,
ifma_link);
fix copy/paste error when clearing ifma flag CID: 1395119 Reported by: vangyzen
2018-08-21 22:59:22 +00:00
ll_ifma->ifma_flags &= ~IFMA_F_ENQUEUED;
Fix in6_multi double free This is actually several different bugs: - The code is not designed to handle inpcb deletion after interface deletion - add reference for inpcb membership - The multicast address has to be removed from interface lists when the refcount goes to zero OR when the interface goes away - decouple list disconnect from refcount (v6 only for now) - ifmultiaddr can exist past being on interface lists - add flag for tracking whether or not it's enqueued - deferring freeing moptions makes the incpb cleanup code simpler but opens the door wider still to races - call inp_gcmoptions synchronously after dropping the the inpcb lock Fundamentally multicast needs a rewrite - but keep applying band-aids for now. Tested by: kp Reported by: novel, kp, lwhsu
2018-08-15 20:23:08 +00:00
}
Implement reference counting for ifmultiaddr, in_multi, and in6_multi structures. Detect when ifnet instances are detached from the network stack and perform appropriate cleanup to prevent memory leaks. This has been implemented in such a way as to be backwards ABI compatible. Kernel consumers are changed to use if_delmulti_ifma(); in_delmulti() is unable to detect interface removal by design, as it performs searches on structures which are removed with the interface. With this architectural change, the panics FreeBSD users have experienced with carp and pfsync should be resolved. Obtained from: p4 branch bms_netdev Reviewed by: andre Sponsored by: Garance A Drosehn Idea from: NetBSD MFC after: 1 month
2007-03-20 00:36:10 +00:00
}
Protect link layer network interface multicast address list manipulation using ifp->if_addr_mtx: - Initialize if_addr_mtx when ifnet is initialized. - Destroy if_addr_mtx when ifnet is torn down. - Rename ifmaof_ifpforaddr() to if_findmulti(); assert if_addr_mtx. Staticize. - Extract ifmultiaddr allocation and initialization into if_allocmulti(); accept a 'mflags' argument to indicate whether or not sleeping is permitted. This centralizes error handling and address duplication. - Extract ifmultiaddr tear-down and deallocation in if_freemulti(). - Re-structure if_addmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Make use of non-sleeping allocations. Annotate the fact that we only generate routing socket events for explicit address addition, not implicit link layer address addition. - Re-structure if_delmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Annotate the lack of a routing socket event for implicit link layer address removal. - De-spl all and sundry. Problem reported by: Ed Maste <emaste at phaedrus dot sandvine dot ca> MFC after: 1 week
2005-08-02 23:23:26 +00:00
if_freemulti(ll_ifma);
Implement reference counting for ifmultiaddr, in_multi, and in6_multi structures. Detect when ifnet instances are detached from the network stack and perform appropriate cleanup to prevent memory leaks. This has been implemented in such a way as to be backwards ABI compatible. Kernel consumers are changed to use if_delmulti_ifma(); in_delmulti() is unable to detect interface removal by design, as it performs searches on structures which are removed with the interface. With this architectural change, the panics FreeBSD users have experienced with carp and pfsync should be resolved. Obtained from: p4 branch bms_netdev Reviewed by: andre Sponsored by: Garance A Drosehn Idea from: NetBSD MFC after: 1 month
2007-03-20 00:36:10 +00:00
}
Checkpoint the beginnings of the new kernel interface for multicast group memberships. This is not actually operative at the moment (a lot of other code still needs to be changed), but this seemed like a useful reference point to check in so that others (i.e. Bill Fenner) have fair warning of where we are going.
1997-01-07 19:15:32 +00:00
}
r333175 introduced deferred deletion of multicast addresses in order to permit the driver ioctl to sleep on commands to the NIC when updating multicast filters. More generally this permitted driver's to use an sx as a softc lock. Unfortunately this change introduced a race whereby a a multicast update would still be queued for deletion when ifconfig deleted the interface thus calling down in to _purgemaddrs and synchronously deleting _all_ of the multicast addresses on the interface. Synchronously remove all external references to a multicast address before enqueueing for delete. Reported by: lwhsu Approved by: sbruno
2018-05-06 20:34:13 +00:00
#ifdef INVARIANTS
if (ifp) {
struct ifmultiaddr *ifmatmp;
Implement reference counting for ifmultiaddr, in_multi, and in6_multi structures. Detect when ifnet instances are detached from the network stack and perform appropriate cleanup to prevent memory leaks. This has been implemented in such a way as to be backwards ABI compatible. Kernel consumers are changed to use if_delmulti_ifma(); in_delmulti() is unable to detect interface removal by design, as it performs searches on structures which are removed with the interface. With this architectural change, the panics FreeBSD users have experienced with carp and pfsync should be resolved. Obtained from: p4 branch bms_netdev Reviewed by: andre Sponsored by: Garance A Drosehn Idea from: NetBSD MFC after: 1 month
2007-03-20 00:36:10 +00:00
ifnet: Replace if_addr_lock rwlock with epoch + mutex Run on LLNW canaries and tested by pho@ gallatin: Using a 14-core, 28-HTT single socket E5-2697 v3 with a 40GbE MLX5 based ConnectX 4-LX NIC, I see an almost 12% improvement in received packet rate, and a larger improvement in bytes delivered all the way to userspace. When the host receiving 64 streams of netperf -H $DUT -t UDP_STREAM -- -m 1, I see, using nstat -I mce0 1 before the patch: InMpps OMpps InGbs OGbs err TCP Est %CPU syscalls csw irq GBfree 4.98 0.00 4.42 0.00 4235592 33 83.80 4720653 2149771 1235 247.32 4.73 0.00 4.20 0.00 4025260 33 82.99 4724900 2139833 1204 247.32 4.72 0.00 4.20 0.00 4035252 33 82.14 4719162 2132023 1264 247.32 4.71 0.00 4.21 0.00 4073206 33 83.68 4744973 2123317 1347 247.32 4.72 0.00 4.21 0.00 4061118 33 80.82 4713615 2188091 1490 247.32 4.72 0.00 4.21 0.00 4051675 33 85.29 4727399 2109011 1205 247.32 4.73 0.00 4.21 0.00 4039056 33 84.65 4724735 2102603 1053 247.32 After the patch InMpps OMpps InGbs OGbs err TCP Est %CPU syscalls csw irq GBfree 5.43 0.00 4.20 0.00 3313143 33 84.96 5434214 1900162 2656 245.51 5.43 0.00 4.20 0.00 3308527 33 85.24 5439695 1809382 2521 245.51 5.42 0.00 4.19 0.00 3316778 33 87.54 5416028 1805835 2256 245.51 5.42 0.00 4.19 0.00 3317673 33 90.44 5426044 1763056 2332 245.51 5.42 0.00 4.19 0.00 3314839 33 88.11 5435732 1792218 2499 245.52 5.44 0.00 4.19 0.00 3293228 33 91.84 5426301 1668597 2121 245.52 Similarly, netperf reports 230Mb/s before the patch, and 270Mb/s after the patch Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15366
2018-05-18 20:13:34 +00:00
CK_STAILQ_FOREACH(ifmatmp, &ifp->if_multiaddrs, ifma_link)
r333175 introduced deferred deletion of multicast addresses in order to permit the driver ioctl to sleep on commands to the NIC when updating multicast filters. More generally this permitted driver's to use an sx as a softc lock. Unfortunately this change introduced a race whereby a a multicast update would still be queued for deletion when ifconfig deleted the interface thus calling down in to _purgemaddrs and synchronously deleting _all_ of the multicast addresses on the interface. Synchronously remove all external references to a multicast address before enqueueing for delete. Reported by: lwhsu Approved by: sbruno
2018-05-06 20:34:13 +00:00
MPASS(ifma != ifmatmp);
}
#endif
Implement reference counting for ifmultiaddr, in_multi, and in6_multi structures. Detect when ifnet instances are detached from the network stack and perform appropriate cleanup to prevent memory leaks. This has been implemented in such a way as to be backwards ABI compatible. Kernel consumers are changed to use if_delmulti_ifma(); in_delmulti() is unable to detect interface removal by design, as it performs searches on structures which are removed with the interface. With this architectural change, the panics FreeBSD users have experienced with carp and pfsync should be resolved. Obtained from: p4 branch bms_netdev Reviewed by: andre Sponsored by: Garance A Drosehn Idea from: NetBSD MFC after: 1 month
2007-03-20 00:36:10 +00:00
if_freemulti(ifma);
Protect link layer network interface multicast address list manipulation using ifp->if_addr_mtx: - Initialize if_addr_mtx when ifnet is initialized. - Destroy if_addr_mtx when ifnet is torn down. - Rename ifmaof_ifpforaddr() to if_findmulti(); assert if_addr_mtx. Staticize. - Extract ifmultiaddr allocation and initialization into if_allocmulti(); accept a 'mflags' argument to indicate whether or not sleeping is permitted. This centralizes error handling and address duplication. - Extract ifmultiaddr tear-down and deallocation in if_freemulti(). - Re-structure if_addmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Make use of non-sleeping allocations. Annotate the fact that we only generate routing socket events for explicit address addition, not implicit link layer address addition. - Re-structure if_delmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Annotate the lack of a routing socket event for implicit link layer address removal. - De-spl all and sundry. Problem reported by: Ed Maste <emaste at phaedrus dot sandvine dot ca> MFC after: 1 week
2005-08-02 23:23:26 +00:00
/*
Implement reference counting for ifmultiaddr, in_multi, and in6_multi structures. Detect when ifnet instances are detached from the network stack and perform appropriate cleanup to prevent memory leaks. This has been implemented in such a way as to be backwards ABI compatible. Kernel consumers are changed to use if_delmulti_ifma(); in_delmulti() is unable to detect interface removal by design, as it performs searches on structures which are removed with the interface. With this architectural change, the panics FreeBSD users have experienced with carp and pfsync should be resolved. Obtained from: p4 branch bms_netdev Reviewed by: andre Sponsored by: Garance A Drosehn Idea from: NetBSD MFC after: 1 month
2007-03-20 00:36:10 +00:00
* The last reference to this instance of struct ifmultiaddr
* was released; the hardware should be notified of this change.
Protect link layer network interface multicast address list manipulation using ifp->if_addr_mtx: - Initialize if_addr_mtx when ifnet is initialized. - Destroy if_addr_mtx when ifnet is torn down. - Rename ifmaof_ifpforaddr() to if_findmulti(); assert if_addr_mtx. Staticize. - Extract ifmultiaddr allocation and initialization into if_allocmulti(); accept a 'mflags' argument to indicate whether or not sleeping is permitted. This centralizes error handling and address duplication. - Extract ifmultiaddr tear-down and deallocation in if_freemulti(). - Re-structure if_addmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Make use of non-sleeping allocations. Annotate the fact that we only generate routing socket events for explicit address addition, not implicit link layer address addition. - Re-structure if_delmulti() to hold if_addr_mtx around manipulation of the ifnet multicast address list and reference count manipulation. Annotate the lack of a routing socket event for implicit link layer address removal. - De-spl all and sundry. Problem reported by: Ed Maste <emaste at phaedrus dot sandvine dot ca> MFC after: 1 week
2005-08-02 23:23:26 +00:00
*/
Implement reference counting for ifmultiaddr, in_multi, and in6_multi structures. Detect when ifnet instances are detached from the network stack and perform appropriate cleanup to prevent memory leaks. This has been implemented in such a way as to be backwards ABI compatible. Kernel consumers are changed to use if_delmulti_ifma(); in_delmulti() is unable to detect interface removal by design, as it performs searches on structures which are removed with the interface. With this architectural change, the panics FreeBSD users have experienced with carp and pfsync should be resolved. Obtained from: p4 branch bms_netdev Reviewed by: andre Sponsored by: Garance A Drosehn Idea from: NetBSD MFC after: 1 month
2007-03-20 00:36:10 +00:00
return 1;
Checkpoint the beginnings of the new kernel interface for multicast group memberships. This is not actually operative at the moment (a lot of other code still needs to be changed), but this seemed like a useful reference point to check in so that others (i.e. Bill Fenner) have fair warning of where we are going.
1997-01-07 19:15:32 +00:00
}
Export the functionality of SIOCSIFLLADDR with if_setlladdr() and add some more rigorous sanity checking in the process. Reviewed by: freebsd-net
2000-08-15 00:48:38 +00:00
/*
* Set the link layer address on an interface.
*
* At this time we only support certain types of interfaces,
* and we don't allow the length of the address to change.
Fix lladdr change propagation for on vlans on top of it. Fix lladdr update when setting mac address manually. Fix lladdr_event for slave ports addition. MFC after: 4 weeks Sponsored by: Yandex LLC Differential Revision: https://reviews.freebsd.org/D4004
2015-11-01 19:59:04 +00:00
*
* Set noinline to be dtrace-friendly
Export the functionality of SIOCSIFLLADDR with if_setlladdr() and add some more rigorous sanity checking in the process. Reviewed by: freebsd-net
2000-08-15 00:48:38 +00:00
*/
Fix lladdr change propagation for on vlans on top of it. Fix lladdr update when setting mac address manually. Fix lladdr_event for slave ports addition. MFC after: 4 weeks Sponsored by: Yandex LLC Differential Revision: https://reviews.freebsd.org/D4004
2015-11-01 19:59:04 +00:00
__noinline int
Export the functionality of SIOCSIFLLADDR with if_setlladdr() and add some more rigorous sanity checking in the process. Reviewed by: freebsd-net
2000-08-15 00:48:38 +00:00
if_setlladdr(struct ifnet *ifp, const u_char *lladdr, int len)
{
struct sockaddr_dl *sdl;
struct ifaddr *ifa;
Add missing 'struct ifreq ifr;' that was forgotten in the last commit.
2002-04-10 06:07:16 +00:00
struct ifreq ifr;
Widen NET_EPOCH coverage. When epoch(9) was introduced to network stack, it was basically dropped in place of existing locking, which was mutexes and rwlocks. For the sake of performance mutex covered areas were as small as possible, so became epoch covered areas. However, epoch doesn't introduce any contention, it just delays memory reclaim. So, there is no point to minimise epoch covered areas in sense of performance. Meanwhile entering/exiting epoch also has non-zero CPU usage, so doing this less often is a win. Not the least is also code maintainability. In the new paradigm we can assume that at any stage of processing a packet, we are inside network epoch. This makes coding both input and output path way easier. On output path we already enter epoch quite early - in the ip_output(), in the ip6_output(). This patch does the same for the input path. All ISR processing, network related callouts, other ways of packet injection to the network stack shall be performed in net_epoch. Any leaf function that walks network configuration now asserts epoch. Tricky part is configuration code paths - ioctls, sysctls. They also call into leaf functions, so some need to be changed. This patch would introduce more epoch recursions (see EPOCH_TRACE) than we had before. They will be cleaned up separately, as several of them aren't trivial. Note, that unlike a lock recursion the epoch recursion is safe and just wastes a bit of resources. Reviewed by: gallatin, hselasky, cy, adrian, kristof Differential Revision: https://reviews.freebsd.org/D19111
2019-10-07 22:40:05 +00:00
- Store pointer to the link-level address right in "struct ifnet" rather than in ifindex_table[]; all (except one) accesses are through ifp anyway. IF_LLADDR() works faster, and all (except one) ifaddr_byindex() users were converted to use ifp->if_addr. - Stop storing a (pointer to) Ethernet address in "struct arpcom", and drop the IFP2ENADDR() macro; all users have been converted to use IF_LLADDR() instead.
2005-11-11 16:04:59 +00:00
ifa = ifp->if_addr;
Remove epoch assertion from if_setlladdr(). Originally this function was protected by IF_ADDR_LOCK(), which was a mutex, so that two simultaneous if_setlladdr() can't execute. Later it was switched to IF_ADDR_RLOCK(), likely by a mistake. Later it was switched to NET_EPOCH_ENTER(). Then I incorrectly added NET_EPOCH_ASSERT() here. In reality ifp->if_addr never goes away and never changes its length. So, doing bcopy() in it is always "safe", meaning it won't dereference a wrong pointer or write into someone's else memory. Of course doing two bcopy() in parallel would result in a mess of two addresses, but net epoch doesn't protect against that, neither IF_ADDR_RLOCK() did. So for now, just remove the assertion and leave for later a proper fix. Reported by: markj
2019-10-08 17:55:45 +00:00
if (ifa == NULL)
return (EINVAL);
UDP: further performance improvements on tx Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409
2018-05-23 21:02:14 +00:00
Export the functionality of SIOCSIFLLADDR with if_setlladdr() and add some more rigorous sanity checking in the process. Reviewed by: freebsd-net
2000-08-15 00:48:38 +00:00
sdl = (struct sockaddr_dl *)ifa->ifa_addr;
Remove epoch assertion from if_setlladdr(). Originally this function was protected by IF_ADDR_LOCK(), which was a mutex, so that two simultaneous if_setlladdr() can't execute. Later it was switched to IF_ADDR_RLOCK(), likely by a mistake. Later it was switched to NET_EPOCH_ENTER(). Then I incorrectly added NET_EPOCH_ASSERT() here. In reality ifp->if_addr never goes away and never changes its length. So, doing bcopy() in it is always "safe", meaning it won't dereference a wrong pointer or write into someone's else memory. Of course doing two bcopy() in parallel would result in a mess of two addresses, but net epoch doesn't protect against that, neither IF_ADDR_RLOCK() did. So for now, just remove the assertion and leave for later a proper fix. Reported by: markj
2019-10-08 17:55:45 +00:00
if (sdl == NULL)
return (EINVAL);
if (len != sdl->sdl_alen) /* don't allow length to change */
return (EINVAL);
Export the functionality of SIOCSIFLLADDR with if_setlladdr() and add some more rigorous sanity checking in the process. Reviewed by: freebsd-net
2000-08-15 00:48:38 +00:00
switch (ifp->if_type) {
- Make IFP2ENADDR() a pointer to IF_LLADDR() rather than another copy of Ethernet address. - Change iso88025_ifattach() and fddi_ifattach() to accept MAC address as an argument, similar to ether_ifattach(), to make this work.
2005-11-11 07:36:14 +00:00
case IFT_ETHER:
Export the functionality of SIOCSIFLLADDR with if_setlladdr() and add some more rigorous sanity checking in the process. Reviewed by: freebsd-net
2000-08-15 00:48:38 +00:00
case IFT_XETHER:
Change the type of the VLAN interface from IFT_PROPVIRTUAL, which was a temporary hack, to IFT_L2VLAN, which is the type assigned by IANA.
2001-04-04 15:10:58 +00:00
case IFT_L2VLAN:
Add hooks into the networking layer to support if_bridge. This changes struct ifnet so a buildworld is necessary. Approved by: mlaier (mentor) Obtained from: NetBSD
2005-06-05 03:13:13 +00:00
case IFT_BRIDGE:
Add the trunk(4) driver for providing link aggregation, failover and fault tolerance. This driver allows aggregation of multiple network interfaces as one virtual interface using a number of different protocols/algorithms. failover - Sends traffic through the secondary port if the master becomes inactive. fec - Supports Cisco Fast EtherChannel. lacp - Supports the IEEE 802.3ad Link Aggregation Control Protocol (LACP) and the Marker Protocol. loadbalance - Static loadbalancing using an outgoing hash. roundrobin - Distributes outgoing traffic using a round-robin scheduler through all active ports. This code was obtained from OpenBSD and this also includes 802.3ad LACP support from agr(4) in NetBSD.
2007-04-10 00:27:25 +00:00
case IFT_IEEE8023ADLAG:
Export the functionality of SIOCSIFLLADDR with if_setlladdr() and add some more rigorous sanity checking in the process. Reviewed by: freebsd-net
2000-08-15 00:48:38 +00:00
bcopy(lladdr, LLADDR(sdl), len);
break;
default:
Remove epoch assertion from if_setlladdr(). Originally this function was protected by IF_ADDR_LOCK(), which was a mutex, so that two simultaneous if_setlladdr() can't execute. Later it was switched to IF_ADDR_RLOCK(), likely by a mistake. Later it was switched to NET_EPOCH_ENTER(). Then I incorrectly added NET_EPOCH_ASSERT() here. In reality ifp->if_addr never goes away and never changes its length. So, doing bcopy() in it is always "safe", meaning it won't dereference a wrong pointer or write into someone's else memory. Of course doing two bcopy() in parallel would result in a mess of two addresses, but net epoch doesn't protect against that, neither IF_ADDR_RLOCK() did. So for now, just remove the assertion and leave for later a proper fix. Reported by: markj
2019-10-08 17:55:45 +00:00
return (ENODEV);
Export the functionality of SIOCSIFLLADDR with if_setlladdr() and add some more rigorous sanity checking in the process. Reviewed by: freebsd-net
2000-08-15 00:48:38 +00:00
}
In if_setlladdr(), use IF_ADDR_LOCK() and ifaddr references to improve the safety of link layer address manipulation. MFC after: 6 weeks
2009-06-24 10:36:48 +00:00
Export the functionality of SIOCSIFLLADDR with if_setlladdr() and add some more rigorous sanity checking in the process. Reviewed by: freebsd-net
2000-08-15 00:48:38 +00:00
/*
* If the interface is already up, we need
* to re-init it in order to reprogram its
* address filter.
*/
if ((ifp->if_flags & IFF_UP) != 0) {
MFp4: - Introduce a helper function if_setflag() containing the code common to ifpromisc() and if_allmulti() instead of duplicating the code poorly, with different bugs. - Call ifp->if_ioctl() in a consistent way: always use more compatible C syntax and check whether ifp->if_ioctl is not NULL prior to the call. MFC after: 1 month
2005-07-14 13:56:51 +00:00
if (ifp->if_ioctl) {
ifp->if_flags &= ~IFF_UP;
ifr.ifr_flags = ifp->if_flags & 0xffff;
ifr.ifr_flagshigh = ifp->if_flags >> 16;
(*ifp->if_ioctl)(ifp, SIOCSIFFLAGS, (caddr_t)&ifr);
ifp->if_flags |= IFF_UP;
ifr.ifr_flags = ifp->if_flags & 0xffff;
ifr.ifr_flagshigh = ifp->if_flags >> 16;
(*ifp->if_ioctl)(ifp, SIOCSIFFLAGS, (caddr_t)&ifr);
}
Export the functionality of SIOCSIFLLADDR with if_setlladdr() and add some more rigorous sanity checking in the process. Reviewed by: freebsd-net
2000-08-15 00:48:38 +00:00
}
Move iflladdr_event eventhandler invocation to if_setlladdr. Suggested by: glebius
2015-11-14 13:34:03 +00:00
EVENTHANDLER_INVOKE(iflladdr_event, ifp);
Remove epoch assertion from if_setlladdr(). Originally this function was protected by IF_ADDR_LOCK(), which was a mutex, so that two simultaneous if_setlladdr() can't execute. Later it was switched to IF_ADDR_RLOCK(), likely by a mistake. Later it was switched to NET_EPOCH_ENTER(). Then I incorrectly added NET_EPOCH_ASSERT() here. In reality ifp->if_addr never goes away and never changes its length. So, doing bcopy() in it is always "safe", meaning it won't dereference a wrong pointer or write into someone's else memory. Of course doing two bcopy() in parallel would result in a mess of two addresses, but net epoch doesn't protect against that, neither IF_ADDR_RLOCK() did. So for now, just remove the assertion and leave for later a proper fix. Reported by: markj
2019-10-08 17:55:45 +00:00
if_setlladdr: don't call ioctl in epoch context PR: 228612 Reported by: markj
2018-05-30 21:46:10 +00:00
return (0);
Export the functionality of SIOCSIFLLADDR with if_setlladdr() and add some more rigorous sanity checking in the process. Reviewed by: freebsd-net
2000-08-15 00:48:38 +00:00
}
Implement interface link header precomputation API. Add if_requestencap() interface method which is capable of calculating various link headers for given interface. Right now there is support for INET/INET6/ARP llheader calculation (IFENCAP_LL type request). Other types are planned to support more complex calculation (L2 multipath lagg nexthops, tunnel encap nexthops, etc..). Reshape 'struct route' to be able to pass additional data (with is length) to prepend to mbuf. These two changes permits routing code to pass pre-calculated nexthop data (like L2 header for route w/gateway) down to the stack eliminating the need for other lookups. It also brings us closer to more complex scenarios like transparently handling MPLS nexthops and tunnel interfaces. Last, but not least, it removes layering violation introduced by flowtable code (ro_lle) and simplifies handling of existing if_output consumers. ARP/ND changes: Make arp/ndp stack pre-calculate link header upon installing/updating lle record. Interface link address change are handled by re-calculating headers for all lles based on if_lladdr event. After these changes, arpresolve()/nd6_resolve() returns full pre-calculated header for supported interfaces thus simplifying if_output(). Move these lookups to separate ether_resolve_addr() function which ether returs error or fully-prepared link header. Add <arp|nd6_>resolve_addr() compat versions to return link addresses instead of pre-calculated data. BPF changes: Raw bpf writes occupied _two_ cases: AF_UNSPEC and pseudo_AF_HDRCMPLT. Despite the naming, both of there have ther header "complete". The only difference is that interface source mac has to be filled by OS for AF_UNSPEC (controlled via BIOCGHDRCMPLT). This logic has to stay inside BPF and not pollute if_output() routines. Convert BPF to pass prepend data via new 'struct route' mechanism. Note that it does not change non-optimized if_output(): ro_prepend handling is purely optional. Side note: hackish pseudo_AF_HDRCMPLT is supported for ethernet and FDDI. It is not needed for ethernet anymore. The only remaining FDDI user is dev/pdq mostly untouched since 2007. FDDI support was eliminated from OpenBSD in 2013 (sys/net/if_fddisubr.c rev 1.65). Flowtable changes: Flowtable violates layering by saving (and not correctly managing) rtes/lles. Instead of passing lle pointer, pass pointer to pre-calculated header data from that lle. Differential Revision: https://reviews.freebsd.org/D4102
2015-12-31 05:03:27 +00:00
/*
* Compat function for handling basic encapsulation requests.
* Not converted stacks (FDDI, IB, ..) supports traditional
* output model: ARP (and other similar L2 protocols) are handled
* inside output routine, arpresolve/nd6_resolve() returns MAC
* address instead of full prepend.
*
* This function creates calculated header==MAC for IPv4/IPv6 and
* returns EAFNOSUPPORT (which is then handled in ARP code) for other
* address families.
*/
static int
if_requestencap_default(struct ifnet *ifp, struct if_encap_req *req)
{
if (req->rtype != IFENCAP_LL)
return (EOPNOTSUPP);
if (req->bufsize < req->lladdr_len)
return (ENOMEM);
switch (req->family) {
case AF_INET:
case AF_INET6:
break;
default:
return (EAFNOSUPPORT);
}
/* Copy lladdr to storage as is */
memmove(req->buf, req->lladdr, req->lladdr_len);
req->bufsize = req->lladdr_len;
req->lladdr_off = 0;
return (0);
}
Deduplicate the code. Add generic function if_tunnel_check_nesting() that does check for allowed nesting level for tunneling interfaces and also does loop detection. Use it in gif(4), gre(4) and me(4) interfaces. Differential Revision: https://reviews.freebsd.org/D16162
2018-07-09 11:03:28 +00:00
/*
* Tunnel interfaces can nest, also they may cause infinite recursion
* calls when misconfigured. We'll prevent this by detecting loops.
* High nesting level may cause stack exhaustion. We'll prevent this
* by introducing upper limit.
*
* Return 0, if tunnel nesting count is equal or less than limit.
*/
int
if_tunnel_check_nesting(struct ifnet *ifp, struct mbuf *m, uint32_t cookie,
int limit)
{
struct m_tag *mtag;
int count;
count = 1;
mtag = NULL;
while ((mtag = m_tag_locate(m, cookie, 0, mtag)) != NULL) {
if (*(struct ifnet **)(mtag + 1) == ifp) {
log(LOG_NOTICE, "%s: loop detected\n", if_name(ifp));
return (EIO);
}
count++;
}
if (count > limit) {
log(LOG_NOTICE,
"%s: if_output recursively called too many times(%d)\n",
if_name(ifp), count);
return (EIO);
}
mtag = m_tag_alloc(cookie, 0, sizeof(struct ifnet *), M_NOWAIT);
if (mtag == NULL)
return (ENOMEM);
*(struct ifnet **)(mtag + 1) = ifp;
m_tag_prepend(m, mtag);
return (0);
}
Persistently store NIC's hardware MAC address, and add a way to retrive it The MAC address reported by `ifconfig ${nic} ether' does not always match the address in the hardware, as reported by the driver during attach. In particular, NICs which are components of a lagg(4) interface all report the same MAC. When attaching, the NIC driver passes the MAC address it read from the hardware as an argument to ether_ifattach(). Keep a second copy of it, and create ioctl(SIOCGHWADDR) to return it. Teach `ifconfig' to report it along with the active MAC address. PR: 194386 Reviewed by: glebius MFC after: 1 week Sponsored by: Panasas Differential Revision: https://reviews.freebsd.org/D10609
2017-05-10 22:13:47 +00:00
/*
* Get the link layer address that was read from the hardware at attach.
*
* This is only set by Ethernet NICs (IFT_ETHER), but laggX interfaces re-type
* their component interfaces as IFT_IEEE8023ADLAG.
*/
int
if_gethwaddr(struct ifnet *ifp, struct ifreq *ifr)
{
if (ifp->if_hw_addr == NULL)
return (ENODEV);
switch (ifp->if_type) {
case IFT_ETHER:
case IFT_IEEE8023ADLAG:
bcopy(ifp->if_hw_addr, ifr->ifr_addr.sa_data, ifp->if_addrlen);
return (0);
default:
return (ENODEV);
}
}
Replace the if_name and if_unit members of struct ifnet with new members if_xname, if_dname, and if_dunit. if_xname is the name of the interface and if_dname/unit are the driver name and instance. This change paves the way for interface renaming and enhanced pseudo device creation and configuration symantics. Approved By: re (in principle) Reviewed By: njl, imp Tested On: i386, amd64, sparc64 Obtained From: NetBSD (if_xname)
2003-10-31 18:32:15 +00:00
/*
* The name argument must be a pointer to storage which will last as
* long as the interface does. For physical devices, the result of
* device_get_name(dev) is a good choice and for pseudo-devices a
* static string works well.
*/
void
if_initname(struct ifnet *ifp, const char *name, int unit)
{
ifp->if_dname = name;
ifp->if_dunit = unit;
if (unit != IF_DUNIT_NONE)
snprintf(ifp->if_xname, IFNAMSIZ, "%s%d", name, unit);
else
strlcpy(ifp->if_xname, name, IFNAMSIZ);
}
Add a new helper function if_printf() modeled on device_printf(). The function takes a struct ifnet pointer followed by the usual printf arguments and prints "<interfacename>: " before the results of printf. Since this is the primary form of printf calls in network device drivers and accounts for most uses of the ifnet menber if_unit, this significantly simplifies many printf()s.
2002-09-24 17:35:08 +00:00
int
Slight cleanup of interface event logging. Make if_printf() use vlog() instead of vprintf(). This means it can no longer return the number of characters printed, as it used to, but every single call to if_printf() in the entire kernel ignores the return value anyway; just return 0 so we don't have to change the prototype. Consistently use if_printf() throughout sys/net/if.c, instead of a mixture of if_printf() and log(). In ifa_maintain_loopback_route(), don't needlessly log an error if we either failed to add a route because it already existed or failed to remove one because it did not. We still return an error code, though. MFC after: 1 week
2018-05-11 00:19:49 +00:00
if_printf(struct ifnet *ifp, const char *fmt, ...)
Add a new helper function if_printf() modeled on device_printf(). The function takes a struct ifnet pointer followed by the usual printf arguments and prints "<interfacename>: " before the results of printf. Since this is the primary form of printf calls in network device drivers and accounts for most uses of the ifnet menber if_unit, this significantly simplifies many printf()s.
2002-09-24 17:35:08 +00:00
{
Slight cleanup of interface event logging. Make if_printf() use vlog() instead of vprintf(). This means it can no longer return the number of characters printed, as it used to, but every single call to if_printf() in the entire kernel ignores the return value anyway; just return 0 so we don't have to change the prototype. Consistently use if_printf() throughout sys/net/if.c, instead of a mixture of if_printf() and log(). In ifa_maintain_loopback_route(), don't needlessly log an error if we either failed to add a route because it already existed or failed to remove one because it did not. We still return an error code, though. MFC after: 1 week
2018-05-11 00:19:49 +00:00
char if_fmt[256];
Add a new helper function if_printf() modeled on device_printf(). The function takes a struct ifnet pointer followed by the usual printf arguments and prints "<interfacename>: " before the results of printf. Since this is the primary form of printf calls in network device drivers and accounts for most uses of the ifnet menber if_unit, this significantly simplifies many printf()s.
2002-09-24 17:35:08 +00:00
va_list ap;
Slight cleanup of interface event logging. Make if_printf() use vlog() instead of vprintf(). This means it can no longer return the number of characters printed, as it used to, but every single call to if_printf() in the entire kernel ignores the return value anyway; just return 0 so we don't have to change the prototype. Consistently use if_printf() throughout sys/net/if.c, instead of a mixture of if_printf() and log(). In ifa_maintain_loopback_route(), don't needlessly log an error if we either failed to add a route because it already existed or failed to remove one because it did not. We still return an error code, though. MFC after: 1 week
2018-05-11 00:19:49 +00:00
snprintf(if_fmt, sizeof(if_fmt), "%s: %s", ifp->if_xname, fmt);
Add a new helper function if_printf() modeled on device_printf(). The function takes a struct ifnet pointer followed by the usual printf arguments and prints "<interfacename>: " before the results of printf. Since this is the primary form of printf calls in network device drivers and accounts for most uses of the ifnet menber if_unit, this significantly simplifies many printf()s.
2002-09-24 17:35:08 +00:00
va_start(ap, fmt);
Slight cleanup of interface event logging. Make if_printf() use vlog() instead of vprintf(). This means it can no longer return the number of characters printed, as it used to, but every single call to if_printf() in the entire kernel ignores the return value anyway; just return 0 so we don't have to change the prototype. Consistently use if_printf() throughout sys/net/if.c, instead of a mixture of if_printf() and log(). In ifa_maintain_loopback_route(), don't needlessly log an error if we either failed to add a route because it already existed or failed to remove one because it did not. We still return an error code, though. MFC after: 1 week
2018-05-11 00:19:49 +00:00
vlog(LOG_INFO, if_fmt, ap);
Add a new helper function if_printf() modeled on device_printf(). The function takes a struct ifnet pointer followed by the usual printf arguments and prints "<interfacename>: " before the results of printf. Since this is the primary form of printf calls in network device drivers and accounts for most uses of the ifnet menber if_unit, this significantly simplifies many printf()s.
2002-09-24 17:35:08 +00:00
va_end(ap);
Slight cleanup of interface event logging. Make if_printf() use vlog() instead of vprintf(). This means it can no longer return the number of characters printed, as it used to, but every single call to if_printf() in the entire kernel ignores the return value anyway; just return 0 so we don't have to change the prototype. Consistently use if_printf() throughout sys/net/if.c, instead of a mixture of if_printf() and log(). In ifa_maintain_loopback_route(), don't needlessly log an error if we either failed to add a route because it already existed or failed to remove one because it did not. We still return an error code, though. MFC after: 1 week
2018-05-11 00:19:49 +00:00
return (0);
Add a new helper function if_printf() modeled on device_printf(). The function takes a struct ifnet pointer followed by the usual printf arguments and prints "<interfacename>: " before the results of printf. Since this is the primary form of printf calls in network device drivers and accounts for most uses of the ifnet menber if_unit, this significantly simplifies many printf()s.
2002-09-24 17:35:08 +00:00
}
Add a new network interface flag, IFF_NEEDSGIANT, which will allow device drivers to declare that the ifp->if_start() method implemented by the driver requires Giant in order to operate correctly. Add a 'struct task' to 'struct ifnet' that can be used to execute a deferred ifp->if_start() in the event that if_start needs to be called in a Giant-free environment. To do this, introduce if_start(), a wrapper function for ifp->if_start(). If the interface can run MPSAFE, it directly dispatches into the interface start routine. If it can't run MPSAFE, we're running with debug.mpsafenet != 0, and Giant isn't currently held, the task is queued to execute in a swi holding Giant via if_start_deferred(). Modify if_handoff() to use if_start() instead of direct dispatch. Modify 802.11 to use if_start() instead of direct dispatch. This is intended to provide increased compatibility for non-MPSAFE network device drivers in the presence of Giant-free operation via asynchronous dispatch. However, this commit does not mark any network interfaces as IFF_NEEDSGIANT.
2004-07-27 23:20:45 +00:00
void
if_start(struct ifnet *ifp)
{
Remove IFF_NEEDSGIANT, a compatibility infrastructure introduced in FreeBSD 5.x to allow network device drivers to run with Giant despite the network stack being Giant-free. This significantly simplifies calls into ioctl() on network interfaces, especially in the multicast code, as well as eliminates deferred invocation of interface if_start routines. Disable the build on device drivers still depending on IFF_NEEDSGIANT as they no longer compile. They will be removed in a few weeks if they haven't been made MPSAFE in that time. Disabled drivers: if_ar if_axe if_aue if_cdce if_cue if_kue if_ray if_rue if_rum if_sr if_udav if_ural if_zyd Drivers that were already disabled because of tty changes: if_ppp if_sl Discussed on: arch@
2009-03-15 14:21:05 +00:00
(*(ifp)->if_start)(ifp);
Add a new network interface flag, IFF_NEEDSGIANT, which will allow device drivers to declare that the ifp->if_start() method implemented by the driver requires Giant in order to operate correctly. Add a 'struct task' to 'struct ifnet' that can be used to execute a deferred ifp->if_start() in the event that if_start needs to be called in a Giant-free environment. To do this, introduce if_start(), a wrapper function for ifp->if_start(). If the interface can run MPSAFE, it directly dispatches into the interface start routine. If it can't run MPSAFE, we're running with debug.mpsafenet != 0, and Giant isn't currently held, the task is queued to execute in a swi holding Giant via if_start_deferred(). Modify if_handoff() to use if_start() instead of direct dispatch. Modify 802.11 to use if_start() instead of direct dispatch. This is intended to provide increased compatibility for non-MPSAFE network device drivers in the presence of Giant-free operation via asynchronous dispatch. However, this commit does not mark any network interfaces as IFF_NEEDSGIANT.
2004-07-27 23:20:45 +00:00
}
- bump __FreeBSD version to reflect added buf_ring, memory barriers, and ifnet functions - add memory barriers to <machine/atomic.h> - update drivers to only conditionally define their own - add lockless producer / consumer ring buffer - remove ring buffer implementation from cxgb and update its callers - add if_transmit(struct ifnet *ifp, struct mbuf *m) to ifnet to allow drivers to efficiently manage multiple hardware queues (i.e. not serialize all packets through one ifq) - expose if_qflush to allow drivers to flush any driver managed queues This work was supported by Bitgravity Inc. and Chelsio Inc.
2008-11-22 05:55:56 +00:00
/*
* Backwards compatibility interface for drivers
* that have not implemented it
*/
static int
if_transmit(struct ifnet *ifp, struct mbuf *m)
{
int error;
IFQ_HANDOFF(ifp, m, error);
return (error);
}
Add if_input_default() method, that will be used for if_input initialization, when no input method specified before if_attach(). This prevents panics when if_input() method called directly e.g. from bpf(4) code. PR: 192426 Reviewed by: glebius MFC after: 1 week
2015-03-12 14:55:33 +00:00
static void
if_input_default(struct ifnet *ifp __unused, struct mbuf *m)
{
m_freem(m);
}
Move if_handoff() from an inline in if_var.h to a function to if.c in orden to harden the ABI for 5.x; this will permit us to modify the locking in the ifnet packet dispatch without requiring drivers to be recompiled. MFC after: 3 days Discussed at: EuroBSDCon Developer's Summit
2004-10-30 09:39:13 +00:00
int
if_handoff(struct ifqueue *ifq, struct mbuf *m, struct ifnet *ifp, int adjust)
{
int active = 0;
IF_LOCK(ifq);
if (_IF_QFULL(ifq)) {
IF_UNLOCK(ifq);
Finally, convert counters in struct ifnet to counter(9). Sponsored by: Netflix Sponsored by: Nginx, Inc.
2014-09-28 08:57:07 +00:00
if_inc_counter(ifp, IFCOUNTER_OQDROPS, 1);
Move if_handoff() from an inline in if_var.h to a function to if.c in orden to harden the ABI for 5.x; this will permit us to modify the locking in the ifnet packet dispatch without requiring drivers to be recompiled. MFC after: 3 days Discussed at: EuroBSDCon Developer's Summit
2004-10-30 09:39:13 +00:00
m_freem(m);
return (0);
}
if (ifp != NULL) {
Finally, convert counters in struct ifnet to counter(9). Sponsored by: Netflix Sponsored by: Nginx, Inc.
2014-09-28 08:57:07 +00:00
if_inc_counter(ifp, IFCOUNTER_OBYTES, m->m_pkthdr.len + adjust);
Move if_handoff() from an inline in if_var.h to a function to if.c in orden to harden the ABI for 5.x; this will permit us to modify the locking in the ifnet packet dispatch without requiring drivers to be recompiled. MFC after: 3 days Discussed at: EuroBSDCon Developer's Summit
2004-10-30 09:39:13 +00:00
if (m->m_flags & (M_BCAST|M_MCAST))
Finally, convert counters in struct ifnet to counter(9). Sponsored by: Netflix Sponsored by: Nginx, Inc.
2014-09-28 08:57:07 +00:00
if_inc_counter(ifp, IFCOUNTER_OMCASTS, 1);
Rename IFF_RUNNING to IFF_DRV_RUNNING, IFF_OACTIVE to IFF_DRV_OACTIVE, and move both flags from ifnet.if_flags to ifnet.if_drv_flags, making and documenting the locking of these flags the responsibility of the device driver, not the network stack. The flags for these two fields will be mutually exclusive so that they can be exposed to user space as though they were stored in the same variable. Provide #defines to provide the old names #ifndef _KERNEL, so that user applications (such as ifconfig) can use the old flag names. Using the old names in a device driver will result in a compile error in order to help device driver writers adopt the new model. When exposing the interface flags to user space, via interface ioctls or routing sockets, or the two fields together. Since the driver flags cannot currently be set for user space, no new logic is currently required to handle this case. Add some assertions that general purpose network stack routines, such as if_setflags(), are not improperly used on driver-owned flags. With this change, a large number of very minor network stack races are closed, subject to correct device driver locking. Most were likely never triggered. Driver sweep to follow; many thanks to pjd and bz for the line-by-line review they gave this patch. Reviewed by: pjd, bz MFC after: 7 days
2005-08-09 10:16:17 +00:00
active = ifp->if_drv_flags & IFF_DRV_OACTIVE;
Move if_handoff() from an inline in if_var.h to a function to if.c in orden to harden the ABI for 5.x; this will permit us to modify the locking in the ifnet packet dispatch without requiring drivers to be recompiled. MFC after: 3 days Discussed at: EuroBSDCon Developer's Summit
2004-10-30 09:39:13 +00:00
}
_IF_ENQUEUE(ifq, m);
IF_UNLOCK(ifq);
if (ifp != NULL && !active)
Remove IFF_NEEDSGIANT, a compatibility infrastructure introduced in FreeBSD 5.x to allow network device drivers to run with Giant despite the network stack being Giant-free. This significantly simplifies calls into ioctl() on network interfaces, especially in the multicast code, as well as eliminates deferred invocation of interface if_start routines. Disable the build on device drivers still depending on IFF_NEEDSGIANT as they no longer compile. They will be removed in a few weeks if they haven't been made MPSAFE in that time. Disabled drivers: if_ar if_axe if_aue if_cdce if_cue if_kue if_ray if_rue if_rum if_sr if_udav if_ural if_zyd Drivers that were already disabled because of tty changes: if_ppp if_sl Discussed on: arch@
2009-03-15 14:21:05 +00:00
(*(ifp)->if_start)(ifp);
Move if_handoff() from an inline in if_var.h to a function to if.c in orden to harden the ABI for 5.x; this will permit us to modify the locking in the ifnet packet dispatch without requiring drivers to be recompiled. MFC after: 3 days Discussed at: EuroBSDCon Developer's Summit
2004-10-30 09:39:13 +00:00
return (1);
}
Stop embedding struct ifnet at the top of driver softcs. Instead the struct ifnet or the layer 2 common structure it was embedded in have been replaced with a struct ifnet pointer to be filled by a call to the new function, if_alloc(). The layer 2 common structure is also allocated via if_alloc() based on the interface type. It is hung off the new struct ifnet member, if_l2com. This change removes the size of these structures from the kernel ABI and will allow us to better manage them as interfaces come and go. Other changes of note: - Struct arpcom is no longer referenced in normal interface code. Instead the Ethernet address is accessed via the IFP2ENADDR() macro. To enforce this ac_enaddr has been renamed to _ac_enaddr. - The second argument to ether_ifattach is now always the mac address from driver private storage rather than sometimes being ac_enaddr. Reviewed by: sobomax, sam
2005-06-10 16:49:24 +00:00
void
if_register_com_alloc(u_char type,
if_com_alloc_t *a, if_com_free_t *f)
{
KASSERT(if_com_alloc[type] == NULL,
("if_register_com_alloc: %d already registered", type));
KASSERT(if_com_free[type] == NULL,
("if_register_com_alloc: %d free already registered", type));
if_com_alloc[type] = a;
if_com_free[type] = f;
}
void
if_deregister_com_alloc(u_char type)
{
Fix KASSERT conditions in if_deregister_com_alloc().
2006-06-11 22:09:28 +00:00
KASSERT(if_com_alloc[type] != NULL,
Stop embedding struct ifnet at the top of driver softcs. Instead the struct ifnet or the layer 2 common structure it was embedded in have been replaced with a struct ifnet pointer to be filled by a call to the new function, if_alloc(). The layer 2 common structure is also allocated via if_alloc() based on the interface type. It is hung off the new struct ifnet member, if_l2com. This change removes the size of these structures from the kernel ABI and will allow us to better manage them as interfaces come and go. Other changes of note: - Struct arpcom is no longer referenced in normal interface code. Instead the Ethernet address is accessed via the IFP2ENADDR() macro. To enforce this ac_enaddr has been renamed to _ac_enaddr. - The second argument to ether_ifattach is now always the mac address from driver private storage rather than sometimes being ac_enaddr. Reviewed by: sobomax, sam
2005-06-10 16:49:24 +00:00
("if_deregister_com_alloc: %d not registered", type));
Fix KASSERT conditions in if_deregister_com_alloc().
2006-06-11 22:09:28 +00:00
KASSERT(if_com_free[type] != NULL,
Stop embedding struct ifnet at the top of driver softcs. Instead the struct ifnet or the layer 2 common structure it was embedded in have been replaced with a struct ifnet pointer to be filled by a call to the new function, if_alloc(). The layer 2 common structure is also allocated via if_alloc() based on the interface type. It is hung off the new struct ifnet member, if_l2com. This change removes the size of these structures from the kernel ABI and will allow us to better manage them as interfaces come and go. Other changes of note: - Struct arpcom is no longer referenced in normal interface code. Instead the Ethernet address is accessed via the IFP2ENADDR() macro. To enforce this ac_enaddr has been renamed to _ac_enaddr. - The second argument to ether_ifattach is now always the mac address from driver private storage rather than sometimes being ac_enaddr. Reviewed by: sobomax, sam
2005-06-10 16:49:24 +00:00
("if_deregister_com_alloc: %d free not registered", type));
if_com_alloc[type] = NULL;
if_com_free[type] = NULL;
}
Introduce a procedural interface to the ifnet structure. The new interface allows the ifnet structure to be defined as an opaque type in NIC drivers. This then allows the ifnet structure to be changed without a need to change or recompile NIC drivers. Put differently, NIC drivers can be written and compiled once and be used with different network stack implementations, provided of course that those network stack implementations have an API and ABI compatible interface. This commit introduces the 'if_t' type to replace 'struct ifnet *' as the type of a network interface. The 'if_t' type is defined as 'void *' to enable the compiler to perform type conversion to 'struct ifnet *' and vice versa where needed and without warnings. The functions that implement the API are the only functions that need to have an explicit cast. The MII code has been converted to use the driver API to avoid unnecessary code churn. Code churn comes from having to work with both converted and unconverted drivers in correlation with having callback functions that take an interface. By converting the MII code first, the callback functions can be defined so that the compiler will perform the typecasts automatically. As soon as all drivers have been converted, the if_t type can be redefined as needed and the API functions can be fix to not need an explicit cast. The immediate benefactors of this change are: 1. Juniper Networks - The network stack implementation in Junos is entirely different from FreeBSD's one and this change allows Juniper to build "stock" NIC drivers that can be used in combination with both the FreeBSD and Junos stacks. 2. FreeBSD - This change opens the door towards changing ifnet and implementing new features and optimizations in the network stack without it requiring a change in the many NIC drivers FreeBSD has. Submitted by: Anuranjan Shukla <anshukla@juniper.net> Reviewed by: glebius@ Obtained from: Juniper Networks, Inc.
2014-06-02 17:54:39 +00:00
/* API for driver access to network stack owned ifnet.*/
uint64_t
It is actually possible to have if_t a typedef to non-void type, and keep both converted to drvapi and non-converted drivers compilable. o Make if_t typedef to struct ifnet *. o Remove shim functions. Sponsored by: Netflix Sponsored by: Nginx, Inc.
2014-08-31 12:48:13 +00:00
if_setbaudrate(struct ifnet *ifp, uint64_t baudrate)
Introduce a procedural interface to the ifnet structure. The new interface allows the ifnet structure to be defined as an opaque type in NIC drivers. This then allows the ifnet structure to be changed without a need to change or recompile NIC drivers. Put differently, NIC drivers can be written and compiled once and be used with different network stack implementations, provided of course that those network stack implementations have an API and ABI compatible interface. This commit introduces the 'if_t' type to replace 'struct ifnet *' as the type of a network interface. The 'if_t' type is defined as 'void *' to enable the compiler to perform type conversion to 'struct ifnet *' and vice versa where needed and without warnings. The functions that implement the API are the only functions that need to have an explicit cast. The MII code has been converted to use the driver API to avoid unnecessary code churn. Code churn comes from having to work with both converted and unconverted drivers in correlation with having callback functions that take an interface. By converting the MII code first, the callback functions can be defined so that the compiler will perform the typecasts automatically. As soon as all drivers have been converted, the if_t type can be redefined as needed and the API functions can be fix to not need an explicit cast. The immediate benefactors of this change are: 1. Juniper Networks - The network stack implementation in Junos is entirely different from FreeBSD's one and this change allows Juniper to build "stock" NIC drivers that can be used in combination with both the FreeBSD and Junos stacks. 2. FreeBSD - This change opens the door towards changing ifnet and implementing new features and optimizations in the network stack without it requiring a change in the many NIC drivers FreeBSD has. Submitted by: Anuranjan Shukla <anshukla@juniper.net> Reviewed by: glebius@ Obtained from: Juniper Networks, Inc.
2014-06-02 17:54:39 +00:00
{
uint64_t oldbrate;
oldbrate = ifp->if_baudrate;
ifp->if_baudrate = baudrate;
return (oldbrate);
}
uint64_t
if_getbaudrate(if_t ifp)
{
return (((struct ifnet *)ifp)->if_baudrate);
}
int
if_setcapabilities(if_t ifp, int capabilities)
{
((struct ifnet *)ifp)->if_capabilities = capabilities;
return (0);
}
int
if_setcapabilitiesbit(if_t ifp, int setbit, int clearbit)
{
((struct ifnet *)ifp)->if_capabilities |= setbit;
((struct ifnet *)ifp)->if_capabilities &= ~clearbit;
return (0);
}
int
if_getcapabilities(if_t ifp)
{
return ((struct ifnet *)ifp)->if_capabilities;
}
int
if_setcapenable(if_t ifp, int capabilities)
{
((struct ifnet *)ifp)->if_capenable = capabilities;
return (0);
}
int
if_setcapenablebit(if_t ifp, int setcap, int clearcap)
{
if(setcap)
((struct ifnet *)ifp)->if_capenable |= setcap;
if(clearcap)
((struct ifnet *)ifp)->if_capenable &= ~clearcap;
return (0);
}
const char *
if_getdname(if_t ifp)
{
return ((struct ifnet *)ifp)->if_dname;
}
int
if_togglecapenable(if_t ifp, int togglecap)
{
((struct ifnet *)ifp)->if_capenable ^= togglecap;
return (0);
}
int
if_getcapenable(if_t ifp)
{
return ((struct ifnet *)ifp)->if_capenable;
}
/*
* This is largely undesirable because it ties ifnet to a device, but does
* provide flexiblity for an embedded product vendor. Should be used with
* the understanding that it violates the interface boundaries, and should be
* a last resort only.
*/
int
if_setdev(if_t ifp, void *dev)
{
return (0);
}
int
if_setdrvflagbits(if_t ifp, int set_flags, int clear_flags)
{
((struct ifnet *)ifp)->if_drv_flags |= set_flags;
((struct ifnet *)ifp)->if_drv_flags &= ~clear_flags;
return (0);
}
int
if_getdrvflags(if_t ifp)
{
return ((struct ifnet *)ifp)->if_drv_flags;
}
int
if_setdrvflags(if_t ifp, int flags)
{
((struct ifnet *)ifp)->if_drv_flags = flags;
return (0);
}
int
if_setflags(if_t ifp, int flags)
{
((struct ifnet *)ifp)->if_flags = flags;
return (0);
}
int
if_setflagbits(if_t ifp, int set, int clear)
{
((struct ifnet *)ifp)->if_flags |= set;
((struct ifnet *)ifp)->if_flags &= ~clear;
return (0);
}
int
if_getflags(if_t ifp)
{
return ((struct ifnet *)ifp)->if_flags;
}
int
if_clearhwassist(if_t ifp)
{
((struct ifnet *)ifp)->if_hwassist = 0;
return (0);
}
int
if_sethwassistbits(if_t ifp, int toset, int toclear)
{
((struct ifnet *)ifp)->if_hwassist |= toset;
((struct ifnet *)ifp)->if_hwassist &= ~toclear;
return (0);
}
int
if_sethwassist(if_t ifp, int hwassist_bit)
{
((struct ifnet *)ifp)->if_hwassist = hwassist_bit;
return (0);
}
int
if_gethwassist(if_t ifp)
{
return ((struct ifnet *)ifp)->if_hwassist;
}
int
if_setmtu(if_t ifp, int mtu)
{
((struct ifnet *)ifp)->if_mtu = mtu;
return (0);
}
int
if_getmtu(if_t ifp)
{
return ((struct ifnet *)ifp)->if_mtu;
}
Make checks for rt_mtu generic: Some virtual if drivers has (ab)used ifa ifa_rtrequest hook to enforce route MTU to be not bigger that interface MTU. While ifa_rtrequest hooking might be an option in some situation, it is not feasible to do MTU checks there: generic (or per-domain) routing code is perfectly capable of doing this. We currrently have 3 places where MTU is altered: 1) route addition. In this case domain overrides radix _addroute callback (in[6]_addroute) and all necessary checks/fixes are/can be done there. 2) route change (especially, GW change). In this case, there are no explicit per-domain calls, but one can override rte by setting ifa_rtrequest hook to domain handler (inet6 does this). 3) ifconfig ifaceX mtu YYYY In this case, we have no callbacks, but ip[6]_output performes runtime checks and decreases rt_mtu if necessary. Generally, the goals are to be able to handle all MTU changes in control plane, not in runtime part, and properly deal with increased interface MTU. This commit changes the following: * removes hooks setting MTU from drivers side * adds proper per-doman MTU checks for case 1) * adds generic MTU check for case 2) * The latter is done by using new dom_ifmtu callback since if_mtu denotes L3 interface MTU, e.g. maximum trasmitted _packet_ size. However, IPv6 mtu might be different from if_mtu one (e.g. default 1280) for some cases, so we need an abstract way to know maximum MTU size for given interface and domain. * moves rt_setmetrics() before MTU/ifa_rtrequest hooks since it copies user-supplied data which must be checked. * removes RT_LOCK_ASSERT() from other ifa_rtrequest hooks to be able to use this functions on new non-inserted rte. More changes will follow soon. MFC after: 1 month Sponsored by: Yandex LLC
2014-11-06 13:13:09 +00:00
int
if_getmtu_family(if_t ifp, int family)
{
struct domain *dp;
for (dp = domains; dp; dp = dp->dom_next) {
if (dp->dom_family == family && dp->dom_ifmtu != NULL)
return (dp->dom_ifmtu((struct ifnet *)ifp));
}
return (((struct ifnet *)ifp)->if_mtu);
}
Provide new KPI for network drivers to access lists of interface addresses. The KPI doesn't reveal neither how addresses are stored, how the access to them is synchronized, neither reveal struct ifaddr and struct ifmaddr. Reviewed by: gallatin, erj, hselasky, philip, stevek Differential Revision: https://reviews.freebsd.org/D21943
2019-10-10 23:42:55 +00:00
/*
* Methods for drivers to access interface unicast and multicast
* link level addresses. Driver shall not know 'struct ifaddr' neither
* 'struct ifmultiaddr'.
*/
Add two extra functions that basically give count of addresses on interface. Such function could been implemented on top of the if_foreach_llm?addr(), but several drivers need counting, so avoid copy-n-paste inside the drivers.
2019-10-10 23:44:56 +00:00
u_int
if_lladdr_count(if_t ifp)
{
struct epoch_tracker et;
struct ifaddr *ifa;
u_int count;
count = 0;
NET_EPOCH_ENTER(et);
CK_STAILQ_FOREACH(ifa, &ifp->if_addrhead, ifa_link)
if (ifa->ifa_addr->sa_family == AF_LINK)
count++;
NET_EPOCH_EXIT(et);
return (count);
}
Provide new KPI for network drivers to access lists of interface addresses. The KPI doesn't reveal neither how addresses are stored, how the access to them is synchronized, neither reveal struct ifaddr and struct ifmaddr. Reviewed by: gallatin, erj, hselasky, philip, stevek Differential Revision: https://reviews.freebsd.org/D21943
2019-10-10 23:42:55 +00:00
u_int
if_foreach_lladdr(if_t ifp, iflladdr_cb_t cb, void *cb_arg)
{
struct epoch_tracker et;
struct ifaddr *ifa;
u_int count;
MPASS(cb);
count = 0;
NET_EPOCH_ENTER(et);
CK_STAILQ_FOREACH(ifa, &ifp->if_addrhead, ifa_link) {
if (ifa->ifa_addr->sa_family != AF_LINK)
continue;
count += (*cb)(cb_arg, (struct sockaddr_dl *)ifa->ifa_addr,
count);
}
NET_EPOCH_EXIT(et);
return (count);
}
Add two extra functions that basically give count of addresses on interface. Such function could been implemented on top of the if_foreach_llm?addr(), but several drivers need counting, so avoid copy-n-paste inside the drivers.
2019-10-10 23:44:56 +00:00
u_int
if_llmaddr_count(if_t ifp)
{
struct epoch_tracker et;
struct ifmultiaddr *ifma;
int count;
count = 0;
NET_EPOCH_ENTER(et);
CK_STAILQ_FOREACH(ifma, &ifp->if_multiaddrs, ifma_link)
if (ifma->ifma_addr->sa_family == AF_LINK)
count++;
NET_EPOCH_EXIT(et);
return (count);
}
Provide new KPI for network drivers to access lists of interface addresses. The KPI doesn't reveal neither how addresses are stored, how the access to them is synchronized, neither reveal struct ifaddr and struct ifmaddr. Reviewed by: gallatin, erj, hselasky, philip, stevek Differential Revision: https://reviews.freebsd.org/D21943
2019-10-10 23:42:55 +00:00
u_int
if_foreach_llmaddr(if_t ifp, iflladdr_cb_t cb, void *cb_arg)
{
struct epoch_tracker et;
struct ifmultiaddr *ifma;
u_int count;
MPASS(cb);
count = 0;
NET_EPOCH_ENTER(et);
CK_STAILQ_FOREACH(ifma, &ifp->if_multiaddrs, ifma_link) {
if (ifma->ifma_addr->sa_family != AF_LINK)
continue;
count += (*cb)(cb_arg, (struct sockaddr_dl *)ifma->ifma_addr,
count);
}
NET_EPOCH_EXIT(et);
return (count);
}
Introduce a procedural interface to the ifnet structure. The new interface allows the ifnet structure to be defined as an opaque type in NIC drivers. This then allows the ifnet structure to be changed without a need to change or recompile NIC drivers. Put differently, NIC drivers can be written and compiled once and be used with different network stack implementations, provided of course that those network stack implementations have an API and ABI compatible interface. This commit introduces the 'if_t' type to replace 'struct ifnet *' as the type of a network interface. The 'if_t' type is defined as 'void *' to enable the compiler to perform type conversion to 'struct ifnet *' and vice versa where needed and without warnings. The functions that implement the API are the only functions that need to have an explicit cast. The MII code has been converted to use the driver API to avoid unnecessary code churn. Code churn comes from having to work with both converted and unconverted drivers in correlation with having callback functions that take an interface. By converting the MII code first, the callback functions can be defined so that the compiler will perform the typecasts automatically. As soon as all drivers have been converted, the if_t type can be redefined as needed and the API functions can be fix to not need an explicit cast. The immediate benefactors of this change are: 1. Juniper Networks - The network stack implementation in Junos is entirely different from FreeBSD's one and this change allows Juniper to build "stock" NIC drivers that can be used in combination with both the FreeBSD and Junos stacks. 2. FreeBSD - This change opens the door towards changing ifnet and implementing new features and optimizations in the network stack without it requiring a change in the many NIC drivers FreeBSD has. Submitted by: Anuranjan Shukla <anshukla@juniper.net> Reviewed by: glebius@ Obtained from: Juniper Networks, Inc.
2014-06-02 17:54:39 +00:00
int
if_setsoftc(if_t ifp, void *softc)
{
((struct ifnet *)ifp)->if_softc = softc;
return (0);
}
void *
if_getsoftc(if_t ifp)
{
return ((struct ifnet *)ifp)->if_softc;
}
void
if_setrcvif(struct mbuf *m, if_t ifp)
{
Restructure mbuf send tags to provide stronger guarantees. - Perform ifp mismatch checks (to determine if a send tag is allocated for a different ifp than the one the packet is being output on), in ip_output() and ip6_output(). This avoids sending packets with send tags to ifnet drivers that don't support send tags. Since we are now checking for ifp mismatches before invoking if_output, we can now try to allocate a new tag before invoking if_output sending the original packet on the new tag if allocation succeeds. To avoid code duplication for the fragment and unfragmented cases, add ip_output_send() and ip6_output_send() as wrappers around if_output and nd6_output_ifp, respectively. All of the logic for setting send tags and dealing with send tag-related errors is done in these wrapper functions. For pseudo interfaces that wrap other network interfaces (vlan and lagg), wrapper send tags are now allocated so that ip*_output see the wrapper ifp as the ifp in the send tag. The if_transmit routines rewrite the send tags after performing an ifp mismatch check. If an ifp mismatch is detected, the transmit routines fail with EAGAIN. - To provide clearer life cycle management of send tags, especially in the presence of vlan and lagg wrapper tags, add a reference count to send tags managed via m_snd_tag_ref() and m_snd_tag_rele(). Provide a helper function (m_snd_tag_init()) for use by drivers supporting send tags. m_snd_tag_init() takes care of the if_ref on the ifp meaning that code alloating send tags via if_snd_tag_alloc no longer has to manage that manually. Similarly, m_snd_tag_rele drops the refcount on the ifp after invoking if_snd_tag_free when the last reference to a send tag is dropped. This also closes use after free races if there are pending packets in driver tx rings after the socket is closed (e.g. from tcpdrop). In order for m_free to work reliably, add a new CSUM_SND_TAG flag in csum_flags to indicate 'snd_tag' is set (rather than 'rcvif'). Drivers now also check this flag instead of checking snd_tag against NULL. This avoids false positive matches when a forwarded packet has a non-NULL rcvif that was treated as a send tag. - cxgbe was relying on snd_tag_free being called when the inp was detached so that it could kick the firmware to flush any pending work on the flow. This is because the driver doesn't require ACK messages from the firmware for every request, but instead does a kind of manual interrupt coalescing by only setting a flag to request a completion on a subset of requests. If all of the in-flight requests don't have the flag when the tag is detached from the inp, the flow might never return the credits. The current snd_tag_free command issues a flush command to force the credits to return. However, the credit return is what also frees the mbufs, and since those mbufs now hold references on the tag, this meant that snd_tag_free would never be called. To fix, explicitly drop the mbuf's reference on the snd tag when the mbuf is queued in the firmware work queue. This means that once the inp's reference on the tag goes away and all in-flight mbufs have been queued to the firmware, tag's refcount will drop to zero and snd_tag_free will kick in and send the flush request. Note that we need to avoid doing this in the middle of ethofld_tx(), so the driver grabs a temporary reference on the tag around that loop to defer the free to the end of the function in case it sends the last mbuf to the queue after the inp has dropped its reference on the tag. - mlx5 preallocates send tags and was using the ifp pointer even when the send tag wasn't in use. Explicitly use the ifp from other data structures instead. - Sprinkle some assertions in various places to assert that received packets don't have a send tag, and that other places that overwrite rcvif (e.g. 802.11 transmit) don't clobber a send tag pointer. Reviewed by: gallatin, hselasky, rgrimes, ae Sponsored by: Netflix Differential Revision: https://reviews.freebsd.org/D20117
2019-05-24 22:30:40 +00:00
MPASS((m->m_pkthdr.csum_flags & CSUM_SND_TAG) == 0);
Introduce a procedural interface to the ifnet structure. The new interface allows the ifnet structure to be defined as an opaque type in NIC drivers. This then allows the ifnet structure to be changed without a need to change or recompile NIC drivers. Put differently, NIC drivers can be written and compiled once and be used with different network stack implementations, provided of course that those network stack implementations have an API and ABI compatible interface. This commit introduces the 'if_t' type to replace 'struct ifnet *' as the type of a network interface. The 'if_t' type is defined as 'void *' to enable the compiler to perform type conversion to 'struct ifnet *' and vice versa where needed and without warnings. The functions that implement the API are the only functions that need to have an explicit cast. The MII code has been converted to use the driver API to avoid unnecessary code churn. Code churn comes from having to work with both converted and unconverted drivers in correlation with having callback functions that take an interface. By converting the MII code first, the callback functions can be defined so that the compiler will perform the typecasts automatically. As soon as all drivers have been converted, the if_t type can be redefined as needed and the API functions can be fix to not need an explicit cast. The immediate benefactors of this change are: 1. Juniper Networks - The network stack implementation in Junos is entirely different from FreeBSD's one and this change allows Juniper to build "stock" NIC drivers that can be used in combination with both the FreeBSD and Junos stacks. 2. FreeBSD - This change opens the door towards changing ifnet and implementing new features and optimizations in the network stack without it requiring a change in the many NIC drivers FreeBSD has. Submitted by: Anuranjan Shukla <anshukla@juniper.net> Reviewed by: glebius@ Obtained from: Juniper Networks, Inc.
2014-06-02 17:54:39 +00:00
m->m_pkthdr.rcvif = (struct ifnet *)ifp;
}
void
if_setvtag(struct mbuf *m, uint16_t tag)
{
m->m_pkthdr.ether_vtag = tag;
}
uint16_t
if_getvtag(struct mbuf *m)
{
return (m->m_pkthdr.ether_vtag);
}
int
if_sendq_empty(if_t ifp)
{
return IFQ_DRV_IS_EMPTY(&((struct ifnet *)ifp)->if_snd);
}
struct ifaddr *
if_getifaddr(if_t ifp)
{
return ((struct ifnet *)ifp)->if_addr;
}
int
if_getamcount(if_t ifp)
{
return ((struct ifnet *)ifp)->if_amcount;
}
int
if_setsendqready(if_t ifp)
{
IFQ_SET_READY(&((struct ifnet *)ifp)->if_snd);
return (0);
}
int
if_setsendqlen(if_t ifp, int tx_desc_count)
{
IFQ_SET_MAXLEN(&((struct ifnet *)ifp)->if_snd, tx_desc_count);
((struct ifnet *)ifp)->if_snd.ifq_drv_maxlen = tx_desc_count;
return (0);
}
int
if_vlantrunkinuse(if_t ifp)
{
return ((struct ifnet *)ifp)->if_vlantrunk != NULL?1:0;
}
int
if_input(if_t ifp, struct mbuf* sendmp)
{
(*((struct ifnet *)ifp)->if_input)((struct ifnet *)ifp, sendmp);
return (0);
}
/* XXX */
#ifndef ETH_ADDR_LEN
#define ETH_ADDR_LEN 6
#endif
int
if_setupmultiaddr(if_t ifp, void *mta, int *cnt, int max)
{
struct ifmultiaddr *ifma;
uint8_t *lmta = (uint8_t *)mta;
int mcnt = 0;
ifnet: Replace if_addr_lock rwlock with epoch + mutex Run on LLNW canaries and tested by pho@ gallatin: Using a 14-core, 28-HTT single socket E5-2697 v3 with a 40GbE MLX5 based ConnectX 4-LX NIC, I see an almost 12% improvement in received packet rate, and a larger improvement in bytes delivered all the way to userspace. When the host receiving 64 streams of netperf -H $DUT -t UDP_STREAM -- -m 1, I see, using nstat -I mce0 1 before the patch: InMpps OMpps InGbs OGbs err TCP Est %CPU syscalls csw irq GBfree 4.98 0.00 4.42 0.00 4235592 33 83.80 4720653 2149771 1235 247.32 4.73 0.00 4.20 0.00 4025260 33 82.99 4724900 2139833 1204 247.32 4.72 0.00 4.20 0.00 4035252 33 82.14 4719162 2132023 1264 247.32 4.71 0.00 4.21 0.00 4073206 33 83.68 4744973 2123317 1347 247.32 4.72 0.00 4.21 0.00 4061118 33 80.82 4713615 2188091 1490 247.32 4.72 0.00 4.21 0.00 4051675 33 85.29 4727399 2109011 1205 247.32 4.73 0.00 4.21 0.00 4039056 33 84.65 4724735 2102603 1053 247.32 After the patch InMpps OMpps InGbs OGbs err TCP Est %CPU syscalls csw irq GBfree 5.43 0.00 4.20 0.00 3313143 33 84.96 5434214 1900162 2656 245.51 5.43 0.00 4.20 0.00 3308527 33 85.24 5439695 1809382 2521 245.51 5.42 0.00 4.19 0.00 3316778 33 87.54 5416028 1805835 2256 245.51 5.42 0.00 4.19 0.00 3317673 33 90.44 5426044 1763056 2332 245.51 5.42 0.00 4.19 0.00 3314839 33 88.11 5435732 1792218 2499 245.52 5.44 0.00 4.19 0.00 3293228 33 91.84 5426301 1668597 2121 245.52 Similarly, netperf reports 230Mb/s before the patch, and 270Mb/s after the patch Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15366
2018-05-18 20:13:34 +00:00
CK_STAILQ_FOREACH(ifma, &((struct ifnet *)ifp)->if_multiaddrs, ifma_link) {
Introduce a procedural interface to the ifnet structure. The new interface allows the ifnet structure to be defined as an opaque type in NIC drivers. This then allows the ifnet structure to be changed without a need to change or recompile NIC drivers. Put differently, NIC drivers can be written and compiled once and be used with different network stack implementations, provided of course that those network stack implementations have an API and ABI compatible interface. This commit introduces the 'if_t' type to replace 'struct ifnet *' as the type of a network interface. The 'if_t' type is defined as 'void *' to enable the compiler to perform type conversion to 'struct ifnet *' and vice versa where needed and without warnings. The functions that implement the API are the only functions that need to have an explicit cast. The MII code has been converted to use the driver API to avoid unnecessary code churn. Code churn comes from having to work with both converted and unconverted drivers in correlation with having callback functions that take an interface. By converting the MII code first, the callback functions can be defined so that the compiler will perform the typecasts automatically. As soon as all drivers have been converted, the if_t type can be redefined as needed and the API functions can be fix to not need an explicit cast. The immediate benefactors of this change are: 1. Juniper Networks - The network stack implementation in Junos is entirely different from FreeBSD's one and this change allows Juniper to build "stock" NIC drivers that can be used in combination with both the FreeBSD and Junos stacks. 2. FreeBSD - This change opens the door towards changing ifnet and implementing new features and optimizations in the network stack without it requiring a change in the many NIC drivers FreeBSD has. Submitted by: Anuranjan Shukla <anshukla@juniper.net> Reviewed by: glebius@ Obtained from: Juniper Networks, Inc.
2014-06-02 17:54:39 +00:00
if (ifma->ifma_addr->sa_family != AF_LINK)
continue;
if (mcnt == max)
break;
bcopy(LLADDR((struct sockaddr_dl *)ifma->ifma_addr),
&lmta[mcnt * ETH_ADDR_LEN], ETH_ADDR_LEN);
mcnt++;
}
*cnt = mcnt;
return (0);
}
int
if_multiaddr_array(if_t ifp, void *mta, int *cnt, int max)
{
int error;
if_maddr_rlock(ifp);
error = if_setupmultiaddr(ifp, mta, cnt, max);
if_maddr_runlock(ifp);
return (error);
}
int
if_multiaddr_count(if_t ifp, int max)
{
struct ifmultiaddr *ifma;
int count;
count = 0;
if_maddr_rlock(ifp);
ifnet: Replace if_addr_lock rwlock with epoch + mutex Run on LLNW canaries and tested by pho@ gallatin: Using a 14-core, 28-HTT single socket E5-2697 v3 with a 40GbE MLX5 based ConnectX 4-LX NIC, I see an almost 12% improvement in received packet rate, and a larger improvement in bytes delivered all the way to userspace. When the host receiving 64 streams of netperf -H $DUT -t UDP_STREAM -- -m 1, I see, using nstat -I mce0 1 before the patch: InMpps OMpps InGbs OGbs err TCP Est %CPU syscalls csw irq GBfree 4.98 0.00 4.42 0.00 4235592 33 83.80 4720653 2149771 1235 247.32 4.73 0.00 4.20 0.00 4025260 33 82.99 4724900 2139833 1204 247.32 4.72 0.00 4.20 0.00 4035252 33 82.14 4719162 2132023 1264 247.32 4.71 0.00 4.21 0.00 4073206 33 83.68 4744973 2123317 1347 247.32 4.72 0.00 4.21 0.00 4061118 33 80.82 4713615 2188091 1490 247.32 4.72 0.00 4.21 0.00 4051675 33 85.29 4727399 2109011 1205 247.32 4.73 0.00 4.21 0.00 4039056 33 84.65 4724735 2102603 1053 247.32 After the patch InMpps OMpps InGbs OGbs err TCP Est %CPU syscalls csw irq GBfree 5.43 0.00 4.20 0.00 3313143 33 84.96 5434214 1900162 2656 245.51 5.43 0.00 4.20 0.00 3308527 33 85.24 5439695 1809382 2521 245.51 5.42 0.00 4.19 0.00 3316778 33 87.54 5416028 1805835 2256 245.51 5.42 0.00 4.19 0.00 3317673 33 90.44 5426044 1763056 2332 245.51 5.42 0.00 4.19 0.00 3314839 33 88.11 5435732 1792218 2499 245.52 5.44 0.00 4.19 0.00 3293228 33 91.84 5426301 1668597 2121 245.52 Similarly, netperf reports 230Mb/s before the patch, and 270Mb/s after the patch Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15366
2018-05-18 20:13:34 +00:00
CK_STAILQ_FOREACH(ifma, &((struct ifnet *)ifp)->if_multiaddrs, ifma_link) {
Introduce a procedural interface to the ifnet structure. The new interface allows the ifnet structure to be defined as an opaque type in NIC drivers. This then allows the ifnet structure to be changed without a need to change or recompile NIC drivers. Put differently, NIC drivers can be written and compiled once and be used with different network stack implementations, provided of course that those network stack implementations have an API and ABI compatible interface. This commit introduces the 'if_t' type to replace 'struct ifnet *' as the type of a network interface. The 'if_t' type is defined as 'void *' to enable the compiler to perform type conversion to 'struct ifnet *' and vice versa where needed and without warnings. The functions that implement the API are the only functions that need to have an explicit cast. The MII code has been converted to use the driver API to avoid unnecessary code churn. Code churn comes from having to work with both converted and unconverted drivers in correlation with having callback functions that take an interface. By converting the MII code first, the callback functions can be defined so that the compiler will perform the typecasts automatically. As soon as all drivers have been converted, the if_t type can be redefined as needed and the API functions can be fix to not need an explicit cast. The immediate benefactors of this change are: 1. Juniper Networks - The network stack implementation in Junos is entirely different from FreeBSD's one and this change allows Juniper to build "stock" NIC drivers that can be used in combination with both the FreeBSD and Junos stacks. 2. FreeBSD - This change opens the door towards changing ifnet and implementing new features and optimizations in the network stack without it requiring a change in the many NIC drivers FreeBSD has. Submitted by: Anuranjan Shukla <anshukla@juniper.net> Reviewed by: glebius@ Obtained from: Juniper Networks, Inc.
2014-06-02 17:54:39 +00:00
if (ifma->ifma_addr->sa_family != AF_LINK)
continue;
count++;
if (count == max)
break;
}
if_maddr_runlock(ifp);
return (count);
}
Import the 'iflib' API library for network drivers. From the author: "iflib is a library to eliminate the need for frequently duplicated device independent logic propagated (poorly) across many network drivers." Participation is purely optional. The IFLIB kernel config option is provided for drivers that want to transition between legacy and iflib modes of operation. ixl and ixgbe driver conversions will be committed shortly. We hope to see participation from the Broadcom and maybe Chelsio drivers in the near future. Submitted by: mmacy@nextbsd.org Reviewed by: gallatin Differential Revision: D5211
2016-05-18 04:35:58 +00:00
int
if_multi_apply(struct ifnet *ifp, int (*filter)(void *, struct ifmultiaddr *, int), void *arg)
{
struct ifmultiaddr *ifma;
int cnt = 0;
if_maddr_rlock(ifp);
ifnet: Replace if_addr_lock rwlock with epoch + mutex Run on LLNW canaries and tested by pho@ gallatin: Using a 14-core, 28-HTT single socket E5-2697 v3 with a 40GbE MLX5 based ConnectX 4-LX NIC, I see an almost 12% improvement in received packet rate, and a larger improvement in bytes delivered all the way to userspace. When the host receiving 64 streams of netperf -H $DUT -t UDP_STREAM -- -m 1, I see, using nstat -I mce0 1 before the patch: InMpps OMpps InGbs OGbs err TCP Est %CPU syscalls csw irq GBfree 4.98 0.00 4.42 0.00 4235592 33 83.80 4720653 2149771 1235 247.32 4.73 0.00 4.20 0.00 4025260 33 82.99 4724900 2139833 1204 247.32 4.72 0.00 4.20 0.00 4035252 33 82.14 4719162 2132023 1264 247.32 4.71 0.00 4.21 0.00 4073206 33 83.68 4744973 2123317 1347 247.32 4.72 0.00 4.21 0.00 4061118 33 80.82 4713615 2188091 1490 247.32 4.72 0.00 4.21 0.00 4051675 33 85.29 4727399 2109011 1205 247.32 4.73 0.00 4.21 0.00 4039056 33 84.65 4724735 2102603 1053 247.32 After the patch InMpps OMpps InGbs OGbs err TCP Est %CPU syscalls csw irq GBfree 5.43 0.00 4.20 0.00 3313143 33 84.96 5434214 1900162 2656 245.51 5.43 0.00 4.20 0.00 3308527 33 85.24 5439695 1809382 2521 245.51 5.42 0.00 4.19 0.00 3316778 33 87.54 5416028 1805835 2256 245.51 5.42 0.00 4.19 0.00 3317673 33 90.44 5426044 1763056 2332 245.51 5.42 0.00 4.19 0.00 3314839 33 88.11 5435732 1792218 2499 245.52 5.44 0.00 4.19 0.00 3293228 33 91.84 5426301 1668597 2121 245.52 Similarly, netperf reports 230Mb/s before the patch, and 270Mb/s after the patch Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15366
2018-05-18 20:13:34 +00:00
CK_STAILQ_FOREACH(ifma, &ifp->if_multiaddrs, ifma_link)
Import the 'iflib' API library for network drivers. From the author: "iflib is a library to eliminate the need for frequently duplicated device independent logic propagated (poorly) across many network drivers." Participation is purely optional. The IFLIB kernel config option is provided for drivers that want to transition between legacy and iflib modes of operation. ixl and ixgbe driver conversions will be committed shortly. We hope to see participation from the Broadcom and maybe Chelsio drivers in the near future. Submitted by: mmacy@nextbsd.org Reviewed by: gallatin Differential Revision: D5211
2016-05-18 04:35:58 +00:00
cnt += filter(arg, ifma, cnt);
if_maddr_runlock(ifp);
return (cnt);
}
Introduce a procedural interface to the ifnet structure. The new interface allows the ifnet structure to be defined as an opaque type in NIC drivers. This then allows the ifnet structure to be changed without a need to change or recompile NIC drivers. Put differently, NIC drivers can be written and compiled once and be used with different network stack implementations, provided of course that those network stack implementations have an API and ABI compatible interface. This commit introduces the 'if_t' type to replace 'struct ifnet *' as the type of a network interface. The 'if_t' type is defined as 'void *' to enable the compiler to perform type conversion to 'struct ifnet *' and vice versa where needed and without warnings. The functions that implement the API are the only functions that need to have an explicit cast. The MII code has been converted to use the driver API to avoid unnecessary code churn. Code churn comes from having to work with both converted and unconverted drivers in correlation with having callback functions that take an interface. By converting the MII code first, the callback functions can be defined so that the compiler will perform the typecasts automatically. As soon as all drivers have been converted, the if_t type can be redefined as needed and the API functions can be fix to not need an explicit cast. The immediate benefactors of this change are: 1. Juniper Networks - The network stack implementation in Junos is entirely different from FreeBSD's one and this change allows Juniper to build "stock" NIC drivers that can be used in combination with both the FreeBSD and Junos stacks. 2. FreeBSD - This change opens the door towards changing ifnet and implementing new features and optimizations in the network stack without it requiring a change in the many NIC drivers FreeBSD has. Submitted by: Anuranjan Shukla <anshukla@juniper.net> Reviewed by: glebius@ Obtained from: Juniper Networks, Inc.
2014-06-02 17:54:39 +00:00
struct mbuf *
if_dequeue(if_t ifp)
{
struct mbuf *m;
IFQ_DRV_DEQUEUE(&((struct ifnet *)ifp)->if_snd, m);
return (m);
}
int
if_sendq_prepend(if_t ifp, struct mbuf *m)
{
IFQ_DRV_PREPEND(&((struct ifnet *)ifp)->if_snd, m);
return (0);
}
int
if_setifheaderlen(if_t ifp, int len)
{
o Remove struct if_data from struct ifnet. Now it is merely API structure for route(4) socket and ifmib(4) sysctl. o Move fields from if_data to ifnet, but keep all statistic counters separate, since they should disappear later. o Provide function if_data_copy() to fill if_data, utilize it in routing socket and ifmib handler. o Provide overridable ifnet(9) method to fetch counters. If no provided, if_get_counters_compat() would be used, that returns old counters. Sponsored by: Netflix Sponsored by: Nginx, Inc.
2014-08-31 06:46:21 +00:00
((struct ifnet *)ifp)->if_hdrlen = len;
Introduce a procedural interface to the ifnet structure. The new interface allows the ifnet structure to be defined as an opaque type in NIC drivers. This then allows the ifnet structure to be changed without a need to change or recompile NIC drivers. Put differently, NIC drivers can be written and compiled once and be used with different network stack implementations, provided of course that those network stack implementations have an API and ABI compatible interface. This commit introduces the 'if_t' type to replace 'struct ifnet *' as the type of a network interface. The 'if_t' type is defined as 'void *' to enable the compiler to perform type conversion to 'struct ifnet *' and vice versa where needed and without warnings. The functions that implement the API are the only functions that need to have an explicit cast. The MII code has been converted to use the driver API to avoid unnecessary code churn. Code churn comes from having to work with both converted and unconverted drivers in correlation with having callback functions that take an interface. By converting the MII code first, the callback functions can be defined so that the compiler will perform the typecasts automatically. As soon as all drivers have been converted, the if_t type can be redefined as needed and the API functions can be fix to not need an explicit cast. The immediate benefactors of this change are: 1. Juniper Networks - The network stack implementation in Junos is entirely different from FreeBSD's one and this change allows Juniper to build "stock" NIC drivers that can be used in combination with both the FreeBSD and Junos stacks. 2. FreeBSD - This change opens the door towards changing ifnet and implementing new features and optimizations in the network stack without it requiring a change in the many NIC drivers FreeBSD has. Submitted by: Anuranjan Shukla <anshukla@juniper.net> Reviewed by: glebius@ Obtained from: Juniper Networks, Inc.
2014-06-02 17:54:39 +00:00
return (0);
}
caddr_t
if_getlladdr(if_t ifp)
{
return (IF_LLADDR((struct ifnet *)ifp));
}
void *
if_gethandle(u_char type)
{
return (if_alloc(type));
}
void
if_bpfmtap(if_t ifh, struct mbuf *m)
{
struct ifnet *ifp = (struct ifnet *)ifh;
BPF_MTAP(ifp, m);
}
void
if_etherbpfmtap(if_t ifh, struct mbuf *m)
{
struct ifnet *ifp = (struct ifnet *)ifh;
ETHER_BPF_MTAP(ifp, m);
}
void
if_vlancap(if_t ifh)
{
struct ifnet *ifp = (struct ifnet *)ifh;
VLAN_CAPABILITIES(ifp);
}
Add the folowing set accessor functions for recently-added members of ifnet structure: if_gethwtsomax(), if_sethwtsomax() - if_hw_tsomax if_gethwtsomaxsegcount(), if_sethwtsomaxsegcount() - if_hw_tsomaxsegcount if_gethwtsomaxsegsize(), if_sethwtsomaxsegsize() - if_hw_tsomaxsegsize Update em and vnic drivers which had already been coverted to use accessor functions for the other ifnet structure members. Reviewed by: erj Approved by: sjg (mentor) Obtained from: Juniper Networks, Inc. Differential Revision: https://reviews.freebsd.org/D8544
2017-01-31 16:12:31 +00:00
int
if_sethwtsomax(if_t ifp, u_int if_hw_tsomax)
{
((struct ifnet *)ifp)->if_hw_tsomax = if_hw_tsomax;
return (0);
}
int
if_sethwtsomaxsegcount(if_t ifp, u_int if_hw_tsomaxsegcount)
{
((struct ifnet *)ifp)->if_hw_tsomaxsegcount = if_hw_tsomaxsegcount;
return (0);
}
int
if_sethwtsomaxsegsize(if_t ifp, u_int if_hw_tsomaxsegsize)
{
((struct ifnet *)ifp)->if_hw_tsomaxsegsize = if_hw_tsomaxsegsize;
return (0);
}
u_int
if_gethwtsomax(if_t ifp)
{
return (((struct ifnet *)ifp)->if_hw_tsomax);
}
u_int
if_gethwtsomaxsegcount(if_t ifp)
{
return (((struct ifnet *)ifp)->if_hw_tsomaxsegcount);
}
u_int
if_gethwtsomaxsegsize(if_t ifp)
{
return (((struct ifnet *)ifp)->if_hw_tsomaxsegsize);
}
Introduce a procedural interface to the ifnet structure. The new interface allows the ifnet structure to be defined as an opaque type in NIC drivers. This then allows the ifnet structure to be changed without a need to change or recompile NIC drivers. Put differently, NIC drivers can be written and compiled once and be used with different network stack implementations, provided of course that those network stack implementations have an API and ABI compatible interface. This commit introduces the 'if_t' type to replace 'struct ifnet *' as the type of a network interface. The 'if_t' type is defined as 'void *' to enable the compiler to perform type conversion to 'struct ifnet *' and vice versa where needed and without warnings. The functions that implement the API are the only functions that need to have an explicit cast. The MII code has been converted to use the driver API to avoid unnecessary code churn. Code churn comes from having to work with both converted and unconverted drivers in correlation with having callback functions that take an interface. By converting the MII code first, the callback functions can be defined so that the compiler will perform the typecasts automatically. As soon as all drivers have been converted, the if_t type can be redefined as needed and the API functions can be fix to not need an explicit cast. The immediate benefactors of this change are: 1. Juniper Networks - The network stack implementation in Junos is entirely different from FreeBSD's one and this change allows Juniper to build "stock" NIC drivers that can be used in combination with both the FreeBSD and Junos stacks. 2. FreeBSD - This change opens the door towards changing ifnet and implementing new features and optimizations in the network stack without it requiring a change in the many NIC drivers FreeBSD has. Submitted by: Anuranjan Shukla <anshukla@juniper.net> Reviewed by: glebius@ Obtained from: Juniper Networks, Inc.
2014-06-02 17:54:39 +00:00
void
if_setinitfn(if_t ifp, void (*init_fn)(void *))
{
((struct ifnet *)ifp)->if_init = init_fn;
}
void
It is actually possible to have if_t a typedef to non-void type, and keep both converted to drvapi and non-converted drivers compilable. o Make if_t typedef to struct ifnet *. o Remove shim functions. Sponsored by: Netflix Sponsored by: Nginx, Inc.
2014-08-31 12:48:13 +00:00
if_setioctlfn(if_t ifp, int (*ioctl_fn)(if_t, u_long, caddr_t))
Introduce a procedural interface to the ifnet structure. The new interface allows the ifnet structure to be defined as an opaque type in NIC drivers. This then allows the ifnet structure to be changed without a need to change or recompile NIC drivers. Put differently, NIC drivers can be written and compiled once and be used with different network stack implementations, provided of course that those network stack implementations have an API and ABI compatible interface. This commit introduces the 'if_t' type to replace 'struct ifnet *' as the type of a network interface. The 'if_t' type is defined as 'void *' to enable the compiler to perform type conversion to 'struct ifnet *' and vice versa where needed and without warnings. The functions that implement the API are the only functions that need to have an explicit cast. The MII code has been converted to use the driver API to avoid unnecessary code churn. Code churn comes from having to work with both converted and unconverted drivers in correlation with having callback functions that take an interface. By converting the MII code first, the callback functions can be defined so that the compiler will perform the typecasts automatically. As soon as all drivers have been converted, the if_t type can be redefined as needed and the API functions can be fix to not need an explicit cast. The immediate benefactors of this change are: 1. Juniper Networks - The network stack implementation in Junos is entirely different from FreeBSD's one and this change allows Juniper to build "stock" NIC drivers that can be used in combination with both the FreeBSD and Junos stacks. 2. FreeBSD - This change opens the door towards changing ifnet and implementing new features and optimizations in the network stack without it requiring a change in the many NIC drivers FreeBSD has. Submitted by: Anuranjan Shukla <anshukla@juniper.net> Reviewed by: glebius@ Obtained from: Juniper Networks, Inc.
2014-06-02 17:54:39 +00:00
{
((struct ifnet *)ifp)->if_ioctl = (void *)ioctl_fn;
}
void
It is actually possible to have if_t a typedef to non-void type, and keep both converted to drvapi and non-converted drivers compilable. o Make if_t typedef to struct ifnet *. o Remove shim functions. Sponsored by: Netflix Sponsored by: Nginx, Inc.
2014-08-31 12:48:13 +00:00
if_setstartfn(if_t ifp, void (*start_fn)(if_t))
Introduce a procedural interface to the ifnet structure. The new interface allows the ifnet structure to be defined as an opaque type in NIC drivers. This then allows the ifnet structure to be changed without a need to change or recompile NIC drivers. Put differently, NIC drivers can be written and compiled once and be used with different network stack implementations, provided of course that those network stack implementations have an API and ABI compatible interface. This commit introduces the 'if_t' type to replace 'struct ifnet *' as the type of a network interface. The 'if_t' type is defined as 'void *' to enable the compiler to perform type conversion to 'struct ifnet *' and vice versa where needed and without warnings. The functions that implement the API are the only functions that need to have an explicit cast. The MII code has been converted to use the driver API to avoid unnecessary code churn. Code churn comes from having to work with both converted and unconverted drivers in correlation with having callback functions that take an interface. By converting the MII code first, the callback functions can be defined so that the compiler will perform the typecasts automatically. As soon as all drivers have been converted, the if_t type can be redefined as needed and the API functions can be fix to not need an explicit cast. The immediate benefactors of this change are: 1. Juniper Networks - The network stack implementation in Junos is entirely different from FreeBSD's one and this change allows Juniper to build "stock" NIC drivers that can be used in combination with both the FreeBSD and Junos stacks. 2. FreeBSD - This change opens the door towards changing ifnet and implementing new features and optimizations in the network stack without it requiring a change in the many NIC drivers FreeBSD has. Submitted by: Anuranjan Shukla <anshukla@juniper.net> Reviewed by: glebius@ Obtained from: Juniper Networks, Inc.
2014-06-02 17:54:39 +00:00
{
((struct ifnet *)ifp)->if_start = (void *)start_fn;
}
void
if_settransmitfn(if_t ifp, if_transmit_fn_t start_fn)
{
((struct ifnet *)ifp)->if_transmit = start_fn;
}
void if_setqflushfn(if_t ifp, if_qflush_fn_t flush_fn)
{
((struct ifnet *)ifp)->if_qflush = flush_fn;
}
Add a function to set if_get_counter method for an ifnet. To be used in the drivers that are already converted to "Juniper drvapi". This can be revisited in future.
2014-09-18 14:38:28 +00:00
void
if_setgetcounterfn(if_t ifp, if_get_counter_t fn)
{
ifp->if_get_counter = fn;
}
Introduce a procedural interface to the ifnet structure. The new interface allows the ifnet structure to be defined as an opaque type in NIC drivers. This then allows the ifnet structure to be changed without a need to change or recompile NIC drivers. Put differently, NIC drivers can be written and compiled once and be used with different network stack implementations, provided of course that those network stack implementations have an API and ABI compatible interface. This commit introduces the 'if_t' type to replace 'struct ifnet *' as the type of a network interface. The 'if_t' type is defined as 'void *' to enable the compiler to perform type conversion to 'struct ifnet *' and vice versa where needed and without warnings. The functions that implement the API are the only functions that need to have an explicit cast. The MII code has been converted to use the driver API to avoid unnecessary code churn. Code churn comes from having to work with both converted and unconverted drivers in correlation with having callback functions that take an interface. By converting the MII code first, the callback functions can be defined so that the compiler will perform the typecasts automatically. As soon as all drivers have been converted, the if_t type can be redefined as needed and the API functions can be fix to not need an explicit cast. The immediate benefactors of this change are: 1. Juniper Networks - The network stack implementation in Junos is entirely different from FreeBSD's one and this change allows Juniper to build "stock" NIC drivers that can be used in combination with both the FreeBSD and Junos stacks. 2. FreeBSD - This change opens the door towards changing ifnet and implementing new features and optimizations in the network stack without it requiring a change in the many NIC drivers FreeBSD has. Submitted by: Anuranjan Shukla <anshukla@juniper.net> Reviewed by: glebius@ Obtained from: Juniper Networks, Inc.
2014-06-02 17:54:39 +00:00
/* Revisit these - These are inline functions originally. */
int
drbr_inuse_drv(if_t ifh, struct buf_ring *br)
{
Fix endless recursion in sys/net/if.c's drbr_inuse_drv(), found by clang 3.7.0. Reviewed by: marcel
2015-06-23 18:48:41 +00:00
return drbr_inuse(ifh, br);
Introduce a procedural interface to the ifnet structure. The new interface allows the ifnet structure to be defined as an opaque type in NIC drivers. This then allows the ifnet structure to be changed without a need to change or recompile NIC drivers. Put differently, NIC drivers can be written and compiled once and be used with different network stack implementations, provided of course that those network stack implementations have an API and ABI compatible interface. This commit introduces the 'if_t' type to replace 'struct ifnet *' as the type of a network interface. The 'if_t' type is defined as 'void *' to enable the compiler to perform type conversion to 'struct ifnet *' and vice versa where needed and without warnings. The functions that implement the API are the only functions that need to have an explicit cast. The MII code has been converted to use the driver API to avoid unnecessary code churn. Code churn comes from having to work with both converted and unconverted drivers in correlation with having callback functions that take an interface. By converting the MII code first, the callback functions can be defined so that the compiler will perform the typecasts automatically. As soon as all drivers have been converted, the if_t type can be redefined as needed and the API functions can be fix to not need an explicit cast. The immediate benefactors of this change are: 1. Juniper Networks - The network stack implementation in Junos is entirely different from FreeBSD's one and this change allows Juniper to build "stock" NIC drivers that can be used in combination with both the FreeBSD and Junos stacks. 2. FreeBSD - This change opens the door towards changing ifnet and implementing new features and optimizations in the network stack without it requiring a change in the many NIC drivers FreeBSD has. Submitted by: Anuranjan Shukla <anshukla@juniper.net> Reviewed by: glebius@ Obtained from: Juniper Networks, Inc.
2014-06-02 17:54:39 +00:00
}
struct mbuf*
drbr_dequeue_drv(if_t ifh, struct buf_ring *br)
{
return drbr_dequeue(ifh, br);
}
int
drbr_needs_enqueue_drv(if_t ifh, struct buf_ring *br)
{
return drbr_needs_enqueue(ifh, br);
}
int
drbr_enqueue_drv(if_t ifh, struct buf_ring *br, struct mbuf *m)
{
return drbr_enqueue(ifh, br, m);
}
Reference in New Issue Copy Permalink