d/freebsd-dev
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
0738dfc386
freebsd-dev/sys/kern/uipc_usrreq.c

2180 lines
55 KiB
C
Raw Normal View History

/* -> /*- for copyright notices, minor format tweaks as necessary
2005-01-06 23:35:40 +00:00
/*-
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
* Copyright (c) 1982, 1986, 1989, 1991, 1993
Canonicalize the UNIX domain socket copyright layout: original holders before more recent holders. MFC after: 3 days
2005-09-23 12:41:06 +00:00
* The Regents of the University of California.
Change unp_mtx to supporting recursion, and do not drop the unp_mtx over sonewconn() in unp_connect(). This avoids a race that occurs due to v_socket being an uncounted reference, as the lock was being released in order to call sonewconn(), which otherwise recurses into the UNIX domain socket code via pru_attach, as well as holding the lock over a sleeping memory allocation in uipc_attach(). Switch to a non-sleeping memory allocation during UNIX domain socket attach. This fix non-ideal in that it requires enabling recursion, but is a much smaller change than moving to using true references for v_socket. The reported panic occurs in unp_connect() following the return of sonewconn(). Update copyright year. Panic reported by: jhb
2007-02-14 12:22:11 +00:00
* Copyright (c) 2004-2007 Robert N. M. Watson
Canonicalize the UNIX domain socket copyright layout: original holders before more recent holders. MFC after: 3 days
2005-09-23 12:41:06 +00:00
* All rights reserved.
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* 4. Neither the name of the University nor the names of its contributors
* may be used to endorse or promote products derived from this software
* without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND
* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE
* FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
* OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
* LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
* OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
* SUCH DAMAGE.
*
Make networking domains drop-ins, through the magic of GNU ld. (Some day, there may even be LKMs.) Also, change the internal name of `unixdomain' to `localdomain' since AF_LOCAL is now the preferred name of this family. Declare netisr correctly and in the right place.
1995-05-11 00:13:26 +00:00
* From: @(#)uipc_usrreq.c 8.3 (Berkeley) 1/4/94
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
*/
Add additional comments to the top of the UNIX domain socket implementation providing some high level pointers regarding the implementation.
2006-07-23 20:06:45 +00:00
/*
* UNIX Domain (Local) Sockets
*
* This is an implementation of UNIX (local) domain sockets. Each socket has
* an associated struct unpcb (UNIX protocol control block). Stream sockets
* may be connected to 0 or 1 other socket. Datagram sockets may be
* connected to 0, 1, or many other sockets. Sockets may be created and
* connected in pairs (socketpair(2)), or bound/connected to using the file
* system name space. For most purposes, only the receive socket buffer is
* used, as sending on one socket delivers directly to the receive socket
Break introductory comment into two paragraphs to separate material on the garbage collection complications from general discussion of UNIX domain sockets. Staticize unp_addsockcred(). Remove XXX comment regarding Giant and v_socket -- v_socket is protected by the global UNIX domain socket lock.
2007-02-20 10:50:02 +00:00
* buffer of a second socket.
*
* The implementation is substantially complicated by the fact that
* "ancillary data", such as file descriptors or credentials, may be passed
* across UNIX domain sockets. The potential for passing UNIX domain sockets
* over other UNIX domain sockets requires the implementation of a simple
* garbage collector to find and tear down cycles of disconnected sockets.
Minor rearrangement of global variables, comments, etc, in UNIX domain sockets.
2007-02-14 15:05:40 +00:00
*
* TODO:
* SEQPACKET, RDM
* rethink name space problems
* need a proper out-of-band
* lock pushdown
Add additional comments to the top of the UNIX domain socket implementation providing some high level pointers regarding the implementation.
2006-07-23 20:06:45 +00:00
*/
Use __FBSDID().
2003-06-11 00:56:59 +00:00
#include <sys/cdefs.h>
__FBSDID("$FreeBSD$");
Introduce support for Mandatory Access Control and extensible kernel access control. Invoke the necessary MAC entry points to maintain labels on sockets. In particular, invoke entry points during socket allocation and destruction, as well as creation by a process or during an accept-scenario (sonewconn). For UNIX domain sockets, also assign a peer label. As the socket code isn't locked down yet, locking interactions are not yet clear. Various protocol stack socket operations (such as peer label assignment for IPv4) will follow. Obtained from: TrustedBSD Project Sponsored by: DARPA, NAI Labs
2002-07-31 03:03:22 +00:00
#include "opt_mac.h"
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
#include <sys/param.h>
Undo part of the tangle of having sys/lock.h and sys/mutex.h included in other "system" header files. Also help the deprecation of lockmgr.h by making it a sub-include of sys/lock.h and removing sys/lockmgr.h form kernel .c files. Sort sys/*.h includes where possible in affected files. OK'ed by: bde (with reservations)
2001-05-01 08:13:21 +00:00
#include <sys/domain.h>
Revert the change of #includes in sys/filedesc.h and sys/socketvar.h. Requested by: bde Since locking sigio_lock is usually followed by calling pgsigio(), move the declaration of sigio_lock and the definitions of SIGIO_*() to sys/signalvar.h. While I am here, sort include files alphabetically, where possible.
2002-04-30 01:54:54 +00:00
#include <sys/fcntl.h>
Fixed duplicate definitions of M_FILE (one static).
1997-11-23 10:43:49 +00:00
#include <sys/malloc.h> /* XXX must be before <sys/file.h> */
Allow for nmbclusters and maxsockets to be increased via sysctl. An eventhandler is used to update all the various zones that depend on these values.
2006-04-21 09:25:40 +00:00
#include <sys/eventhandler.h>
Create a new branch of the kernel MIB, kern.ipc, to store all of the configurables and instrumentation related to inter-process communication mechanisms. Some variables, like mbuf statistics, are instrumented here for the first time. For mbuf statistics: also keep track of m_copym() and m_pullup() failures, and provide for the user's inspection the compiled-in values of MSIZE, MHLEN, MCLBYTES, and MINCLSIZE.
1997-02-24 20:30:58 +00:00
#include <sys/file.h>
Revert the change of #includes in sys/filedesc.h and sys/socketvar.h. Requested by: bde Since locking sigio_lock is usually followed by calling pgsigio(), move the declaration of sigio_lock and the definitions of SIGIO_*() to sys/signalvar.h. While I am here, sort include files alphabetically, where possible.
2002-04-30 01:54:54 +00:00
#include <sys/filedesc.h>
#include <sys/jail.h>
#include <sys/kernel.h>
#include <sys/lock.h>
Create a new branch of the kernel MIB, kern.ipc, to store all of the configurables and instrumentation related to inter-process communication mechanisms. Some variables, like mbuf statistics, are instrumented here for the first time. For mbuf statistics: also keep track of m_copym() and m_pullup() failures, and provide for the user's inspection the compiled-in values of MSIZE, MHLEN, MCLBYTES, and MINCLSIZE.
1997-02-24 20:30:58 +00:00
#include <sys/mbuf.h>
- Lock access to vrele() with VFS_LOCK_GIANT() rather than mtx_lock(&Giant). Sponsored by: Isilon Systems, Inc.
2006-01-30 08:19:01 +00:00
#include <sys/mount.h>
Revert the change of #includes in sys/filedesc.h and sys/socketvar.h. Requested by: bde Since locking sigio_lock is usually followed by calling pgsigio(), move the declaration of sigio_lock and the definitions of SIGIO_*() to sys/signalvar.h. While I am here, sort include files alphabetically, where possible.
2002-04-30 01:54:54 +00:00
#include <sys/mutex.h>
Create a new branch of the kernel MIB, kern.ipc, to store all of the configurables and instrumentation related to inter-process communication mechanisms. Some variables, like mbuf statistics, are instrumented here for the first time. For mbuf statistics: also keep track of m_copym() and m_pullup() failures, and provide for the user's inspection the compiled-in values of MSIZE, MHLEN, MCLBYTES, and MINCLSIZE.
1997-02-24 20:30:58 +00:00
#include <sys/namei.h>
#include <sys/proc.h>
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
#include <sys/protosw.h>
Revert the change of #includes in sys/filedesc.h and sys/socketvar.h. Requested by: bde Since locking sigio_lock is usually followed by calling pgsigio(), move the declaration of sigio_lock and the definitions of SIGIO_*() to sys/signalvar.h. While I am here, sort include files alphabetically, where possible.
2002-04-30 01:54:54 +00:00
#include <sys/resourcevar.h>
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
#include <sys/rwlock.h>
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
#include <sys/socket.h>
#include <sys/socketvar.h>
Revert the change of #includes in sys/filedesc.h and sys/socketvar.h. Requested by: bde Since locking sigio_lock is usually followed by calling pgsigio(), move the declaration of sigio_lock and the definitions of SIGIO_*() to sys/signalvar.h. While I am here, sort include files alphabetically, where possible.
2002-04-30 01:54:54 +00:00
#include <sys/signalvar.h>
Create a new branch of the kernel MIB, kern.ipc, to store all of the configurables and instrumentation related to inter-process communication mechanisms. Some variables, like mbuf statistics, are instrumented here for the first time. For mbuf statistics: also keep track of m_copym() and m_pullup() failures, and provide for the user's inspection the compiled-in values of MSIZE, MHLEN, MCLBYTES, and MINCLSIZE.
1997-02-24 20:30:58 +00:00
#include <sys/stat.h>
Revert the change of #includes in sys/filedesc.h and sys/socketvar.h. Requested by: bde Since locking sigio_lock is usually followed by calling pgsigio(), move the declaration of sigio_lock and the definitions of SIGIO_*() to sys/signalvar.h. While I am here, sort include files alphabetically, where possible.
2002-04-30 01:54:54 +00:00
#include <sys/sx.h>
Create a new branch of the kernel MIB, kern.ipc, to store all of the configurables and instrumentation related to inter-process communication mechanisms. Some variables, like mbuf statistics, are instrumented here for the first time. For mbuf statistics: also keep track of m_copym() and m_pullup() failures, and provide for the user's inspection the compiled-in values of MSIZE, MHLEN, MCLBYTES, and MINCLSIZE.
1997-02-24 20:30:58 +00:00
#include <sys/sysctl.h>
Revert the change of #includes in sys/filedesc.h and sys/socketvar.h. Requested by: bde Since locking sigio_lock is usually followed by calling pgsigio(), move the declaration of sigio_lock and the definitions of SIGIO_*() to sys/signalvar.h. While I am here, sort include files alphabetically, where possible.
2002-04-30 01:54:54 +00:00
#include <sys/systm.h>
Correct a number of serious and closely related bugs in the UNIX domain socket file descriptor garbage collection code, which is intended to detect and clear cycles of orphaned file descriptors that are "in-flight" in a socket when that socket is closed before they are received. The algorithm present was both run at poor times (resulting in recursion and reentrance), and also buggy in the presence of parallelism. In order to fix these problems, make the following changes: - When there are in-flight sockets and a UNIX domain socket is destroyed, asynchronously schedule the garbage collector, rather than running it synchronously in the current context. This avoids lock order issues when the garbage collection code reenters the UNIX domain socket code, avoiding lock order reversals, deadlocks, etc. Run the code asynchronously in a task queue. - In the garbage collector, when skipping file descriptors that have entered a closing state (i.e., have f_count == 0), re-test the FDEFER flag, and decrement unp_defer. As file descriptors can now transition to a closed state, while the garbage collector is running, it is no longer the case that unp_defer will remain an accurate count of deferred sockets in the mark portion of the GC algorithm. Otherwise, the garbage collector will loop waiting waiting for unp_defer to reach zero, which it will never do as it is skipping file descriptors that were marked in an earlier pass, but now closed. - Acquire the UNIX domain socket subsystem lock in unp_discard() when modifying the unp_rights counter, or a read/write race is risked with other threads also manipulating the counter. While here: - Remove #if 0'd code regarding acquiring the socket buffer sleep lock in the garbage collector, this is not required as we are able to use the socket buffer receive lock to protect scanning the receive buffer for in-flight file descriptors on the socket buffer. - Annotate that the description of the garbage collector implementation is increasingly inaccurate and needs to be updated. - Add counters of the number of deferred garbage collections and recycled file descriptors. This will be removed and is here temporarily for debugging purposes. With these changes in place, the unp_passfd regression test now appears to be passed consistently on UP and SMP systems for extended runs, whereas before it hung quickly or panicked, depending on which bug was triggered. Reported by: Philip Kizer <pckizer at nostrum dot com> MFC after: 2 weeks
2005-11-10 16:06:04 +00:00
#include <sys/taskqueue.h>
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
#include <sys/un.h>
Convert socket structures to be type-stable and add a version number. Define a parameter which indicates the maximum number of sockets in a system, and use this to size the zone allocators used for sockets and for certain PCBs. Convert PF_LOCAL PCB structures to be type-stable and add a version number. Define an external format for infomation about socket structures and use it in several places. Define a mechanism to get all PF_LOCAL and PF_INET PCB lists through sysctl(3) without blocking network interrupts for an unreasonable length of time. This probably still has some bugs and/or race conditions, but it seems to work well enough on my machines. It is now possible for `netstat' to get almost all of its information via the sysctl(3) interface rather than reading kmem (changes to follow).
1998-05-15 20:11:40 +00:00
#include <sys/unpcb.h>
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
#include <sys/vnode.h>
Complete break-out of sys/sys/mac.h into sys/security/mac/mac_framework.h begun with a repo-copy of mac.h to mac_framework.h. sys/mac.h now contains the userspace and user<->kernel API and definitions, with all in-kernel interfaces moved to mac_framework.h, which is now included across most of the kernel instead. This change is the first step in a larger cleanup and sweep of MAC Framework interfaces in the kernel, and will not be MFC'd. Obtained from: TrustedBSD Project Sponsored by: SPARTA
2006-10-22 11:52:19 +00:00
#include <security/mac/mac_framework.h>
Remove references to vm_zone.h and switch over to the new uma API.
2002-03-20 04:11:52 +00:00
#include <vm/uma.h>
Convert socket structures to be type-stable and add a version number. Define a parameter which indicates the maximum number of sockets in a system, and use this to size the zone allocators used for sockets and for certain PCBs. Convert PF_LOCAL PCB structures to be type-stable and add a version number. Define an external format for infomation about socket structures and use it in several places. Define a mechanism to get all PF_LOCAL and PF_INET PCB lists through sysctl(3) without blocking network interrupts for an unreasonable length of time. This probably still has some bugs and/or race conditions, but it seems to work well enough on my machines. It is now possible for `netstat' to get almost all of its information via the sysctl(3) interface rather than reading kmem (changes to follow).
1998-05-15 20:11:40 +00:00
Minor rearrangement of global variables, comments, etc, in UNIX domain sockets.
2007-02-14 15:05:40 +00:00
static uma_zone_t unp_zone;
static unp_gen_t unp_gencnt;
static u_int unp_count; /* Count of local sockets. */
static ino_t unp_ino; /* Prototype for fake inode numbers. */
static int unp_rights; /* File descriptors in flight. */
static struct unp_head unp_shead; /* List of local stream sockets. */
static struct unp_head unp_dhead; /* List of local datagram sockets. */
Convert socket structures to be type-stable and add a version number. Define a parameter which indicates the maximum number of sockets in a system, and use this to size the zone allocators used for sockets and for certain PCBs. Convert PF_LOCAL PCB structures to be type-stable and add a version number. Define an external format for infomation about socket structures and use it in several places. Define a mechanism to get all PF_LOCAL and PF_INET PCB lists through sysctl(3) without blocking network interrupts for an unreasonable length of time. This probably still has some bugs and/or race conditions, but it seems to work well enough on my machines. It is now possible for `netstat' to get almost all of its information via the sysctl(3) interface rather than reading kmem (changes to follow).
1998-05-15 20:11:40 +00:00
Minor rearrangement of global variables, comments, etc, in UNIX domain sockets.
2007-02-14 15:05:40 +00:00
static const struct sockaddr sun_noname = { sizeof(sun_noname), AF_LOCAL };
Convert socket structures to be type-stable and add a version number. Define a parameter which indicates the maximum number of sockets in a system, and use this to size the zone allocators used for sockets and for certain PCBs. Convert PF_LOCAL PCB structures to be type-stable and add a version number. Define an external format for infomation about socket structures and use it in several places. Define a mechanism to get all PF_LOCAL and PF_INET PCB lists through sysctl(3) without blocking network interrupts for an unreasonable length of time. This probably still has some bugs and/or race conditions, but it seems to work well enough on my machines. It is now possible for `netstat' to get almost all of its information via the sysctl(3) interface rather than reading kmem (changes to follow).
1998-05-15 20:11:40 +00:00
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
/*
Minor rearrangement of global variables, comments, etc, in UNIX domain sockets.
2007-02-14 15:05:40 +00:00
* Garbage collection of cyclic file descriptor/socket references occurs
* asynchronously in a taskqueue context in order to avoid recursion and
* reentrance in the UNIX domain socket, file descriptor, and socket layer
* code. See unp_gc() for a full description.
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
*/
Minor rearrangement of global variables, comments, etc, in UNIX domain sockets.
2007-02-14 15:05:40 +00:00
static struct task unp_gc_task;
Another mega commit to staticize things.
1995-12-14 09:55:16 +00:00
Move various UNIX socket global variables and sysctls from the middle of the file to the top.
2006-07-23 10:19:04 +00:00
/*
* Both send and receive buffers are allocated PIPSIZ bytes of buffering for
* stream sockets, although the total for sender and receiver is actually
* only PIPSIZ.
*
* Datagram sockets really use the sendspace as the maximum datagram size,
* and don't really want to reserve the sendspace. Their recvspace should be
* large enough for at least one max-size datagram plus address.
*/
#ifndef PIPSIZ
#define PIPSIZ 8192
#endif
static u_long unpst_sendspace = PIPSIZ;
static u_long unpst_recvspace = PIPSIZ;
static u_long unpdg_sendspace = 2*1024; /* really max datagram size */
static u_long unpdg_recvspace = 4*1024;
Move definition of UNIX domain socket protosw and domain entries from uipc_proto.c to uipc_usrreq.c, making localdomain static. Remove uipc_proto.c as it's no longer used. With this change, UNIX domain sockets are entirely encapsulated in uipc_usrreq.c.
2006-08-07 12:02:43 +00:00
SYSCTL_NODE(_net, PF_LOCAL, local, CTLFLAG_RW, 0, "Local domain");
SYSCTL_NODE(_net_local, SOCK_STREAM, stream, CTLFLAG_RW, 0, "SOCK_STREAM");
SYSCTL_NODE(_net_local, SOCK_DGRAM, dgram, CTLFLAG_RW, 0, "SOCK_DGRAM");
Move various UNIX socket global variables and sysctls from the middle of the file to the top.
2006-07-23 10:19:04 +00:00
SYSCTL_ULONG(_net_local_stream, OID_AUTO, sendspace, CTLFLAG_RW,
&unpst_sendspace, 0, "");
SYSCTL_ULONG(_net_local_stream, OID_AUTO, recvspace, CTLFLAG_RW,
&unpst_recvspace, 0, "");
SYSCTL_ULONG(_net_local_dgram, OID_AUTO, maxdgram, CTLFLAG_RW,
&unpdg_sendspace, 0, "");
SYSCTL_ULONG(_net_local_dgram, OID_AUTO, recvspace, CTLFLAG_RW,
&unpdg_recvspace, 0, "");
SYSCTL_INT(_net_local, OID_AUTO, inflight, CTLFLAG_RD, &unp_rights, 0, "");
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
/*-
* Locking and synchronization:
*
* The global UNIX domain socket rwlock (unp_global_rwlock) protects all
* global variables, including the linked lists tracking the set of allocated
* UNIX domain sockets. The global rwlock also serves to prevent deadlock
* when more than one PCB lock is acquired at a time (i.e., during
* connect()). Finally, the global rwlock protects uncounted references from
* vnodes to sockets bound to those vnodes: to safely dereference the
* v_socket pointer, the global rwlock must be held while a full reference is
* acquired.
*
* UNIX domain sockets each have an unpcb hung off of their so_pcb pointer,
* allocated in pru_attach() and freed in pru_detach(). The validity of that
* pointer is an invariant, so no lock is required to dereference the so_pcb
* pointer if a valid socket reference is held by the caller. In practice,
* this is always true during operations performed on a socket. Each unpcb
* has a back-pointer to its socket, unp_socket, which will be stable under
* the same circumstances.
*
* This pointer may only be safely dereferenced as long as a valid reference
* to the unpcb is held. Typically, this reference will be from the socket,
* or from another unpcb when the referring unpcb's lock is held (in order
* that the reference not be invalidated during use). For example, to follow
* unp->unp_conn->unp_socket, you need unlock the lock on unp, not unp_conn,
* as unp_socket remains valid as long as the reference to unp_conn is valid.
*
* Fields of unpcbss are locked using a per-unpcb lock, unp_mtx. Individual
* atomic reads without the lock may be performed "lockless", but more
* complex reads and read-modify-writes require the mutex to be held. No
* lock order is defined between unpcb locks -- multiple unpcb locks may be
* acquired at the same time only when holding the global UNIX domain socket
* rwlock exclusively, which prevents deadlocks.
Annotate the current UNIX domain socket locking strategies, order, strengths, and weaknesses in a comment. Assert a copyright over the changes made as part of the locking work.
2004-08-16 01:52:04 +00:00
*
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
* Blocking with UNIX domain sockets is a tricky issue: unlike most network
* protocols, bind() is a non-atomic operation, and connect() requires
* potential sleeping in the protocol, due to potentially waiting on local or
* distributed file systems. We try to separate "lookup" operations, which
* may sleep, and the IPC operations themselves, which typically can occur
* with relative atomicity as locks can be held over the entire operation.
Annotate the current UNIX domain socket locking strategies, order, strengths, and weaknesses in a comment. Assert a copyright over the changes made as part of the locking work.
2004-08-16 01:52:04 +00:00
*
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
* Another tricky issue is simultaneous multi-threaded or multi-process
* access to a single UNIX domain socket. These are handled by the flags
* UNP_CONNECTING and UNP_BINDING, which prevent concurrent connecting or
* binding, both of which involve dropping UNIX domain socket locks in order
* to perform namei() and other file system operations.
Annotate the current UNIX domain socket locking strategies, order, strengths, and weaknesses in a comment. Assert a copyright over the changes made as part of the locking work.
2004-08-16 01:52:04 +00:00
*/
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
static struct rwlock unp_global_rwlock;
#define UNP_GLOBAL_LOCK_INIT() rw_init(&unp_global_rwlock, \
"unp_global_rwlock")
#define UNP_GLOBAL_LOCK_ASSERT() rw_assert(&unp_global_rwlock, \
RA_LOCKED)
#define UNP_GLOBAL_UNLOCK_ASSERT() rw_assert(&unp_global_rwlock, \
RA_UNLOCKED)
#define UNP_GLOBAL_WLOCK() rw_wlock(&unp_global_rwlock)
#define UNP_GLOBAL_WUNLOCK() rw_wunlock(&unp_global_rwlock)
#define UNP_GLOBAL_WLOCK_ASSERT() rw_assert(&unp_global_rwlock, \
RA_WLOCKED)
#define UNP_GLOBAL_WOWNED() rw_wowned(&unp_global_rwlock)
#define UNP_GLOBAL_RLOCK() rw_rlock(&unp_global_rwlock)
#define UNP_GLOBAL_RUNLOCK() rw_runlock(&unp_global_rwlock)
#define UNP_GLOBAL_RLOCK_ASSERT() rw_assert(&unp_global_rwlock, \
RA_RLOCKED)
#define UNP_PCB_LOCK_INIT(unp) mtx_init(&(unp)->unp_mtx, \
"unp_mtx", "unp_mtx", \
MTX_DUPOK|MTX_DEF|MTX_RECURSE)
#define UNP_PCB_LOCK_DESTROY(unp) mtx_destroy(&(unp)->unp_mtx)
#define UNP_PCB_LOCK(unp) mtx_lock(&(unp)->unp_mtx)
#define UNP_PCB_UNLOCK(unp) mtx_unlock(&(unp)->unp_mtx)
#define UNP_PCB_LOCK_ASSERT(unp) mtx_assert(&(unp)->unp_mtx, MA_OWNED)
static int unp_connect(struct socket *, struct sockaddr *,
Minor rearrangement of global variables, comments, etc, in UNIX domain sockets.
2007-02-14 15:05:40 +00:00
struct thread *);
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
static int unp_connect2(struct socket *so, struct socket *so2, int);
static void unp_disconnect(struct unpcb *unp, struct unpcb *unp2);
static void unp_shutdown(struct unpcb *);
static void unp_drop(struct unpcb *, int);
static void unp_gc(__unused void *, int);
static void unp_scan(struct mbuf *, void (*)(struct file *));
static void unp_mark(struct file *);
static void unp_discard(struct file *);
static void unp_freerights(struct file **, int);
static int unp_internalize(struct mbuf **, struct thread *);
Break introductory comment into two paragraphs to separate material on the garbage collection complications from general discussion of UNIX domain sockets. Staticize unp_addsockcred(). Remove XXX comment regarding Giant and v_socket -- v_socket is protected by the global UNIX domain socket lock.
2007-02-20 10:50:02 +00:00
static struct mbuf *unp_addsockcred(struct thread *, struct mbuf *);
Another mega commit to staticize things.
1995-12-14 09:55:16 +00:00
Move definition of UNIX domain socket protosw and domain entries from uipc_proto.c to uipc_usrreq.c, making localdomain static. Remove uipc_proto.c as it's no longer used. With this change, UNIX domain sockets are entirely encapsulated in uipc_usrreq.c.
2006-08-07 12:02:43 +00:00
/*
* Definitions of protocols supported in the LOCAL domain.
*/
static struct domain localdomain;
static struct protosw localsw[] = {
{
.pr_type = SOCK_STREAM,
.pr_domain = &localdomain,
.pr_flags = PR_CONNREQUIRED|PR_WANTRCVD|PR_RIGHTS,
.pr_ctloutput = &uipc_ctloutput,
.pr_usrreqs = &uipc_usrreqs
},
{
.pr_type = SOCK_DGRAM,
.pr_domain = &localdomain,
.pr_flags = PR_ATOMIC|PR_ADDR|PR_RIGHTS,
.pr_usrreqs = &uipc_usrreqs
},
};
static struct domain localdomain = {
.dom_family = AF_LOCAL,
.dom_name = "local",
.dom_init = unp_init,
.dom_externalize = unp_externalize,
.dom_dispose = unp_dispose,
.dom_protosw = localsw,
.dom_protoswNPROTOSW = &localsw[sizeof(localsw)/sizeof(localsw[0])]
};
DOMAIN_SET(local);
Change protocol switch pru_abort() API so that it returns void rather than an int, as an error here is not meaningful. Modify soabort() to unconditionally free the socket on the return of pru_abort(), and modify most protocols to no longer conditionally free the socket, since the caller will do this. This commit likely leaves parts of netinet and netinet6 in a situation where they may panic or leak memory, as they have not are not fully updated by this commit. This will be corrected shortly in followup commits to these components. MFC after: 3 months
2006-04-01 15:15:05 +00:00
static void
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
uipc_abort(struct socket *so)
{
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
struct unpcb *unp, *unp2;
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
Always acquire the UNIX domain socket subsystem lock (UNP lock) before dereferencing sotounpcb() and checking its value, as so_pcb is protected by protocol locking, not subsystem locking. This prevents races during close() by one thread and use of ths socket in another. unp_bind() now assert the UNP lock, and uipc_bind() now acquires the lock around calls to unp_bind().
2004-08-16 04:41:03 +00:00
unp = sotounpcb(so);
Modify UNIX domain sockets to guarantee, and assume, that so_pcb is always defined for an in-use socket. This allows us to eliminate countless tests of whether so_pcb is non-NULL, eliminating dozens of error cases. For now, retain the call to sotryfree() in the uipc_abort() path, but this will eventually move to soabort(). These new assumptions should be largely correct, and will become more so as the socket/pcb reference model is fixed. Removing the notion that so_pcb can be non-NULL is a critical step towards further fine-graining of the UNIX domain socket locking, as the so_pcb reference no longer needs to be protected using locks, instead it is a property of the socket life cycle.
2006-03-17 13:52:57 +00:00
KASSERT(unp != NULL, ("uipc_abort: unp == NULL"));
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
UNP_GLOBAL_WLOCK();
UNP_PCB_LOCK(unp);
unp2 = unp->unp_conn;
if (unp2 != NULL) {
UNP_PCB_LOCK(unp2);
unp_drop(unp2, ECONNABORTED);
UNP_PCB_UNLOCK(unp2);
}
UNP_PCB_UNLOCK(unp);
UNP_GLOBAL_WUNLOCK();
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
}
static int
Fix all areas of the system (or at least all those in LINT) to avoid storing socket addresses in mbufs. (Socket buffers are the one exception.) A number of kernel APIs needed to get fixed in order to make this happen. Also, fix three protocol families which kept PCBs in mbufs to not malloc them instead. Delete some old compatibility cruft while we're at it, and add some new routines in the in_cksum family.
1997-08-16 19:16:27 +00:00
uipc_accept(struct socket *so, struct sockaddr **nam)
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
{
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
struct unpcb *unp, *unp2;
Introduce a subsystem lock around UNIX domain sockets in order to protect global and allocated variables. This strategy is derived from work originally developed by BSDi for BSD/OS, and applied to FreeBSD by Sam Leffler: - Add unp_mtx, a global mutex which will protect all UNIX domain socket related variables, structures, etc. - Add UNP_LOCK(), UNP_UNLOCK(), UNP_LOCK_ASSERT() macros. - Acquire unp_mtx on entering most UNIX domain socket code, drop/re-acquire around calls into VFS, and release it on return. - Avoid performing sodupsockaddr() while holding the mutex, so in general move to allocating storage before acquiring the mutex to copy the data. - Make a stack copy of the xucred rather than copying out while holding unp_mtx. Copy the peer credential out after releasing the mutex. - Add additional assertions of vnode locks following VOP_CREATE(). A few notes: - Use of an sx lock for the file list mutex may cause problems with regard to unp_mtx when garbage collection passed file descriptors. - The locking in unp_pcblist() for sysctl monitoring is correct subject to the unpcb zone not returning memory for reuse by other subsystems (consistent with similar existing concerns). - Sam's version of this change, as with the BSD/OS version, made use of both a global lock and per-unpcb locks. However, in practice, the global lock covered all accesses, so I have simplified out the unpcb locks in the interest of getting this merged faster (reducing the overhead but not sacrificing granularity in most cases). We will want to explore possibilities for improving lock granularity in this code in the future. Submitted by: sam Sponsored by: FreeBSD Foundatiuon Obtained from: BSD/OS 5 snapshot provided by BSDi
2004-06-10 21:34:38 +00:00
const struct sockaddr *sa;
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
/*
Re-wrap and other minor formatting and punctuation fixes for UNIX domain socket comments.
2006-07-22 17:24:55 +00:00
* Pass back name of connected socket, if it was bound and we are
* still connected (our peer may have closed already!).
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
*/
Modify UNIX domain sockets to guarantee, and assume, that so_pcb is always defined for an in-use socket. This allows us to eliminate countless tests of whether so_pcb is non-NULL, eliminating dozens of error cases. For now, retain the call to sotryfree() in the uipc_abort() path, but this will eventually move to soabort(). These new assumptions should be largely correct, and will become more so as the socket/pcb reference model is fixed. Removing the notion that so_pcb can be non-NULL is a critical step towards further fine-graining of the UNIX domain socket locking, as the so_pcb reference no longer needs to be protected using locks, instead it is a property of the socket life cycle.
2006-03-17 13:52:57 +00:00
unp = sotounpcb(so);
KASSERT(unp != NULL, ("uipc_accept: unp == NULL"));
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
Introduce a subsystem lock around UNIX domain sockets in order to protect global and allocated variables. This strategy is derived from work originally developed by BSDi for BSD/OS, and applied to FreeBSD by Sam Leffler: - Add unp_mtx, a global mutex which will protect all UNIX domain socket related variables, structures, etc. - Add UNP_LOCK(), UNP_UNLOCK(), UNP_LOCK_ASSERT() macros. - Acquire unp_mtx on entering most UNIX domain socket code, drop/re-acquire around calls into VFS, and release it on return. - Avoid performing sodupsockaddr() while holding the mutex, so in general move to allocating storage before acquiring the mutex to copy the data. - Make a stack copy of the xucred rather than copying out while holding unp_mtx. Copy the peer credential out after releasing the mutex. - Add additional assertions of vnode locks following VOP_CREATE(). A few notes: - Use of an sx lock for the file list mutex may cause problems with regard to unp_mtx when garbage collection passed file descriptors. - The locking in unp_pcblist() for sysctl monitoring is correct subject to the unpcb zone not returning memory for reuse by other subsystems (consistent with similar existing concerns). - Sam's version of this change, as with the BSD/OS version, made use of both a global lock and per-unpcb locks. However, in practice, the global lock covered all accesses, so I have simplified out the unpcb locks in the interest of getting this merged faster (reducing the overhead but not sacrificing granularity in most cases). We will want to explore possibilities for improving lock granularity in this code in the future. Submitted by: sam Sponsored by: FreeBSD Foundatiuon Obtained from: BSD/OS 5 snapshot provided by BSDi
2004-06-10 21:34:38 +00:00
*nam = malloc(sizeof(struct sockaddr_un), M_SONAME, M_WAITOK);
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
UNP_GLOBAL_RLOCK();
unp2 = unp->unp_conn;
if (unp2 != NULL && unp2->unp_addr != NULL) {
UNP_PCB_LOCK(unp2);
sa = (struct sockaddr *) unp2->unp_addr;
bcopy(sa, *nam, sa->sa_len);
UNP_PCB_UNLOCK(unp2);
} else {
Introduce a subsystem lock around UNIX domain sockets in order to protect global and allocated variables. This strategy is derived from work originally developed by BSDi for BSD/OS, and applied to FreeBSD by Sam Leffler: - Add unp_mtx, a global mutex which will protect all UNIX domain socket related variables, structures, etc. - Add UNP_LOCK(), UNP_UNLOCK(), UNP_LOCK_ASSERT() macros. - Acquire unp_mtx on entering most UNIX domain socket code, drop/re-acquire around calls into VFS, and release it on return. - Avoid performing sodupsockaddr() while holding the mutex, so in general move to allocating storage before acquiring the mutex to copy the data. - Make a stack copy of the xucred rather than copying out while holding unp_mtx. Copy the peer credential out after releasing the mutex. - Add additional assertions of vnode locks following VOP_CREATE(). A few notes: - Use of an sx lock for the file list mutex may cause problems with regard to unp_mtx when garbage collection passed file descriptors. - The locking in unp_pcblist() for sysctl monitoring is correct subject to the unpcb zone not returning memory for reuse by other subsystems (consistent with similar existing concerns). - Sam's version of this change, as with the BSD/OS version, made use of both a global lock and per-unpcb locks. However, in practice, the global lock covered all accesses, so I have simplified out the unpcb locks in the interest of getting this merged faster (reducing the overhead but not sacrificing granularity in most cases). We will want to explore possibilities for improving lock granularity in this code in the future. Submitted by: sam Sponsored by: FreeBSD Foundatiuon Obtained from: BSD/OS 5 snapshot provided by BSDi
2004-06-10 21:34:38 +00:00
sa = &sun_noname;
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
bcopy(sa, *nam, sa->sa_len);
}
UNP_GLOBAL_RUNLOCK();
Mechanical whitespace cleanup; parenthesize return values; other minor style nits.
2004-01-11 19:48:19 +00:00
return (0);
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
}
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
static int
KSE Milestone 2 Note ALL MODULES MUST BE RECOMPILED make the kernel aware that there are smaller units of scheduling than the process. (but only allow one thread per process at this time). This is functionally equivalent to teh previousl -current except that there is a thread associated with each process. Sorry john! (your next MFC will be a doosie!) Reviewed by: peter@freebsd.org, dillon@freebsd.org X-MFC after: ha ha ha ha
2001-09-12 08:38:13 +00:00
uipc_attach(struct socket *so, int proto, struct thread *td)
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
{
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
u_long sendspace, recvspace;
Since unp_attach() and unp_detach() are now called only from uipc_attach() and uipc_detach(), merge them into their calling functions.
2006-07-23 10:25:28 +00:00
struct unpcb *unp;
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
int error, locked;
Since unp_attach() and unp_detach() are now called only from uipc_attach() and uipc_detach(), merge them into their calling functions.
2006-07-23 10:25:28 +00:00
KASSERT(so->so_pcb == NULL, ("uipc_attach: so_pcb != NULL"));
if (so->so_snd.sb_hiwat == 0 || so->so_rcv.sb_hiwat == 0) {
switch (so->so_type) {
case SOCK_STREAM:
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
sendspace = unpst_sendspace;
recvspace = unpst_recvspace;
Since unp_attach() and unp_detach() are now called only from uipc_attach() and uipc_detach(), merge them into their calling functions.
2006-07-23 10:25:28 +00:00
break;
case SOCK_DGRAM:
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
sendspace = unpdg_sendspace;
recvspace = unpdg_recvspace;
Since unp_attach() and unp_detach() are now called only from uipc_attach() and uipc_detach(), merge them into their calling functions.
2006-07-23 10:25:28 +00:00
break;
default:
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
panic("uipc_attach");
Since unp_attach() and unp_detach() are now called only from uipc_attach() and uipc_detach(), merge them into their calling functions.
2006-07-23 10:25:28 +00:00
}
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
error = soreserve(so, sendspace, recvspace);
Since unp_attach() and unp_detach() are now called only from uipc_attach() and uipc_detach(), merge them into their calling functions.
2006-07-23 10:25:28 +00:00
if (error)
return (error);
}
Change unp_mtx to supporting recursion, and do not drop the unp_mtx over sonewconn() in unp_connect(). This avoids a race that occurs due to v_socket being an uncounted reference, as the lock was being released in order to call sonewconn(), which otherwise recurses into the UNIX domain socket code via pru_attach, as well as holding the lock over a sleeping memory allocation in uipc_attach(). Switch to a non-sleeping memory allocation during UNIX domain socket attach. This fix non-ideal in that it requires enabling recursion, but is a much smaller change than moving to using true references for v_socket. The reported panic occurs in unp_connect() following the return of sonewconn(). Update copyright year. Panic reported by: jhb
2007-02-14 12:22:11 +00:00
unp = uma_zalloc(unp_zone, M_NOWAIT | M_ZERO);
Since unp_attach() and unp_detach() are now called only from uipc_attach() and uipc_detach(), merge them into their calling functions.
2006-07-23 10:25:28 +00:00
if (unp == NULL)
return (ENOBUFS);
LIST_INIT(&unp->unp_refs);
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
UNP_PCB_LOCK_INIT(unp);
Since unp_attach() and unp_detach() are now called only from uipc_attach() and uipc_detach(), merge them into their calling functions.
2006-07-23 10:25:28 +00:00
unp->unp_socket = so;
so->so_pcb = unp;
- Close a race between enumerating UNIX domain socket pcb structures via sysctl and socket teardown by adding a reference count to the UNIX domain pcb object and fixing the sysctl that enumerates unpcbs to grab a reference on each unpcb while it builds the list to copy out to userland. - Close a race between UNIX domain pcb garbage collection (unp_gc()) and file descriptor teardown (fdrop()) by adding a new garbage collection flag FWAIT. unp_gc() sets FWAIT while it walks the message buffers in a UNIX domain socket looking for nested file descriptor references and clears the flag when it is finished. fdrop() checks to see if the flag is set on a file descriptor whose refcount just dropped to 0 and waits for unp_gc() to clear the flag before completely destroying the file descriptor. MFC after: 1 week Reviewed by: rwatson Submitted by: ups Hopefully makes the panics go away: mx1
2007-01-05 19:59:46 +00:00
unp->unp_refcount = 1;
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
locked = 0;
/*
* uipc_attach() may be called indirectly from within the UNIX domain
* socket code via sonewconn() in unp_connect(). Since rwlocks can
* not be recursed, we do the closest thing.
*/
if (!UNP_GLOBAL_WOWNED()) {
UNP_GLOBAL_WLOCK();
locked = 1;
}
Since unp_attach() and unp_detach() are now called only from uipc_attach() and uipc_detach(), merge them into their calling functions.
2006-07-23 10:25:28 +00:00
unp->unp_gencnt = ++unp_gencnt;
unp_count++;
Minor white space tweaks.
2006-08-13 23:16:59 +00:00
LIST_INSERT_HEAD(so->so_type == SOCK_DGRAM ? &unp_dhead : &unp_shead,
unp, unp_link);
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
if (locked)
UNP_GLOBAL_WUNLOCK();
Since unp_attach() and unp_detach() are now called only from uipc_attach() and uipc_detach(), merge them into their calling functions.
2006-07-23 10:25:28 +00:00
return (0);
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
}
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
static int
KSE Milestone 2 Note ALL MODULES MUST BE RECOMPILED make the kernel aware that there are smaller units of scheduling than the process. (but only allow one thread per process at this time). This is functionally equivalent to teh previousl -current except that there is a thread associated with each process. Sorry john! (your next MFC will be a doosie!) Reviewed by: peter@freebsd.org, dillon@freebsd.org X-MFC after: ha ha ha ha
2001-09-12 08:38:13 +00:00
uipc_bind(struct socket *so, struct sockaddr *nam, struct thread *td)
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
{
Merge unp_bind() into uipc_bind(), as it is called only from uipc_bind().
2006-07-23 11:02:12 +00:00
struct sockaddr_un *soun = (struct sockaddr_un *)nam;
struct vattr vattr;
int error, namelen;
struct nameidata nd;
Always acquire the UNIX domain socket subsystem lock (UNP lock) before dereferencing sotounpcb() and checking its value, as so_pcb is protected by protocol locking, not subsystem locking. This prevents races during close() by one thread and use of ths socket in another. unp_bind() now assert the UNP lock, and uipc_bind() now acquires the lock around calls to unp_bind().
2004-08-16 04:41:03 +00:00
struct unpcb *unp;
Merge unp_bind() into uipc_bind(), as it is called only from uipc_bind().
2006-07-23 11:02:12 +00:00
struct vnode *vp;
struct mount *mp;
char *buf;
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
Always acquire the UNIX domain socket subsystem lock (UNP lock) before dereferencing sotounpcb() and checking its value, as so_pcb is protected by protocol locking, not subsystem locking. This prevents races during close() by one thread and use of ths socket in another. unp_bind() now assert the UNP lock, and uipc_bind() now acquires the lock around calls to unp_bind().
2004-08-16 04:41:03 +00:00
unp = sotounpcb(so);
Modify UNIX domain sockets to guarantee, and assume, that so_pcb is always defined for an in-use socket. This allows us to eliminate countless tests of whether so_pcb is non-NULL, eliminating dozens of error cases. For now, retain the call to sotryfree() in the uipc_abort() path, but this will eventually move to soabort(). These new assumptions should be largely correct, and will become more so as the socket/pcb reference model is fixed. Removing the notion that so_pcb can be non-NULL is a critical step towards further fine-graining of the UNIX domain socket locking, as the so_pcb reference no longer needs to be protected using locks, instead it is a property of the socket life cycle.
2006-03-17 13:52:57 +00:00
KASSERT(unp != NULL, ("uipc_bind: unp == NULL"));
Add two new unpcb flags, UNP_BINDING and UNP_CONNECTING, which will be used to mark UNIX domain sockets as being in the process of binding or connecting. Use these to prevent simultaneous bind or connect operations by multiple threads or processes on the same socket at the same time, which closes race conditions present in the UNIX domain socket implementation since inception.
2006-07-23 12:01:14 +00:00
namelen = soun->sun_len - offsetof(struct sockaddr_un, sun_path);
if (namelen <= 0)
return (EINVAL);
Merge unp_bind() into uipc_bind(), as it is called only from uipc_bind().
2006-07-23 11:02:12 +00:00
/*
Add two new unpcb flags, UNP_BINDING and UNP_CONNECTING, which will be used to mark UNIX domain sockets as being in the process of binding or connecting. Use these to prevent simultaneous bind or connect operations by multiple threads or processes on the same socket at the same time, which closes race conditions present in the UNIX domain socket implementation since inception.
2006-07-23 12:01:14 +00:00
* We don't allow simultaneous bind() calls on a single UNIX domain
* socket, so flag in-progress operations, and return an error if an
* operation is already in progress.
*
* Historically, we have not allowed a socket to be rebound, so this
* also returns an error. Not allowing re-binding certainly
* simplifies the implementation and avoids a great many possible
* failure modes.
Merge unp_bind() into uipc_bind(), as it is called only from uipc_bind().
2006-07-23 11:02:12 +00:00
*/
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
UNP_PCB_LOCK(unp);
Merge unp_bind() into uipc_bind(), as it is called only from uipc_bind().
2006-07-23 11:02:12 +00:00
if (unp->unp_vnode != NULL) {
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
UNP_PCB_UNLOCK(unp);
Merge unp_bind() into uipc_bind(), as it is called only from uipc_bind().
2006-07-23 11:02:12 +00:00
return (EINVAL);
}
Add two new unpcb flags, UNP_BINDING and UNP_CONNECTING, which will be used to mark UNIX domain sockets as being in the process of binding or connecting. Use these to prevent simultaneous bind or connect operations by multiple threads or processes on the same socket at the same time, which closes race conditions present in the UNIX domain socket implementation since inception.
2006-07-23 12:01:14 +00:00
if (unp->unp_flags & UNP_BINDING) {
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
UNP_PCB_UNLOCK(unp);
Add two new unpcb flags, UNP_BINDING and UNP_CONNECTING, which will be used to mark UNIX domain sockets as being in the process of binding or connecting. Use these to prevent simultaneous bind or connect operations by multiple threads or processes on the same socket at the same time, which closes race conditions present in the UNIX domain socket implementation since inception.
2006-07-23 12:01:14 +00:00
return (EALREADY);
Merge unp_bind() into uipc_bind(), as it is called only from uipc_bind().
2006-07-23 11:02:12 +00:00
}
Add two new unpcb flags, UNP_BINDING and UNP_CONNECTING, which will be used to mark UNIX domain sockets as being in the process of binding or connecting. Use these to prevent simultaneous bind or connect operations by multiple threads or processes on the same socket at the same time, which closes race conditions present in the UNIX domain socket implementation since inception.
2006-07-23 12:01:14 +00:00
unp->unp_flags |= UNP_BINDING;
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
UNP_PCB_UNLOCK(unp);
Merge unp_bind() into uipc_bind(), as it is called only from uipc_bind().
2006-07-23 11:02:12 +00:00
buf = malloc(namelen + 1, M_TEMP, M_WAITOK);
strlcpy(buf, soun->sun_path, namelen + 1);
mtx_lock(&Giant);
restart:
mtx_assert(&Giant, MA_OWNED);
NDINIT(&nd, CREATE, NOFOLLOW | LOCKPARENT | SAVENAME, UIO_SYSSPACE,
buf, td);
/* SHOULD BE ABLE TO ADOPT EXISTING AND wakeup() ALA FIFO's */
error = namei(&nd);
if (error)
Add two new unpcb flags, UNP_BINDING and UNP_CONNECTING, which will be used to mark UNIX domain sockets as being in the process of binding or connecting. Use these to prevent simultaneous bind or connect operations by multiple threads or processes on the same socket at the same time, which closes race conditions present in the UNIX domain socket implementation since inception.
2006-07-23 12:01:14 +00:00
goto error;
Merge unp_bind() into uipc_bind(), as it is called only from uipc_bind().
2006-07-23 11:02:12 +00:00
vp = nd.ni_vp;
if (vp != NULL || vn_start_write(nd.ni_dvp, &mp, V_NOWAIT) != 0) {
NDFREE(&nd, NDF_ONLY_PNBUF);
if (nd.ni_dvp == vp)
vrele(nd.ni_dvp);
else
vput(nd.ni_dvp);
if (vp != NULL) {
vrele(vp);
error = EADDRINUSE;
Add two new unpcb flags, UNP_BINDING and UNP_CONNECTING, which will be used to mark UNIX domain sockets as being in the process of binding or connecting. Use these to prevent simultaneous bind or connect operations by multiple threads or processes on the same socket at the same time, which closes race conditions present in the UNIX domain socket implementation since inception.
2006-07-23 12:01:14 +00:00
goto error;
Merge unp_bind() into uipc_bind(), as it is called only from uipc_bind().
2006-07-23 11:02:12 +00:00
}
error = vn_start_write(NULL, &mp, V_XSLEEP | PCATCH);
if (error)
Add two new unpcb flags, UNP_BINDING and UNP_CONNECTING, which will be used to mark UNIX domain sockets as being in the process of binding or connecting. Use these to prevent simultaneous bind or connect operations by multiple threads or processes on the same socket at the same time, which closes race conditions present in the UNIX domain socket implementation since inception.
2006-07-23 12:01:14 +00:00
goto error;
Merge unp_bind() into uipc_bind(), as it is called only from uipc_bind().
2006-07-23 11:02:12 +00:00
goto restart;
}
VATTR_NULL(&vattr);
vattr.va_type = VSOCK;
vattr.va_mode = (ACCESSPERMS & ~td->td_proc->p_fd->fd_cmask);
#ifdef MAC
error = mac_check_vnode_create(td->td_ucred, nd.ni_dvp, &nd.ni_cnd,
&vattr);
#endif
if (error == 0) {
VOP_LEASE(nd.ni_dvp, td, td->td_ucred, LEASE_WRITE);
error = VOP_CREATE(nd.ni_dvp, &nd.ni_vp, &nd.ni_cnd, &vattr);
}
NDFREE(&nd, NDF_ONLY_PNBUF);
vput(nd.ni_dvp);
if (error) {
vn_finished_write(mp);
Add two new unpcb flags, UNP_BINDING and UNP_CONNECTING, which will be used to mark UNIX domain sockets as being in the process of binding or connecting. Use these to prevent simultaneous bind or connect operations by multiple threads or processes on the same socket at the same time, which closes race conditions present in the UNIX domain socket implementation since inception.
2006-07-23 12:01:14 +00:00
goto error;
Merge unp_bind() into uipc_bind(), as it is called only from uipc_bind().
2006-07-23 11:02:12 +00:00
}
vp = nd.ni_vp;
Add two new unpcb flags, UNP_BINDING and UNP_CONNECTING, which will be used to mark UNIX domain sockets as being in the process of binding or connecting. Use these to prevent simultaneous bind or connect operations by multiple threads or processes on the same socket at the same time, which closes race conditions present in the UNIX domain socket implementation since inception.
2006-07-23 12:01:14 +00:00
ASSERT_VOP_LOCKED(vp, "uipc_bind");
Merge unp_bind() into uipc_bind(), as it is called only from uipc_bind().
2006-07-23 11:02:12 +00:00
soun = (struct sockaddr_un *)sodupsockaddr(nam, M_WAITOK);
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
UNP_GLOBAL_WLOCK();
UNP_PCB_LOCK(unp);
Merge unp_bind() into uipc_bind(), as it is called only from uipc_bind().
2006-07-23 11:02:12 +00:00
vp->v_socket = unp->unp_socket;
unp->unp_vnode = vp;
unp->unp_addr = soun;
Add two new unpcb flags, UNP_BINDING and UNP_CONNECTING, which will be used to mark UNIX domain sockets as being in the process of binding or connecting. Use these to prevent simultaneous bind or connect operations by multiple threads or processes on the same socket at the same time, which closes race conditions present in the UNIX domain socket implementation since inception.
2006-07-23 12:01:14 +00:00
unp->unp_flags &= ~UNP_BINDING;
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
UNP_PCB_UNLOCK(unp);
UNP_GLOBAL_WUNLOCK();
Merge unp_bind() into uipc_bind(), as it is called only from uipc_bind().
2006-07-23 11:02:12 +00:00
VOP_UNLOCK(vp, 0, td);
vn_finished_write(mp);
Add two new unpcb flags, UNP_BINDING and UNP_CONNECTING, which will be used to mark UNIX domain sockets as being in the process of binding or connecting. Use these to prevent simultaneous bind or connect operations by multiple threads or processes on the same socket at the same time, which closes race conditions present in the UNIX domain socket implementation since inception.
2006-07-23 12:01:14 +00:00
mtx_unlock(&Giant);
free(buf, M_TEMP);
return (0);
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
Add two new unpcb flags, UNP_BINDING and UNP_CONNECTING, which will be used to mark UNIX domain sockets as being in the process of binding or connecting. Use these to prevent simultaneous bind or connect operations by multiple threads or processes on the same socket at the same time, which closes race conditions present in the UNIX domain socket implementation since inception.
2006-07-23 12:01:14 +00:00
error:
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
UNP_PCB_LOCK(unp);
Add two new unpcb flags, UNP_BINDING and UNP_CONNECTING, which will be used to mark UNIX domain sockets as being in the process of binding or connecting. Use these to prevent simultaneous bind or connect operations by multiple threads or processes on the same socket at the same time, which closes race conditions present in the UNIX domain socket implementation since inception.
2006-07-23 12:01:14 +00:00
unp->unp_flags &= ~UNP_BINDING;
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
UNP_PCB_UNLOCK(unp);
Merge unp_bind() into uipc_bind(), as it is called only from uipc_bind().
2006-07-23 11:02:12 +00:00
mtx_unlock(&Giant);
free(buf, M_TEMP);
Always acquire the UNIX domain socket subsystem lock (UNP lock) before dereferencing sotounpcb() and checking its value, as so_pcb is protected by protocol locking, not subsystem locking. This prevents races during close() by one thread and use of ths socket in another. unp_bind() now assert the UNP lock, and uipc_bind() now acquires the lock around calls to unp_bind().
2004-08-16 04:41:03 +00:00
return (error);
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
}
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
static int
KSE Milestone 2 Note ALL MODULES MUST BE RECOMPILED make the kernel aware that there are smaller units of scheduling than the process. (but only allow one thread per process at this time). This is functionally equivalent to teh previousl -current except that there is a thread associated with each process. Sorry john! (your next MFC will be a doosie!) Reviewed by: peter@freebsd.org, dillon@freebsd.org X-MFC after: ha ha ha ha
2001-09-12 08:38:13 +00:00
uipc_connect(struct socket *so, struct sockaddr *nam, struct thread *td)
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
{
Introduce a subsystem lock around UNIX domain sockets in order to protect global and allocated variables. This strategy is derived from work originally developed by BSDi for BSD/OS, and applied to FreeBSD by Sam Leffler: - Add unp_mtx, a global mutex which will protect all UNIX domain socket related variables, structures, etc. - Add UNP_LOCK(), UNP_UNLOCK(), UNP_LOCK_ASSERT() macros. - Acquire unp_mtx on entering most UNIX domain socket code, drop/re-acquire around calls into VFS, and release it on return. - Avoid performing sodupsockaddr() while holding the mutex, so in general move to allocating storage before acquiring the mutex to copy the data. - Make a stack copy of the xucred rather than copying out while holding unp_mtx. Copy the peer credential out after releasing the mutex. - Add additional assertions of vnode locks following VOP_CREATE(). A few notes: - Use of an sx lock for the file list mutex may cause problems with regard to unp_mtx when garbage collection passed file descriptors. - The locking in unp_pcblist() for sysctl monitoring is correct subject to the unpcb zone not returning memory for reuse by other subsystems (consistent with similar existing concerns). - Sam's version of this change, as with the BSD/OS version, made use of both a global lock and per-unpcb locks. However, in practice, the global lock covered all accesses, so I have simplified out the unpcb locks in the interest of getting this merged faster (reducing the overhead but not sacrificing granularity in most cases). We will want to explore possibilities for improving lock granularity in this code in the future. Submitted by: sam Sponsored by: FreeBSD Foundatiuon Obtained from: BSD/OS 5 snapshot provided by BSDi
2004-06-10 21:34:38 +00:00
int error;
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
In uipc_connect(), assert that the passed thread is curthread, and pass td into unp_connect() instead of reading curthread.
2004-07-25 23:30:43 +00:00
KASSERT(td == curthread, ("uipc_connect: td != curthread"));
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
UNP_GLOBAL_WLOCK();
In uipc_connect(), assert that the passed thread is curthread, and pass td into unp_connect() instead of reading curthread.
2004-07-25 23:30:43 +00:00
error = unp_connect(so, nam, td);
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
UNP_GLOBAL_WUNLOCK();
Introduce a subsystem lock around UNIX domain sockets in order to protect global and allocated variables. This strategy is derived from work originally developed by BSDi for BSD/OS, and applied to FreeBSD by Sam Leffler: - Add unp_mtx, a global mutex which will protect all UNIX domain socket related variables, structures, etc. - Add UNP_LOCK(), UNP_UNLOCK(), UNP_LOCK_ASSERT() macros. - Acquire unp_mtx on entering most UNIX domain socket code, drop/re-acquire around calls into VFS, and release it on return. - Avoid performing sodupsockaddr() while holding the mutex, so in general move to allocating storage before acquiring the mutex to copy the data. - Make a stack copy of the xucred rather than copying out while holding unp_mtx. Copy the peer credential out after releasing the mutex. - Add additional assertions of vnode locks following VOP_CREATE(). A few notes: - Use of an sx lock for the file list mutex may cause problems with regard to unp_mtx when garbage collection passed file descriptors. - The locking in unp_pcblist() for sysctl monitoring is correct subject to the unpcb zone not returning memory for reuse by other subsystems (consistent with similar existing concerns). - Sam's version of this change, as with the BSD/OS version, made use of both a global lock and per-unpcb locks. However, in practice, the global lock covered all accesses, so I have simplified out the unpcb locks in the interest of getting this merged faster (reducing the overhead but not sacrificing granularity in most cases). We will want to explore possibilities for improving lock granularity in this code in the future. Submitted by: sam Sponsored by: FreeBSD Foundatiuon Obtained from: BSD/OS 5 snapshot provided by BSDi
2004-06-10 21:34:38 +00:00
return (error);
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
}
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
Change semantics of socket close and detach. Add a new protocol switch function, pru_close, to notify protocols that the file descriptor or other consumer of a socket is closing the socket. pru_abort is now a notification of close also, and no longer detaches. pru_detach is no longer used to notify of close, and will be called during socket tear-down by sofree() when all references to a socket evaporate after an earlier call to abort or close the socket. This means detach is now an unconditional teardown of a socket, whereas previously sockets could persist after detach of the protocol retained a reference. This faciliates sharing mutexes between layers of the network stack as the mutex is required during the checking and removal of references at the head of sofree(). With this change, pru_detach can now assume that the mutex will no longer be required by the socket layer after completion, whereas before this was not necessarily true. Reviewed by: gnn
2006-07-21 17:11:15 +00:00
static void
uipc_close(struct socket *so)
{
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
struct unpcb *unp, *unp2;
Change semantics of socket close and detach. Add a new protocol switch function, pru_close, to notify protocols that the file descriptor or other consumer of a socket is closing the socket. pru_abort is now a notification of close also, and no longer detaches. pru_detach is no longer used to notify of close, and will be called during socket tear-down by sofree() when all references to a socket evaporate after an earlier call to abort or close the socket. This means detach is now an unconditional teardown of a socket, whereas previously sockets could persist after detach of the protocol retained a reference. This faciliates sharing mutexes between layers of the network stack as the mutex is required during the checking and removal of references at the head of sofree(). With this change, pru_detach can now assume that the mutex will no longer be required by the socket layer after completion, whereas before this was not necessarily true. Reviewed by: gnn
2006-07-21 17:11:15 +00:00
unp = sotounpcb(so);
KASSERT(unp != NULL, ("uipc_close: unp == NULL"));
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
UNP_GLOBAL_WLOCK();
UNP_PCB_LOCK(unp);
unp2 = unp->unp_conn;
if (unp2 != NULL) {
UNP_PCB_LOCK(unp2);
unp_disconnect(unp, unp2);
UNP_PCB_UNLOCK(unp2);
}
UNP_PCB_UNLOCK(unp);
UNP_GLOBAL_WUNLOCK();
Change semantics of socket close and detach. Add a new protocol switch function, pru_close, to notify protocols that the file descriptor or other consumer of a socket is closing the socket. pru_abort is now a notification of close also, and no longer detaches. pru_detach is no longer used to notify of close, and will be called during socket tear-down by sofree() when all references to a socket evaporate after an earlier call to abort or close the socket. This means detach is now an unconditional teardown of a socket, whereas previously sockets could persist after detach of the protocol retained a reference. This faciliates sharing mutexes between layers of the network stack as the mutex is required during the checking and removal of references at the head of sofree(). With this change, pru_detach can now assume that the mutex will no longer be required by the socket layer after completion, whereas before this was not necessarily true. Reviewed by: gnn
2006-07-21 17:11:15 +00:00
}
Export uipc_connect2() from uipc_usrreq.c instead of unp_connect2(), and consume that interface in portalfs and fifofs instead. In the new world order, unp_connect2() assumes that the unpcb mutex is held, whereas uipc_connect2() validates that the passed sockets are UNIX domain sockets, then grabs the mutex. NB: the portalfs and fifofs code gets down and dirty with UNIX domain sockets. Maybe this is a bad thing.
2004-03-31 01:41:30 +00:00
int
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
uipc_connect2(struct socket *so1, struct socket *so2)
{
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
struct unpcb *unp, *unp2;
Introduce a subsystem lock around UNIX domain sockets in order to protect global and allocated variables. This strategy is derived from work originally developed by BSDi for BSD/OS, and applied to FreeBSD by Sam Leffler: - Add unp_mtx, a global mutex which will protect all UNIX domain socket related variables, structures, etc. - Add UNP_LOCK(), UNP_UNLOCK(), UNP_LOCK_ASSERT() macros. - Acquire unp_mtx on entering most UNIX domain socket code, drop/re-acquire around calls into VFS, and release it on return. - Avoid performing sodupsockaddr() while holding the mutex, so in general move to allocating storage before acquiring the mutex to copy the data. - Make a stack copy of the xucred rather than copying out while holding unp_mtx. Copy the peer credential out after releasing the mutex. - Add additional assertions of vnode locks following VOP_CREATE(). A few notes: - Use of an sx lock for the file list mutex may cause problems with regard to unp_mtx when garbage collection passed file descriptors. - The locking in unp_pcblist() for sysctl monitoring is correct subject to the unpcb zone not returning memory for reuse by other subsystems (consistent with similar existing concerns). - Sam's version of this change, as with the BSD/OS version, made use of both a global lock and per-unpcb locks. However, in practice, the global lock covered all accesses, so I have simplified out the unpcb locks in the interest of getting this merged faster (reducing the overhead but not sacrificing granularity in most cases). We will want to explore possibilities for improving lock granularity in this code in the future. Submitted by: sam Sponsored by: FreeBSD Foundatiuon Obtained from: BSD/OS 5 snapshot provided by BSDi
2004-06-10 21:34:38 +00:00
int error;
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
UNP_GLOBAL_WLOCK();
unp = so1->so_pcb;
Modify UNIX domain sockets to guarantee, and assume, that so_pcb is always defined for an in-use socket. This allows us to eliminate countless tests of whether so_pcb is non-NULL, eliminating dozens of error cases. For now, retain the call to sotryfree() in the uipc_abort() path, but this will eventually move to soabort(). These new assumptions should be largely correct, and will become more so as the socket/pcb reference model is fixed. Removing the notion that so_pcb can be non-NULL is a critical step towards further fine-graining of the UNIX domain socket locking, as the so_pcb reference no longer needs to be protected using locks, instead it is a property of the socket life cycle.
2006-03-17 13:52:57 +00:00
KASSERT(unp != NULL, ("uipc_connect2: unp == NULL"));
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
UNP_PCB_LOCK(unp);
unp2 = so2->so_pcb;
KASSERT(unp2 != NULL, ("uipc_connect2: unp2 == NULL"));
UNP_PCB_LOCK(unp2);
Implement unix(4) socket options LOCAL_CREDS and LOCAL_CONNWAIT. - Add unp_addsockcred() (for LOCAL_CREDS). - Add an argument to unp_connect2() to differentiate between PRU_CONNECT and PRU_CONNECT2. (for LOCAL_CONNWAIT) Obtained from: NetBSD (with some changes)
2005-04-13 00:01:46 +00:00
error = unp_connect2(so1, so2, PRU_CONNECT2);
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
UNP_PCB_UNLOCK(unp2);
UNP_PCB_UNLOCK(unp);
UNP_GLOBAL_WUNLOCK();
Introduce a subsystem lock around UNIX domain sockets in order to protect global and allocated variables. This strategy is derived from work originally developed by BSDi for BSD/OS, and applied to FreeBSD by Sam Leffler: - Add unp_mtx, a global mutex which will protect all UNIX domain socket related variables, structures, etc. - Add UNP_LOCK(), UNP_UNLOCK(), UNP_LOCK_ASSERT() macros. - Acquire unp_mtx on entering most UNIX domain socket code, drop/re-acquire around calls into VFS, and release it on return. - Avoid performing sodupsockaddr() while holding the mutex, so in general move to allocating storage before acquiring the mutex to copy the data. - Make a stack copy of the xucred rather than copying out while holding unp_mtx. Copy the peer credential out after releasing the mutex. - Add additional assertions of vnode locks following VOP_CREATE(). A few notes: - Use of an sx lock for the file list mutex may cause problems with regard to unp_mtx when garbage collection passed file descriptors. - The locking in unp_pcblist() for sysctl monitoring is correct subject to the unpcb zone not returning memory for reuse by other subsystems (consistent with similar existing concerns). - Sam's version of this change, as with the BSD/OS version, made use of both a global lock and per-unpcb locks. However, in practice, the global lock covered all accesses, so I have simplified out the unpcb locks in the interest of getting this merged faster (reducing the overhead but not sacrificing granularity in most cases). We will want to explore possibilities for improving lock granularity in this code in the future. Submitted by: sam Sponsored by: FreeBSD Foundatiuon Obtained from: BSD/OS 5 snapshot provided by BSDi
2004-06-10 21:34:38 +00:00
return (error);
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
}
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
/* control is EOPNOTSUPP */
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
Chance protocol switch method pru_detach() so that it returns void rather than an error. Detaches do not "fail", they other occur or the protocol flags SS_PROTOREF to take ownership of the socket. soclose() no longer looks at so_pcb to see if it's NULL, relying entirely on the protocol to decide whether it's time to free the socket or not using SS_PROTOREF. so_pcb is now entirely owned and managed by the protocol code. Likewise, no longer test so_pcb in other socket functions, such as soreceive(), which have no business digging into protocol internals. Protocol detach routines no longer try to free the socket on detach, this is performed in the socket code if the protocol permits it. In rts_detach(), no longer test for rp != NULL in detach, and likewise in other protocols that don't permit a NULL so_pcb, reduce the incidence of testing for it during detach. netinet and netinet6 are not fully updated to this change, which will be in an upcoming commit. In their current state they may leak memory or panic. MFC after: 3 months
2006-04-01 15:42:02 +00:00
static void
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
uipc_detach(struct socket *so)
{
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
struct unpcb *unp, *unp2;
- Close a race between enumerating UNIX domain socket pcb structures via sysctl and socket teardown by adding a reference count to the UNIX domain pcb object and fixing the sysctl that enumerates unpcbs to grab a reference on each unpcb while it builds the list to copy out to userland. - Close a race between UNIX domain pcb garbage collection (unp_gc()) and file descriptor teardown (fdrop()) by adding a new garbage collection flag FWAIT. unp_gc() sets FWAIT while it walks the message buffers in a UNIX domain socket looking for nested file descriptor references and clears the flag when it is finished. fdrop() checks to see if the flag is set on a file descriptor whose refcount just dropped to 0 and waits for unp_gc() to clear the flag before completely destroying the file descriptor. MFC after: 1 week Reviewed by: rwatson Submitted by: ups Hopefully makes the panics go away: mx1
2007-01-05 19:59:46 +00:00
struct sockaddr_un *saved_unp_addr;
Since unp_attach() and unp_detach() are now called only from uipc_attach() and uipc_detach(), merge them into their calling functions.
2006-07-23 10:25:28 +00:00
struct vnode *vp;
- Close a race between enumerating UNIX domain socket pcb structures via sysctl and socket teardown by adding a reference count to the UNIX domain pcb object and fixing the sysctl that enumerates unpcbs to grab a reference on each unpcb while it builds the list to copy out to userland. - Close a race between UNIX domain pcb garbage collection (unp_gc()) and file descriptor teardown (fdrop()) by adding a new garbage collection flag FWAIT. unp_gc() sets FWAIT while it walks the message buffers in a UNIX domain socket looking for nested file descriptor references and clears the flag when it is finished. fdrop() checks to see if the flag is set on a file descriptor whose refcount just dropped to 0 and waits for unp_gc() to clear the flag before completely destroying the file descriptor. MFC after: 1 week Reviewed by: rwatson Submitted by: ups Hopefully makes the panics go away: mx1
2007-01-05 19:59:46 +00:00
int freeunp, local_unp_rights;
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
Always acquire the UNIX domain socket subsystem lock (UNP lock) before dereferencing sotounpcb() and checking its value, as so_pcb is protected by protocol locking, not subsystem locking. This prevents races during close() by one thread and use of ths socket in another. unp_bind() now assert the UNP lock, and uipc_bind() now acquires the lock around calls to unp_bind().
2004-08-16 04:41:03 +00:00
unp = sotounpcb(so);
Modify UNIX domain sockets to guarantee, and assume, that so_pcb is always defined for an in-use socket. This allows us to eliminate countless tests of whether so_pcb is non-NULL, eliminating dozens of error cases. For now, retain the call to sotryfree() in the uipc_abort() path, but this will eventually move to soabort(). These new assumptions should be largely correct, and will become more so as the socket/pcb reference model is fixed. Removing the notion that so_pcb can be non-NULL is a critical step towards further fine-graining of the UNIX domain socket locking, as the so_pcb reference no longer needs to be protected using locks, instead it is a property of the socket life cycle.
2006-03-17 13:52:57 +00:00
KASSERT(unp != NULL, ("uipc_detach: unp == NULL"));
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
UNP_GLOBAL_WLOCK();
UNP_PCB_LOCK(unp);
Since unp_attach() and unp_detach() are now called only from uipc_attach() and uipc_detach(), merge them into their calling functions.
2006-07-23 10:25:28 +00:00
LIST_REMOVE(unp, unp_link);
unp->unp_gencnt = ++unp_gencnt;
--unp_count;
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
/*
* XXXRW: Should assert vp->v_socket == so.
*/
Since unp_attach() and unp_detach() are now called only from uipc_attach() and uipc_detach(), merge them into their calling functions.
2006-07-23 10:25:28 +00:00
if ((vp = unp->unp_vnode) != NULL) {
unp->unp_vnode->v_socket = NULL;
unp->unp_vnode = NULL;
}
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
unp2 = unp->unp_conn;
if (unp2 != NULL) {
UNP_PCB_LOCK(unp2);
unp_disconnect(unp, unp2);
UNP_PCB_UNLOCK(unp2);
}
/*
* We hold the global lock, so it's OK to acquire multiple pcb locks
* at a time.
*/
Since unp_attach() and unp_detach() are now called only from uipc_attach() and uipc_detach(), merge them into their calling functions.
2006-07-23 10:25:28 +00:00
while (!LIST_EMPTY(&unp->unp_refs)) {
struct unpcb *ref = LIST_FIRST(&unp->unp_refs);
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
UNP_PCB_LOCK(ref);
Since unp_attach() and unp_detach() are now called only from uipc_attach() and uipc_detach(), merge them into their calling functions.
2006-07-23 10:25:28 +00:00
unp_drop(ref, ECONNRESET);
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
UNP_PCB_UNLOCK(ref);
Since unp_attach() and unp_detach() are now called only from uipc_attach() and uipc_detach(), merge them into their calling functions.
2006-07-23 10:25:28 +00:00
}
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
UNP_GLOBAL_WUNLOCK();
Since unp_attach() and unp_detach() are now called only from uipc_attach() and uipc_detach(), merge them into their calling functions.
2006-07-23 10:25:28 +00:00
unp->unp_socket->so_pcb = NULL;
local_unp_rights = unp_rights;
- Close a race between enumerating UNIX domain socket pcb structures via sysctl and socket teardown by adding a reference count to the UNIX domain pcb object and fixing the sysctl that enumerates unpcbs to grab a reference on each unpcb while it builds the list to copy out to userland. - Close a race between UNIX domain pcb garbage collection (unp_gc()) and file descriptor teardown (fdrop()) by adding a new garbage collection flag FWAIT. unp_gc() sets FWAIT while it walks the message buffers in a UNIX domain socket looking for nested file descriptor references and clears the flag when it is finished. fdrop() checks to see if the flag is set on a file descriptor whose refcount just dropped to 0 and waits for unp_gc() to clear the flag before completely destroying the file descriptor. MFC after: 1 week Reviewed by: rwatson Submitted by: ups Hopefully makes the panics go away: mx1
2007-01-05 19:59:46 +00:00
saved_unp_addr = unp->unp_addr;
unp->unp_addr = NULL;
unp->unp_refcount--;
freeunp = (unp->unp_refcount == 0);
if (saved_unp_addr != NULL)
FREE(saved_unp_addr, M_SONAME);
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
if (freeunp) {
UNP_PCB_LOCK_DESTROY(unp);
- Close a race between enumerating UNIX domain socket pcb structures via sysctl and socket teardown by adding a reference count to the UNIX domain pcb object and fixing the sysctl that enumerates unpcbs to grab a reference on each unpcb while it builds the list to copy out to userland. - Close a race between UNIX domain pcb garbage collection (unp_gc()) and file descriptor teardown (fdrop()) by adding a new garbage collection flag FWAIT. unp_gc() sets FWAIT while it walks the message buffers in a UNIX domain socket looking for nested file descriptor references and clears the flag when it is finished. fdrop() checks to see if the flag is set on a file descriptor whose refcount just dropped to 0 and waits for unp_gc() to clear the flag before completely destroying the file descriptor. MFC after: 1 week Reviewed by: rwatson Submitted by: ups Hopefully makes the panics go away: mx1
2007-01-05 19:59:46 +00:00
uma_zfree(unp_zone, unp);
In uipc_close(), we no longer always free the unpcb, as the last reference may be dropped later. In this case, always unlock the unpcb so as not to leak the lock. Found by: kris (BugMagnet)
2007-03-12 14:52:00 +00:00
} else
UNP_PCB_UNLOCK(unp);
Since unp_attach() and unp_detach() are now called only from uipc_attach() and uipc_detach(), merge them into their calling functions.
2006-07-23 10:25:28 +00:00
if (vp) {
int vfslocked;
vfslocked = VFS_LOCK_GIANT(vp->v_mount);
vrele(vp);
VFS_UNLOCK_GIANT(vfslocked);
}
if (local_unp_rights)
taskqueue_enqueue(taskqueue_thread, &unp_gc_task);
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
}
static int
uipc_disconnect(struct socket *so)
{
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
struct unpcb *unp, *unp2;
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
Always acquire the UNIX domain socket subsystem lock (UNP lock) before dereferencing sotounpcb() and checking its value, as so_pcb is protected by protocol locking, not subsystem locking. This prevents races during close() by one thread and use of ths socket in another. unp_bind() now assert the UNP lock, and uipc_bind() now acquires the lock around calls to unp_bind().
2004-08-16 04:41:03 +00:00
unp = sotounpcb(so);
Modify UNIX domain sockets to guarantee, and assume, that so_pcb is always defined for an in-use socket. This allows us to eliminate countless tests of whether so_pcb is non-NULL, eliminating dozens of error cases. For now, retain the call to sotryfree() in the uipc_abort() path, but this will eventually move to soabort(). These new assumptions should be largely correct, and will become more so as the socket/pcb reference model is fixed. Removing the notion that so_pcb can be non-NULL is a critical step towards further fine-graining of the UNIX domain socket locking, as the so_pcb reference no longer needs to be protected using locks, instead it is a property of the socket life cycle.
2006-03-17 13:52:57 +00:00
KASSERT(unp != NULL, ("uipc_disconnect: unp == NULL"));
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
UNP_GLOBAL_WLOCK();
UNP_PCB_LOCK(unp);
unp2 = unp->unp_conn;
if (unp2 != NULL) {
UNP_PCB_LOCK(unp2);
unp_disconnect(unp, unp2);
UNP_PCB_UNLOCK(unp2);
}
UNP_PCB_UNLOCK(unp);
UNP_GLOBAL_WUNLOCK();
Mechanical whitespace cleanup; parenthesize return values; other minor style nits.
2004-01-11 19:48:19 +00:00
return (0);
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
}
static int
Push the assignment of a new or updated so_qlimit from solisten() following the protocol pru_listen() call to solisten_proto(), so that it occurs under the socket lock acquisition that also sets SO_ACCEPTCONN. This requires passing the new backlog parameter to the protocol, which also allows the protocol to be aware of changes in queue limit should it wish to do something about the new queue limit. This continues a move towards the socket layer acting as a library for the protocol. Bump __FreeBSD_version due to a change in the in-kernel protocol interface. This change has been tested with IPv4 and UNIX domain sockets, but not other protocols.
2005-10-30 19:44:40 +00:00
uipc_listen(struct socket *so, int backlog, struct thread *td)
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
{
Always acquire the UNIX domain socket subsystem lock (UNP lock) before dereferencing sotounpcb() and checking its value, as so_pcb is protected by protocol locking, not subsystem locking. This prevents races during close() by one thread and use of ths socket in another. unp_bind() now assert the UNP lock, and uipc_bind() now acquires the lock around calls to unp_bind().
2004-08-16 04:41:03 +00:00
struct unpcb *unp;
Introduce a subsystem lock around UNIX domain sockets in order to protect global and allocated variables. This strategy is derived from work originally developed by BSDi for BSD/OS, and applied to FreeBSD by Sam Leffler: - Add unp_mtx, a global mutex which will protect all UNIX domain socket related variables, structures, etc. - Add UNP_LOCK(), UNP_UNLOCK(), UNP_LOCK_ASSERT() macros. - Acquire unp_mtx on entering most UNIX domain socket code, drop/re-acquire around calls into VFS, and release it on return. - Avoid performing sodupsockaddr() while holding the mutex, so in general move to allocating storage before acquiring the mutex to copy the data. - Make a stack copy of the xucred rather than copying out while holding unp_mtx. Copy the peer credential out after releasing the mutex. - Add additional assertions of vnode locks following VOP_CREATE(). A few notes: - Use of an sx lock for the file list mutex may cause problems with regard to unp_mtx when garbage collection passed file descriptors. - The locking in unp_pcblist() for sysctl monitoring is correct subject to the unpcb zone not returning memory for reuse by other subsystems (consistent with similar existing concerns). - Sam's version of this change, as with the BSD/OS version, made use of both a global lock and per-unpcb locks. However, in practice, the global lock covered all accesses, so I have simplified out the unpcb locks in the interest of getting this merged faster (reducing the overhead but not sacrificing granularity in most cases). We will want to explore possibilities for improving lock granularity in this code in the future. Submitted by: sam Sponsored by: FreeBSD Foundatiuon Obtained from: BSD/OS 5 snapshot provided by BSDi
2004-06-10 21:34:38 +00:00
int error;
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
Always acquire the UNIX domain socket subsystem lock (UNP lock) before dereferencing sotounpcb() and checking its value, as so_pcb is protected by protocol locking, not subsystem locking. This prevents races during close() by one thread and use of ths socket in another. unp_bind() now assert the UNP lock, and uipc_bind() now acquires the lock around calls to unp_bind().
2004-08-16 04:41:03 +00:00
unp = sotounpcb(so);
Modify UNIX domain sockets to guarantee, and assume, that so_pcb is always defined for an in-use socket. This allows us to eliminate countless tests of whether so_pcb is non-NULL, eliminating dozens of error cases. For now, retain the call to sotryfree() in the uipc_abort() path, but this will eventually move to soabort(). These new assumptions should be largely correct, and will become more so as the socket/pcb reference model is fixed. Removing the notion that so_pcb can be non-NULL is a critical step towards further fine-graining of the UNIX domain socket locking, as the so_pcb reference no longer needs to be protected using locks, instead it is a property of the socket life cycle.
2006-03-17 13:52:57 +00:00
KASSERT(unp != NULL, ("uipc_listen: unp == NULL"));
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
UNP_PCB_LOCK(unp);
Modify UNIX domain sockets to guarantee, and assume, that so_pcb is always defined for an in-use socket. This allows us to eliminate countless tests of whether so_pcb is non-NULL, eliminating dozens of error cases. For now, retain the call to sotryfree() in the uipc_abort() path, but this will eventually move to soabort(). These new assumptions should be largely correct, and will become more so as the socket/pcb reference model is fixed. Removing the notion that so_pcb can be non-NULL is a critical step towards further fine-graining of the UNIX domain socket locking, as the so_pcb reference no longer needs to be protected using locks, instead it is a property of the socket life cycle.
2006-03-17 13:52:57 +00:00
if (unp->unp_vnode == NULL) {
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
UNP_PCB_UNLOCK(unp);
Always acquire the UNIX domain socket subsystem lock (UNP lock) before dereferencing sotounpcb() and checking its value, as so_pcb is protected by protocol locking, not subsystem locking. This prevents races during close() by one thread and use of ths socket in another. unp_bind() now assert the UNP lock, and uipc_bind() now acquires the lock around calls to unp_bind().
2004-08-16 04:41:03 +00:00
return (EINVAL);
}
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
SOCK_LOCK(so);
error = solisten_proto_check(so);
if (error == 0) {
cru2x(td->td_ucred, &unp->unp_peercred);
unp->unp_flags |= UNP_HAVEPCCACHED;
solisten_proto(so, backlog);
}
SOCK_UNLOCK(so);
UNP_PCB_UNLOCK(unp);
Introduce a subsystem lock around UNIX domain sockets in order to protect global and allocated variables. This strategy is derived from work originally developed by BSDi for BSD/OS, and applied to FreeBSD by Sam Leffler: - Add unp_mtx, a global mutex which will protect all UNIX domain socket related variables, structures, etc. - Add UNP_LOCK(), UNP_UNLOCK(), UNP_LOCK_ASSERT() macros. - Acquire unp_mtx on entering most UNIX domain socket code, drop/re-acquire around calls into VFS, and release it on return. - Avoid performing sodupsockaddr() while holding the mutex, so in general move to allocating storage before acquiring the mutex to copy the data. - Make a stack copy of the xucred rather than copying out while holding unp_mtx. Copy the peer credential out after releasing the mutex. - Add additional assertions of vnode locks following VOP_CREATE(). A few notes: - Use of an sx lock for the file list mutex may cause problems with regard to unp_mtx when garbage collection passed file descriptors. - The locking in unp_pcblist() for sysctl monitoring is correct subject to the unpcb zone not returning memory for reuse by other subsystems (consistent with similar existing concerns). - Sam's version of this change, as with the BSD/OS version, made use of both a global lock and per-unpcb locks. However, in practice, the global lock covered all accesses, so I have simplified out the unpcb locks in the interest of getting this merged faster (reducing the overhead but not sacrificing granularity in most cases). We will want to explore possibilities for improving lock granularity in this code in the future. Submitted by: sam Sponsored by: FreeBSD Foundatiuon Obtained from: BSD/OS 5 snapshot provided by BSDi
2004-06-10 21:34:38 +00:00
return (error);
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
}
static int
Fix all areas of the system (or at least all those in LINT) to avoid storing socket addresses in mbufs. (Socket buffers are the one exception.) A number of kernel APIs needed to get fixed in order to make this happen. Also, fix three protocol families which kept PCBs in mbufs to not malloc them instead. Delete some old compatibility cruft while we're at it, and add some new routines in the in_cksum family.
1997-08-16 19:16:27 +00:00
uipc_peeraddr(struct socket *so, struct sockaddr **nam)
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
{
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
struct unpcb *unp, *unp2;
Introduce a subsystem lock around UNIX domain sockets in order to protect global and allocated variables. This strategy is derived from work originally developed by BSDi for BSD/OS, and applied to FreeBSD by Sam Leffler: - Add unp_mtx, a global mutex which will protect all UNIX domain socket related variables, structures, etc. - Add UNP_LOCK(), UNP_UNLOCK(), UNP_LOCK_ASSERT() macros. - Acquire unp_mtx on entering most UNIX domain socket code, drop/re-acquire around calls into VFS, and release it on return. - Avoid performing sodupsockaddr() while holding the mutex, so in general move to allocating storage before acquiring the mutex to copy the data. - Make a stack copy of the xucred rather than copying out while holding unp_mtx. Copy the peer credential out after releasing the mutex. - Add additional assertions of vnode locks following VOP_CREATE(). A few notes: - Use of an sx lock for the file list mutex may cause problems with regard to unp_mtx when garbage collection passed file descriptors. - The locking in unp_pcblist() for sysctl monitoring is correct subject to the unpcb zone not returning memory for reuse by other subsystems (consistent with similar existing concerns). - Sam's version of this change, as with the BSD/OS version, made use of both a global lock and per-unpcb locks. However, in practice, the global lock covered all accesses, so I have simplified out the unpcb locks in the interest of getting this merged faster (reducing the overhead but not sacrificing granularity in most cases). We will want to explore possibilities for improving lock granularity in this code in the future. Submitted by: sam Sponsored by: FreeBSD Foundatiuon Obtained from: BSD/OS 5 snapshot provided by BSDi
2004-06-10 21:34:38 +00:00
const struct sockaddr *sa;
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
Modify UNIX domain sockets to guarantee, and assume, that so_pcb is always defined for an in-use socket. This allows us to eliminate countless tests of whether so_pcb is non-NULL, eliminating dozens of error cases. For now, retain the call to sotryfree() in the uipc_abort() path, but this will eventually move to soabort(). These new assumptions should be largely correct, and will become more so as the socket/pcb reference model is fixed. Removing the notion that so_pcb can be non-NULL is a critical step towards further fine-graining of the UNIX domain socket locking, as the so_pcb reference no longer needs to be protected using locks, instead it is a property of the socket life cycle.
2006-03-17 13:52:57 +00:00
unp = sotounpcb(so);
KASSERT(unp != NULL, ("uipc_peeraddr: unp == NULL"));
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
Introduce a subsystem lock around UNIX domain sockets in order to protect global and allocated variables. This strategy is derived from work originally developed by BSDi for BSD/OS, and applied to FreeBSD by Sam Leffler: - Add unp_mtx, a global mutex which will protect all UNIX domain socket related variables, structures, etc. - Add UNP_LOCK(), UNP_UNLOCK(), UNP_LOCK_ASSERT() macros. - Acquire unp_mtx on entering most UNIX domain socket code, drop/re-acquire around calls into VFS, and release it on return. - Avoid performing sodupsockaddr() while holding the mutex, so in general move to allocating storage before acquiring the mutex to copy the data. - Make a stack copy of the xucred rather than copying out while holding unp_mtx. Copy the peer credential out after releasing the mutex. - Add additional assertions of vnode locks following VOP_CREATE(). A few notes: - Use of an sx lock for the file list mutex may cause problems with regard to unp_mtx when garbage collection passed file descriptors. - The locking in unp_pcblist() for sysctl monitoring is correct subject to the unpcb zone not returning memory for reuse by other subsystems (consistent with similar existing concerns). - Sam's version of this change, as with the BSD/OS version, made use of both a global lock and per-unpcb locks. However, in practice, the global lock covered all accesses, so I have simplified out the unpcb locks in the interest of getting this merged faster (reducing the overhead but not sacrificing granularity in most cases). We will want to explore possibilities for improving lock granularity in this code in the future. Submitted by: sam Sponsored by: FreeBSD Foundatiuon Obtained from: BSD/OS 5 snapshot provided by BSDi
2004-06-10 21:34:38 +00:00
*nam = malloc(sizeof(struct sockaddr_un), M_SONAME, M_WAITOK);
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
UNP_PCB_LOCK(unp);
/*
* XXX: It seems that this test always fails even when connection is
* established. So, this else clause is added as workaround to
* return PF_LOCAL sockaddr.
*/
unp2 = unp->unp_conn;
if (unp2 != NULL) {
UNP_PCB_LOCK(unp2);
if (unp2->unp_addr != NULL)
sa = (struct sockaddr *) unp->unp_conn->unp_addr;
else
sa = &sun_noname;
bcopy(sa, *nam, sa->sa_len);
UNP_PCB_UNLOCK(unp2);
} else {
Introduce a subsystem lock around UNIX domain sockets in order to protect global and allocated variables. This strategy is derived from work originally developed by BSDi for BSD/OS, and applied to FreeBSD by Sam Leffler: - Add unp_mtx, a global mutex which will protect all UNIX domain socket related variables, structures, etc. - Add UNP_LOCK(), UNP_UNLOCK(), UNP_LOCK_ASSERT() macros. - Acquire unp_mtx on entering most UNIX domain socket code, drop/re-acquire around calls into VFS, and release it on return. - Avoid performing sodupsockaddr() while holding the mutex, so in general move to allocating storage before acquiring the mutex to copy the data. - Make a stack copy of the xucred rather than copying out while holding unp_mtx. Copy the peer credential out after releasing the mutex. - Add additional assertions of vnode locks following VOP_CREATE(). A few notes: - Use of an sx lock for the file list mutex may cause problems with regard to unp_mtx when garbage collection passed file descriptors. - The locking in unp_pcblist() for sysctl monitoring is correct subject to the unpcb zone not returning memory for reuse by other subsystems (consistent with similar existing concerns). - Sam's version of this change, as with the BSD/OS version, made use of both a global lock and per-unpcb locks. However, in practice, the global lock covered all accesses, so I have simplified out the unpcb locks in the interest of getting this merged faster (reducing the overhead but not sacrificing granularity in most cases). We will want to explore possibilities for improving lock granularity in this code in the future. Submitted by: sam Sponsored by: FreeBSD Foundatiuon Obtained from: BSD/OS 5 snapshot provided by BSDi
2004-06-10 21:34:38 +00:00
sa = &sun_noname;
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
bcopy(sa, *nam, sa->sa_len);
Added comment why this workaround is required. Suggested by: sam MFC after: 1 week
2003-01-22 18:03:06 +00:00
}
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
UNP_PCB_UNLOCK(unp);
Mechanical whitespace cleanup; parenthesize return values; other minor style nits.
2004-01-11 19:48:19 +00:00
return (0);
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
}
static int
uipc_rcvd(struct socket *so, int flags)
{
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
struct unpcb *unp, *unp2;
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
struct socket *so2;
Reduce periods of simultaneous acquisition of various socket buffer locks and the unplock during uipc_rcvd() and uipc_send() by caching certain values from one structure while its locks are held, and applying them to a second structure while its locks are held. If done carefully, this should be correct, and will reduce the amount of work done with the global unp lock held. Tested by: kris (earlier version)
2006-07-11 21:49:54 +00:00
u_int mbcnt, sbcc;
Remove any possibility of hiwat-related race conditions by changing the chgsbsize() call to use a "subject" pointer (&sb.sb_hiwat) and a u_long target to set it to. The whole thing is splnet(). This fixes a problem that jdp has been able to provoke.
2000-08-29 11:28:06 +00:00
u_long newhiwat;
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
Always acquire the UNIX domain socket subsystem lock (UNP lock) before dereferencing sotounpcb() and checking its value, as so_pcb is protected by protocol locking, not subsystem locking. This prevents races during close() by one thread and use of ths socket in another. unp_bind() now assert the UNP lock, and uipc_bind() now acquires the lock around calls to unp_bind().
2004-08-16 04:41:03 +00:00
unp = sotounpcb(so);
Modify UNIX domain sockets to guarantee, and assume, that so_pcb is always defined for an in-use socket. This allows us to eliminate countless tests of whether so_pcb is non-NULL, eliminating dozens of error cases. For now, retain the call to sotryfree() in the uipc_abort() path, but this will eventually move to soabort(). These new assumptions should be largely correct, and will become more so as the socket/pcb reference model is fixed. Removing the notion that so_pcb can be non-NULL is a critical step towards further fine-graining of the UNIX domain socket locking, as the so_pcb reference no longer needs to be protected using locks, instead it is a property of the socket life cycle.
2006-03-17 13:52:57 +00:00
KASSERT(unp != NULL, ("uipc_rcvd: unp == NULL"));
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
if (so->so_type == SOCK_DGRAM)
panic("uipc_rcvd DGRAM?");
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
if (so->so_type != SOCK_STREAM)
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
panic("uipc_rcvd unknown socktype");
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
/*
* Adjust backpressure on sender and wakeup any waiting to write.
*
* The consistency requirements here are a bit complex: we must
* acquire the lock for our own unpcb in order to prevent it from
* disconnecting while in use, changing the unp_conn peer. We do not
* need unp2's lock, since the unp2->unp_socket pointer will remain
* static as long as the unp2 pcb is valid, which it will be until we
* release unp's lock to allow a disconnect. We do need socket
* mutexes for both socket endpoints since we manipulate fields in
* both; we hold both locks at once since we access both
* simultaneously.
*/
SOCKBUF_LOCK(&so->so_rcv);
mbcnt = so->so_rcv.sb_mbcnt;
sbcc = so->so_rcv.sb_cc;
SOCKBUF_UNLOCK(&so->so_rcv);
UNP_PCB_LOCK(unp);
unp2 = unp->unp_conn;
if (unp2 == NULL) {
UNP_PCB_UNLOCK(unp);
return (0);
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
}
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
so2 = unp2->unp_socket;
SOCKBUF_LOCK(&so2->so_snd);
so2->so_snd.sb_mbmax += unp->unp_mbcnt - mbcnt;
newhiwat = so2->so_snd.sb_hiwat + unp->unp_cc - sbcc;
(void)chgsbsize(so2->so_cred->cr_uidinfo, &so2->so_snd.sb_hiwat,
newhiwat, RLIM_INFINITY);
sowwakeup_locked(so2);
unp->unp_mbcnt = mbcnt;
unp->unp_cc = sbcc;
UNP_PCB_UNLOCK(unp);
Mechanical whitespace cleanup; parenthesize return values; other minor style nits.
2004-01-11 19:48:19 +00:00
return (0);
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
}
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
/* pru_rcvoob is EOPNOTSUPP */
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
static int
Fix all areas of the system (or at least all those in LINT) to avoid storing socket addresses in mbufs. (Socket buffers are the one exception.) A number of kernel APIs needed to get fixed in order to make this happen. Also, fix three protocol families which kept PCBs in mbufs to not malloc them instead. Delete some old compatibility cruft while we're at it, and add some new routines in the in_cksum family.
1997-08-16 19:16:27 +00:00
uipc_send(struct socket *so, int flags, struct mbuf *m, struct sockaddr *nam,
style(9)-ize function headers, remove use of 'register'. MFC after: 3 days
2005-02-20 23:22:13 +00:00
struct mbuf *control, struct thread *td)
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
{
In uipc_send() and uipc_rcvd(), store unp->unp_conn pointer in unp2 while working with the second unpcb to make the code more clear.
2006-07-22 18:41:42 +00:00
struct unpcb *unp, *unp2;
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
struct socket *so2;
Reduce periods of simultaneous acquisition of various socket buffer locks and the unplock during uipc_rcvd() and uipc_send() by caching certain values from one structure while its locks are held, and applying them to a second structure while its locks are held. If done carefully, this should be correct, and will reduce the amount of work done with the global unp lock held. Tested by: kris (earlier version)
2006-07-11 21:49:54 +00:00
u_int mbcnt, sbcc;
Remove any possibility of hiwat-related race conditions by changing the chgsbsize() call to use a "subject" pointer (&sb.sb_hiwat) and a u_long target to set it to. The whole thing is splnet(). This fixes a problem that jdp has been able to provoke.
2000-08-29 11:28:06 +00:00
u_long newhiwat;
In uipc_send() and uipc_rcvd(), store unp->unp_conn pointer in unp2 while working with the second unpcb to make the code more clear.
2006-07-22 18:41:42 +00:00
int error = 0;
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
Always acquire the UNIX domain socket subsystem lock (UNP lock) before dereferencing sotounpcb() and checking its value, as so_pcb is protected by protocol locking, not subsystem locking. This prevents races during close() by one thread and use of ths socket in another. unp_bind() now assert the UNP lock, and uipc_bind() now acquires the lock around calls to unp_bind().
2004-08-16 04:41:03 +00:00
unp = sotounpcb(so);
Modify UNIX domain sockets to guarantee, and assume, that so_pcb is always defined for an in-use socket. This allows us to eliminate countless tests of whether so_pcb is non-NULL, eliminating dozens of error cases. For now, retain the call to sotryfree() in the uipc_abort() path, but this will eventually move to soabort(). These new assumptions should be largely correct, and will become more so as the socket/pcb reference model is fixed. Removing the notion that so_pcb can be non-NULL is a critical step towards further fine-graining of the UNIX domain socket locking, as the so_pcb reference no longer needs to be protected using locks, instead it is a property of the socket life cycle.
2006-03-17 13:52:57 +00:00
KASSERT(unp != NULL, ("uipc_send: unp == NULL"));
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
if (flags & PRUS_OOB) {
error = EOPNOTSUPP;
goto release;
}
Prefer NULL to 0 when testing and assigning pointer values.
2004-03-30 02:16:25 +00:00
if (control != NULL && (error = unp_internalize(&control, td)))
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
goto release;
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
if ((nam != NULL) || (flags & PRUS_EOF))
UNP_GLOBAL_WLOCK();
else
UNP_GLOBAL_RLOCK();
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
switch (so->so_type) {
Mechanical whitespace cleanup; parenthesize return values; other minor style nits.
2004-01-11 19:48:19 +00:00
case SOCK_DGRAM:
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
{
Mark sun_noname as const since it's immutable. Update definitions of functions that potentially accept &sun_noname (sbappendaddr(), et al) to accept a const sockaddr pointer.
2004-06-04 04:07:08 +00:00
const struct sockaddr *from;
Remove trailing whitespace.
1995-05-30 08:16:23 +00:00
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
unp2 = unp->unp_conn;
Prefer NULL to 0 when testing and assigning pointer values.
2004-03-30 02:16:25 +00:00
if (nam != NULL) {
Remove two simultaneous acquisitions of multiple unpcb locks from uipc_send in cases where only a global read lock is held by breaking them out and avoiding the unpcb lock acquire in the common case. This avoids deadlocks which manifested with X11, and should also marginally further improve performance. Reported by: sepotvin, brooks
2007-03-01 09:00:42 +00:00
UNP_GLOBAL_WLOCK_ASSERT();
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
if (unp2 != NULL) {
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
error = EISCONN;
break;
}
KSE Milestone 2 Note ALL MODULES MUST BE RECOMPILED make the kernel aware that there are smaller units of scheduling than the process. (but only allow one thread per process at this time). This is functionally equivalent to teh previousl -current except that there is a thread associated with each process. Sorry john! (your next MFC will be a doosie!) Reviewed by: peter@freebsd.org, dillon@freebsd.org X-MFC after: ha ha ha ha
2001-09-12 08:38:13 +00:00
error = unp_connect(so, nam, td);
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
if (error)
break;
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
unp2 = unp->unp_conn;
} else {
if (unp2 == NULL) {
error = ENOTCONN;
break;
}
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
}
Close a race that occurs when using sendto() to connect and send on a UNIX domain socket at the same time as the remote host is closing the new connections as quickly as they open. Since the connect() and send() paths are non-atomic with respect to another, it is possible for the second thread's close() call to disconnect the two sockets as connect() returns, leading to the consumer (which plans to send()) with a NULL kernel pointer to its proposed peer. As a result, after acquiring the UNIX domain socket subsystem lock, we need to revalidate the connection pointers even though connect() has technically succeed, and reurn an error to say that there's no connection on which to perform the send. We might want to rethink the specific errno number, perhaps ECONNRESET would be better. PR: 100940 Reported by: Young Hyun <youngh at caida dot org> MFC after: 2 weeks MFC note: Some adaptation will be required
2006-07-31 23:00:05 +00:00
/*
* Because connect() and send() are non-atomic in a sendto()
* with a target address, it's possible that the socket will
* have disconnected before the send() can run. In that case
* return the slightly counter-intuitive but otherwise
* correct error that the socket is not connected.
*/
if (unp2 == NULL) {
error = ENOTCONN;
break;
}
Remove two simultaneous acquisitions of multiple unpcb locks from uipc_send in cases where only a global read lock is held by breaking them out and avoiding the unpcb lock acquire in the common case. This avoids deadlocks which manifested with X11, and should also marginally further improve performance. Reported by: sepotvin, brooks
2007-03-01 09:00:42 +00:00
/* Lockless read. */
if (unp2->unp_flags & UNP_WANTCRED)
control = unp_addsockcred(td, control);
UNP_PCB_LOCK(unp);
Prefer NULL to 0 when testing and assigning pointer values.
2004-03-30 02:16:25 +00:00
if (unp->unp_addr != NULL)
Fix all areas of the system (or at least all those in LINT) to avoid storing socket addresses in mbufs. (Socket buffers are the one exception.) A number of kernel APIs needed to get fixed in order to make this happen. Also, fix three protocol families which kept PCBs in mbufs to not malloc them instead. Delete some old compatibility cruft while we're at it, and add some new routines in the in_cksum family.
1997-08-16 19:16:27 +00:00
from = (struct sockaddr *)unp->unp_addr;
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
else
from = &sun_noname;
Remove two simultaneous acquisitions of multiple unpcb locks from uipc_send in cases where only a global read lock is held by breaking them out and avoiding the unpcb lock acquire in the common case. This avoids deadlocks which manifested with X11, and should also marginally further improve performance. Reported by: sepotvin, brooks
2007-03-01 09:00:42 +00:00
so2 = unp2->unp_socket;
Merge next step in socket buffer locking: - sowakeup() now asserts the socket buffer lock on entry. Move the call to KNOTE higher in sowakeup() so that it is made with the socket buffer lock held for consistency with other calls. Release the socket buffer lock prior to calling into pgsigio(), so_upcall(), or aio_swake(). Locking for this event management will need revisiting in the future, but this model avoids lock order reversals when upcalls into other subsystems result in socket/socket buffer operations. Assert that the socket buffer lock is not held at the end of the function. - Wrapper macros for sowakeup(), sorwakeup() and sowwakeup(), now have _locked versions which assert the socket buffer lock on entry. If a wakeup is required by sb_notify(), invoke sowakeup(); otherwise, unconditionally release the socket buffer lock. This results in the socket buffer lock being released whether a wakeup is required or not. - Break out socantsendmore() into socantsendmore_locked() that asserts the socket buffer lock. socantsendmore() unconditionally locks the socket buffer before calling socantsendmore_locked(). Note that both functions return with the socket buffer unlocked as socantsendmore_locked() calls sowwakeup_locked() which has the same properties. Assert that the socket buffer is unlocked on return. - Break out socantrcvmore() into socantrcvmore_locked() that asserts the socket buffer lock. socantrcvmore() unconditionally locks the socket buffer before calling socantrcvmore_locked(). Note that both functions return with the socket buffer unlocked as socantrcvmore_locked() calls sorwakeup_locked() which has similar properties. Assert that the socket buffer is unlocked on return. - Break out sbrelease() into a sbrelease_locked() that asserts the socket buffer lock. sbrelease() unconditionally locks the socket buffer before calling sbrelease_locked(). sbrelease_locked() now invokes sbflush_locked() instead of sbflush(). - Assert the socket buffer lock in socket buffer sanity check functions sblastrecordchk(), sblastmbufchk(). - Assert the socket buffer lock in SBLINKRECORD(). - Break out various sbappend() functions into sbappend_locked() (and variations on that name) that assert the socket buffer lock. The !_locked() variations unconditionally lock the socket buffer before calling their _locked counterparts. Internally, make sure to call _locked() support routines, etc, if already holding the socket buffer lock. - Break out sbinsertoob() into sbinsertoob_locked() that asserts the socket buffer lock. sbinsertoob() unconditionally locks the socket buffer before calling sbinsertoob_locked(). - Break out sbflush() into sbflush_locked() that asserts the socket buffer lock. sbflush() unconditionally locks the socket buffer before calling sbflush_locked(). Update panic strings for new function names. - Break out sbdrop() into sbdrop_locked() that asserts the socket buffer lock. sbdrop() unconditionally locks the socket buffer before calling sbdrop_locked(). - Break out sbdroprecord() into sbdroprecord_locked() that asserts the socket buffer lock. sbdroprecord() unconditionally locks the socket buffer before calling sbdroprecord_locked(). - sofree() now calls socantsendmore_locked() and re-acquires the socket buffer lock on return. It also now calls sbrelease_locked(). - sorflush() now calls socantrcvmore_locked() and re-acquires the socket buffer lock on return. Clean up/mess up other behavior in sorflush() relating to the temporary stack copy of the socket buffer used with dom_dispose by more properly initializing the temporary copy, and selectively bzeroing/copying more carefully to prevent WITNESS from getting confused by improperly initialized mutexes. Annotate why that's necessary, or at least, needed. - soisconnected() now calls sbdrop_locked() before unlocking the socket buffer to avoid locking overhead. Some parts of this change were: Submitted by: sam Sponsored by: FreeBSD Foundation Obtained from: BSD/OS
2004-06-21 00:20:43 +00:00
SOCKBUF_LOCK(&so2->so_rcv);
if (sbappendaddr_locked(&so2->so_rcv, from, m, control)) {
Reduce the number of unnecessary unlock-relocks on socket buffer mutexes associated with performing a wakeup on the socket buffer: - When performing an sbappend*() followed by a so[rw]wakeup(), explicitly acquire the socket buffer lock and use the _locked() variants of both calls. Note that the _locked() sowakeup() versions unlock the mutex on return. This is done in uipc_send(), divert_packet(), mroute socket_send(), raw_append(), tcp_reass(), tcp_input(), and udp_append(). - When the socket buffer lock is dropped before a sowakeup(), remove the explicit unlock and use the _locked() sowakeup() variant. This is done in soisdisconnecting(), soisdisconnected() when setting the can't send/ receive flags and dropping data, and in uipc_rcvd() which adjusting back-pressure on the sockets. For UNIX domain sockets running mpsafe with a contention-intensive SMP mysql benchmark, this results in a 1.6% query rate improvement due to reduce mutex costs.
2004-06-26 19:10:39 +00:00
sorwakeup_locked(so2);
Prefer NULL to 0 when testing and assigning pointer values.
2004-03-30 02:16:25 +00:00
m = NULL;
control = NULL;
Mechanical whitespace cleanup; parenthesize return values; other minor style nits.
2004-01-11 19:48:19 +00:00
} else {
Merge next step in socket buffer locking: - sowakeup() now asserts the socket buffer lock on entry. Move the call to KNOTE higher in sowakeup() so that it is made with the socket buffer lock held for consistency with other calls. Release the socket buffer lock prior to calling into pgsigio(), so_upcall(), or aio_swake(). Locking for this event management will need revisiting in the future, but this model avoids lock order reversals when upcalls into other subsystems result in socket/socket buffer operations. Assert that the socket buffer lock is not held at the end of the function. - Wrapper macros for sowakeup(), sorwakeup() and sowwakeup(), now have _locked versions which assert the socket buffer lock on entry. If a wakeup is required by sb_notify(), invoke sowakeup(); otherwise, unconditionally release the socket buffer lock. This results in the socket buffer lock being released whether a wakeup is required or not. - Break out socantsendmore() into socantsendmore_locked() that asserts the socket buffer lock. socantsendmore() unconditionally locks the socket buffer before calling socantsendmore_locked(). Note that both functions return with the socket buffer unlocked as socantsendmore_locked() calls sowwakeup_locked() which has the same properties. Assert that the socket buffer is unlocked on return. - Break out socantrcvmore() into socantrcvmore_locked() that asserts the socket buffer lock. socantrcvmore() unconditionally locks the socket buffer before calling socantrcvmore_locked(). Note that both functions return with the socket buffer unlocked as socantrcvmore_locked() calls sorwakeup_locked() which has similar properties. Assert that the socket buffer is unlocked on return. - Break out sbrelease() into a sbrelease_locked() that asserts the socket buffer lock. sbrelease() unconditionally locks the socket buffer before calling sbrelease_locked(). sbrelease_locked() now invokes sbflush_locked() instead of sbflush(). - Assert the socket buffer lock in socket buffer sanity check functions sblastrecordchk(), sblastmbufchk(). - Assert the socket buffer lock in SBLINKRECORD(). - Break out various sbappend() functions into sbappend_locked() (and variations on that name) that assert the socket buffer lock. The !_locked() variations unconditionally lock the socket buffer before calling their _locked counterparts. Internally, make sure to call _locked() support routines, etc, if already holding the socket buffer lock. - Break out sbinsertoob() into sbinsertoob_locked() that asserts the socket buffer lock. sbinsertoob() unconditionally locks the socket buffer before calling sbinsertoob_locked(). - Break out sbflush() into sbflush_locked() that asserts the socket buffer lock. sbflush() unconditionally locks the socket buffer before calling sbflush_locked(). Update panic strings for new function names. - Break out sbdrop() into sbdrop_locked() that asserts the socket buffer lock. sbdrop() unconditionally locks the socket buffer before calling sbdrop_locked(). - Break out sbdroprecord() into sbdroprecord_locked() that asserts the socket buffer lock. sbdroprecord() unconditionally locks the socket buffer before calling sbdroprecord_locked(). - sofree() now calls socantsendmore_locked() and re-acquires the socket buffer lock on return. It also now calls sbrelease_locked(). - sorflush() now calls socantrcvmore_locked() and re-acquires the socket buffer lock on return. Clean up/mess up other behavior in sorflush() relating to the temporary stack copy of the socket buffer used with dom_dispose by more properly initializing the temporary copy, and selectively bzeroing/copying more carefully to prevent WITNESS from getting confused by improperly initialized mutexes. Annotate why that's necessary, or at least, needed. - soisconnected() now calls sbdrop_locked() before unlocking the socket buffer to avoid locking overhead. Some parts of this change were: Submitted by: sam Sponsored by: FreeBSD Foundation Obtained from: BSD/OS
2004-06-21 00:20:43 +00:00
SOCKBUF_UNLOCK(&so2->so_rcv);
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
error = ENOBUFS;
Mechanical whitespace cleanup; parenthesize return values; other minor style nits.
2004-01-11 19:48:19 +00:00
}
Remove two simultaneous acquisitions of multiple unpcb locks from uipc_send in cases where only a global read lock is held by breaking them out and avoiding the unpcb lock acquire in the common case. This avoids deadlocks which manifested with X11, and should also marginally further improve performance. Reported by: sepotvin, brooks
2007-03-01 09:00:42 +00:00
if (nam != NULL) {
UNP_GLOBAL_WLOCK_ASSERT();
UNP_PCB_LOCK(unp2);
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
unp_disconnect(unp, unp2);
Remove two simultaneous acquisitions of multiple unpcb locks from uipc_send in cases where only a global read lock is held by breaking them out and avoiding the unpcb lock acquire in the common case. This avoids deadlocks which manifested with X11, and should also marginally further improve performance. Reported by: sepotvin, brooks
2007-03-01 09:00:42 +00:00
UNP_PCB_UNLOCK(unp2);
}
UNP_PCB_UNLOCK(unp);
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
break;
}
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
case SOCK_STREAM:
Merge in the socket-level support for Transaction TCP.
1995-02-07 02:01:16 +00:00
/*
Re-wrap and other minor formatting and punctuation fixes for UNIX domain socket comments.
2006-07-22 17:24:55 +00:00
* Connect if not connected yet.
*
* Note: A better implementation would complain if not equal
* to the peer's address.
Merge in the socket-level support for Transaction TCP.
1995-02-07 02:01:16 +00:00
*/
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
if ((so->so_state & SS_ISCONNECTED) == 0) {
Prefer NULL to 0 when testing and assigning pointer values.
2004-03-30 02:16:25 +00:00
if (nam != NULL) {
Remove two simultaneous acquisitions of multiple unpcb locks from uipc_send in cases where only a global read lock is held by breaking them out and avoiding the unpcb lock acquire in the common case. This avoids deadlocks which manifested with X11, and should also marginally further improve performance. Reported by: sepotvin, brooks
2007-03-01 09:00:42 +00:00
UNP_GLOBAL_WLOCK_ASSERT();
KSE Milestone 2 Note ALL MODULES MUST BE RECOMPILED make the kernel aware that there are smaller units of scheduling than the process. (but only allow one thread per process at this time). This is functionally equivalent to teh previousl -current except that there is a thread associated with each process. Sorry john! (your next MFC will be a doosie!) Reviewed by: peter@freebsd.org, dillon@freebsd.org X-MFC after: ha ha ha ha
2001-09-12 08:38:13 +00:00
error = unp_connect(so, nam, td);
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
if (error)
break; /* XXX */
} else {
error = ENOTCONN;
break;
}
Remove two simultaneous acquisitions of multiple unpcb locks from uipc_send in cases where only a global read lock is held by breaking them out and avoiding the unpcb lock acquire in the common case. This avoids deadlocks which manifested with X11, and should also marginally further improve performance. Reported by: sepotvin, brooks
2007-03-01 09:00:42 +00:00
}
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
Reduce periods of simultaneous acquisition of various socket buffer locks and the unplock during uipc_rcvd() and uipc_send() by caching certain values from one structure while its locks are held, and applying them to a second structure while its locks are held. If done carefully, this should be correct, and will reduce the amount of work done with the global unp lock held. Tested by: kris (earlier version)
2006-07-11 21:49:54 +00:00
/* Lockless read. */
The socket field so_state is used to hold a variety of socket related flags relating to several aspects of socket functionality. This change breaks out several bits relating to send and receive operation into a new per-socket buffer field, sb_state, in order to facilitate locking. This is required because, in order to provide more granular locking of sockets, different state fields have different locking properties. The following fields are moved to sb_state: SS_CANTRCVMORE (so_state) SS_CANTSENDMORE (so_state) SS_RCVATMARK (so_state) Rename respectively to: SBS_CANTRCVMORE (so_rcv.sb_state) SBS_CANTSENDMORE (so_snd.sb_state) SBS_RCVATMARK (so_rcv.sb_state) This facilitates locking by isolating fields to be located with other identically locked fields, and permits greater granularity in socket locking by avoiding storing fields with different locking semantics in the same short (avoiding locking conflicts). In the future, we may wish to coallesce sb_state and sb_flags; for the time being I leave them separate and there is no additional memory overhead due to the packing/alignment of shorts in the socket buffer structure.
2004-06-14 18:16:22 +00:00
if (so->so_snd.sb_state & SBS_CANTSENDMORE) {
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
error = EPIPE;
break;
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
}
Close a race that occurs when using sendto() to connect and send on a UNIX domain socket at the same time as the remote host is closing the new connections as quickly as they open. Since the connect() and send() paths are non-atomic with respect to another, it is possible for the second thread's close() call to disconnect the two sockets as connect() returns, leading to the consumer (which plans to send()) with a NULL kernel pointer to its proposed peer. As a result, after acquiring the UNIX domain socket subsystem lock, we need to revalidate the connection pointers even though connect() has technically succeed, and reurn an error to say that there's no connection on which to perform the send. We might want to rethink the specific errno number, perhaps ECONNRESET would be better. PR: 100940 Reported by: Young Hyun <youngh at caida dot org> MFC after: 2 weeks MFC note: Some adaptation will be required
2006-07-31 23:00:05 +00:00
/*
* Because connect() and send() are non-atomic in a sendto()
* with a target address, it's possible that the socket will
* have disconnected before the send() can run. In that case
* return the slightly counter-intuitive but otherwise
* correct error that the socket is not connected.
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
*
* Lock order here has to be handled carefully: we hold the
* global lock, so acquiring two unpcb locks is OK. We must
* acquire both before acquiring any socket mutexes. We must
* also acquire the local socket send mutex before the remote
* socket receive mutex. The only tricky thing is making
* sure to acquire the unp2 lock before the local socket send
* lock, or we will experience deadlocks.
Close a race that occurs when using sendto() to connect and send on a UNIX domain socket at the same time as the remote host is closing the new connections as quickly as they open. Since the connect() and send() paths are non-atomic with respect to another, it is possible for the second thread's close() call to disconnect the two sockets as connect() returns, leading to the consumer (which plans to send()) with a NULL kernel pointer to its proposed peer. As a result, after acquiring the UNIX domain socket subsystem lock, we need to revalidate the connection pointers even though connect() has technically succeed, and reurn an error to say that there's no connection on which to perform the send. We might want to rethink the specific errno number, perhaps ECONNRESET would be better. PR: 100940 Reported by: Young Hyun <youngh at caida dot org> MFC after: 2 weeks MFC note: Some adaptation will be required
2006-07-31 23:00:05 +00:00
*/
In uipc_send() and uipc_rcvd(), store unp->unp_conn pointer in unp2 while working with the second unpcb to make the code more clear.
2006-07-22 18:41:42 +00:00
unp2 = unp->unp_conn;
Close a race that occurs when using sendto() to connect and send on a UNIX domain socket at the same time as the remote host is closing the new connections as quickly as they open. Since the connect() and send() paths are non-atomic with respect to another, it is possible for the second thread's close() call to disconnect the two sockets as connect() returns, leading to the consumer (which plans to send()) with a NULL kernel pointer to its proposed peer. As a result, after acquiring the UNIX domain socket subsystem lock, we need to revalidate the connection pointers even though connect() has technically succeed, and reurn an error to say that there's no connection on which to perform the send. We might want to rethink the specific errno number, perhaps ECONNRESET would be better. PR: 100940 Reported by: Young Hyun <youngh at caida dot org> MFC after: 2 weeks MFC note: Some adaptation will be required
2006-07-31 23:00:05 +00:00
if (unp2 == NULL) {
error = ENOTCONN;
break;
}
In uipc_send() and uipc_rcvd(), store unp->unp_conn pointer in unp2 while working with the second unpcb to make the code more clear.
2006-07-22 18:41:42 +00:00
so2 = unp2->unp_socket;
Remove two simultaneous acquisitions of multiple unpcb locks from uipc_send in cases where only a global read lock is held by breaking them out and avoiding the unpcb lock acquire in the common case. This avoids deadlocks which manifested with X11, and should also marginally further improve performance. Reported by: sepotvin, brooks
2007-03-01 09:00:42 +00:00
UNP_PCB_LOCK(unp2);
Merge next step in socket buffer locking: - sowakeup() now asserts the socket buffer lock on entry. Move the call to KNOTE higher in sowakeup() so that it is made with the socket buffer lock held for consistency with other calls. Release the socket buffer lock prior to calling into pgsigio(), so_upcall(), or aio_swake(). Locking for this event management will need revisiting in the future, but this model avoids lock order reversals when upcalls into other subsystems result in socket/socket buffer operations. Assert that the socket buffer lock is not held at the end of the function. - Wrapper macros for sowakeup(), sorwakeup() and sowwakeup(), now have _locked versions which assert the socket buffer lock on entry. If a wakeup is required by sb_notify(), invoke sowakeup(); otherwise, unconditionally release the socket buffer lock. This results in the socket buffer lock being released whether a wakeup is required or not. - Break out socantsendmore() into socantsendmore_locked() that asserts the socket buffer lock. socantsendmore() unconditionally locks the socket buffer before calling socantsendmore_locked(). Note that both functions return with the socket buffer unlocked as socantsendmore_locked() calls sowwakeup_locked() which has the same properties. Assert that the socket buffer is unlocked on return. - Break out socantrcvmore() into socantrcvmore_locked() that asserts the socket buffer lock. socantrcvmore() unconditionally locks the socket buffer before calling socantrcvmore_locked(). Note that both functions return with the socket buffer unlocked as socantrcvmore_locked() calls sorwakeup_locked() which has similar properties. Assert that the socket buffer is unlocked on return. - Break out sbrelease() into a sbrelease_locked() that asserts the socket buffer lock. sbrelease() unconditionally locks the socket buffer before calling sbrelease_locked(). sbrelease_locked() now invokes sbflush_locked() instead of sbflush(). - Assert the socket buffer lock in socket buffer sanity check functions sblastrecordchk(), sblastmbufchk(). - Assert the socket buffer lock in SBLINKRECORD(). - Break out various sbappend() functions into sbappend_locked() (and variations on that name) that assert the socket buffer lock. The !_locked() variations unconditionally lock the socket buffer before calling their _locked counterparts. Internally, make sure to call _locked() support routines, etc, if already holding the socket buffer lock. - Break out sbinsertoob() into sbinsertoob_locked() that asserts the socket buffer lock. sbinsertoob() unconditionally locks the socket buffer before calling sbinsertoob_locked(). - Break out sbflush() into sbflush_locked() that asserts the socket buffer lock. sbflush() unconditionally locks the socket buffer before calling sbflush_locked(). Update panic strings for new function names. - Break out sbdrop() into sbdrop_locked() that asserts the socket buffer lock. sbdrop() unconditionally locks the socket buffer before calling sbdrop_locked(). - Break out sbdroprecord() into sbdroprecord_locked() that asserts the socket buffer lock. sbdroprecord() unconditionally locks the socket buffer before calling sbdroprecord_locked(). - sofree() now calls socantsendmore_locked() and re-acquires the socket buffer lock on return. It also now calls sbrelease_locked(). - sorflush() now calls socantrcvmore_locked() and re-acquires the socket buffer lock on return. Clean up/mess up other behavior in sorflush() relating to the temporary stack copy of the socket buffer used with dom_dispose by more properly initializing the temporary copy, and selectively bzeroing/copying more carefully to prevent WITNESS from getting confused by improperly initialized mutexes. Annotate why that's necessary, or at least, needed. - soisconnected() now calls sbdrop_locked() before unlocking the socket buffer to avoid locking overhead. Some parts of this change were: Submitted by: sam Sponsored by: FreeBSD Foundation Obtained from: BSD/OS
2004-06-21 00:20:43 +00:00
SOCKBUF_LOCK(&so2->so_rcv);
In uipc_send() and uipc_rcvd(), store unp->unp_conn pointer in unp2 while working with the second unpcb to make the code more clear.
2006-07-22 18:41:42 +00:00
if (unp2->unp_flags & UNP_WANTCRED) {
Implement unix(4) socket options LOCAL_CREDS and LOCAL_CONNWAIT. - Add unp_addsockcred() (for LOCAL_CREDS). - Add an argument to unp_connect2() to differentiate between PRU_CONNECT and PRU_CONNECT2. (for LOCAL_CONNWAIT) Obtained from: NetBSD (with some changes)
2005-04-13 00:01:46 +00:00
/*
Remove two simultaneous acquisitions of multiple unpcb locks from uipc_send in cases where only a global read lock is held by breaking them out and avoiding the unpcb lock acquire in the common case. This avoids deadlocks which manifested with X11, and should also marginally further improve performance. Reported by: sepotvin, brooks
2007-03-01 09:00:42 +00:00
* Credentials are passed only once on SOCK_STREAM.
Implement unix(4) socket options LOCAL_CREDS and LOCAL_CONNWAIT. - Add unp_addsockcred() (for LOCAL_CREDS). - Add an argument to unp_connect2() to differentiate between PRU_CONNECT and PRU_CONNECT2. (for LOCAL_CONNWAIT) Obtained from: NetBSD (with some changes)
2005-04-13 00:01:46 +00:00
*/
In uipc_send() and uipc_rcvd(), store unp->unp_conn pointer in unp2 while working with the second unpcb to make the code more clear.
2006-07-22 18:41:42 +00:00
unp2->unp_flags &= ~UNP_WANTCRED;
Implement unix(4) socket options LOCAL_CREDS and LOCAL_CONNWAIT. - Add unp_addsockcred() (for LOCAL_CREDS). - Add an argument to unp_connect2() to differentiate between PRU_CONNECT and PRU_CONNECT2. (for LOCAL_CONNWAIT) Obtained from: NetBSD (with some changes)
2005-04-13 00:01:46 +00:00
control = unp_addsockcred(td, control);
}
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
/*
Re-wrap and other minor formatting and punctuation fixes for UNIX domain socket comments.
2006-07-22 17:24:55 +00:00
* Send to paired receive port, and then reduce send buffer
* hiwater marks to maintain backpressure. Wake up readers.
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
*/
Prefer NULL to 0 when testing and assigning pointer values.
2004-03-30 02:16:25 +00:00
if (control != NULL) {
Merge next step in socket buffer locking: - sowakeup() now asserts the socket buffer lock on entry. Move the call to KNOTE higher in sowakeup() so that it is made with the socket buffer lock held for consistency with other calls. Release the socket buffer lock prior to calling into pgsigio(), so_upcall(), or aio_swake(). Locking for this event management will need revisiting in the future, but this model avoids lock order reversals when upcalls into other subsystems result in socket/socket buffer operations. Assert that the socket buffer lock is not held at the end of the function. - Wrapper macros for sowakeup(), sorwakeup() and sowwakeup(), now have _locked versions which assert the socket buffer lock on entry. If a wakeup is required by sb_notify(), invoke sowakeup(); otherwise, unconditionally release the socket buffer lock. This results in the socket buffer lock being released whether a wakeup is required or not. - Break out socantsendmore() into socantsendmore_locked() that asserts the socket buffer lock. socantsendmore() unconditionally locks the socket buffer before calling socantsendmore_locked(). Note that both functions return with the socket buffer unlocked as socantsendmore_locked() calls sowwakeup_locked() which has the same properties. Assert that the socket buffer is unlocked on return. - Break out socantrcvmore() into socantrcvmore_locked() that asserts the socket buffer lock. socantrcvmore() unconditionally locks the socket buffer before calling socantrcvmore_locked(). Note that both functions return with the socket buffer unlocked as socantrcvmore_locked() calls sorwakeup_locked() which has similar properties. Assert that the socket buffer is unlocked on return. - Break out sbrelease() into a sbrelease_locked() that asserts the socket buffer lock. sbrelease() unconditionally locks the socket buffer before calling sbrelease_locked(). sbrelease_locked() now invokes sbflush_locked() instead of sbflush(). - Assert the socket buffer lock in socket buffer sanity check functions sblastrecordchk(), sblastmbufchk(). - Assert the socket buffer lock in SBLINKRECORD(). - Break out various sbappend() functions into sbappend_locked() (and variations on that name) that assert the socket buffer lock. The !_locked() variations unconditionally lock the socket buffer before calling their _locked counterparts. Internally, make sure to call _locked() support routines, etc, if already holding the socket buffer lock. - Break out sbinsertoob() into sbinsertoob_locked() that asserts the socket buffer lock. sbinsertoob() unconditionally locks the socket buffer before calling sbinsertoob_locked(). - Break out sbflush() into sbflush_locked() that asserts the socket buffer lock. sbflush() unconditionally locks the socket buffer before calling sbflush_locked(). Update panic strings for new function names. - Break out sbdrop() into sbdrop_locked() that asserts the socket buffer lock. sbdrop() unconditionally locks the socket buffer before calling sbdrop_locked(). - Break out sbdroprecord() into sbdroprecord_locked() that asserts the socket buffer lock. sbdroprecord() unconditionally locks the socket buffer before calling sbdroprecord_locked(). - sofree() now calls socantsendmore_locked() and re-acquires the socket buffer lock on return. It also now calls sbrelease_locked(). - sorflush() now calls socantrcvmore_locked() and re-acquires the socket buffer lock on return. Clean up/mess up other behavior in sorflush() relating to the temporary stack copy of the socket buffer used with dom_dispose by more properly initializing the temporary copy, and selectively bzeroing/copying more carefully to prevent WITNESS from getting confused by improperly initialized mutexes. Annotate why that's necessary, or at least, needed. - soisconnected() now calls sbdrop_locked() before unlocking the socket buffer to avoid locking overhead. Some parts of this change were: Submitted by: sam Sponsored by: FreeBSD Foundation Obtained from: BSD/OS
2004-06-21 00:20:43 +00:00
if (sbappendcontrol_locked(&so2->so_rcv, m, control))
Prefer NULL to 0 when testing and assigning pointer values.
2004-03-30 02:16:25 +00:00
control = NULL;
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
} else
Merge next step in socket buffer locking: - sowakeup() now asserts the socket buffer lock on entry. Move the call to KNOTE higher in sowakeup() so that it is made with the socket buffer lock held for consistency with other calls. Release the socket buffer lock prior to calling into pgsigio(), so_upcall(), or aio_swake(). Locking for this event management will need revisiting in the future, but this model avoids lock order reversals when upcalls into other subsystems result in socket/socket buffer operations. Assert that the socket buffer lock is not held at the end of the function. - Wrapper macros for sowakeup(), sorwakeup() and sowwakeup(), now have _locked versions which assert the socket buffer lock on entry. If a wakeup is required by sb_notify(), invoke sowakeup(); otherwise, unconditionally release the socket buffer lock. This results in the socket buffer lock being released whether a wakeup is required or not. - Break out socantsendmore() into socantsendmore_locked() that asserts the socket buffer lock. socantsendmore() unconditionally locks the socket buffer before calling socantsendmore_locked(). Note that both functions return with the socket buffer unlocked as socantsendmore_locked() calls sowwakeup_locked() which has the same properties. Assert that the socket buffer is unlocked on return. - Break out socantrcvmore() into socantrcvmore_locked() that asserts the socket buffer lock. socantrcvmore() unconditionally locks the socket buffer before calling socantrcvmore_locked(). Note that both functions return with the socket buffer unlocked as socantrcvmore_locked() calls sorwakeup_locked() which has similar properties. Assert that the socket buffer is unlocked on return. - Break out sbrelease() into a sbrelease_locked() that asserts the socket buffer lock. sbrelease() unconditionally locks the socket buffer before calling sbrelease_locked(). sbrelease_locked() now invokes sbflush_locked() instead of sbflush(). - Assert the socket buffer lock in socket buffer sanity check functions sblastrecordchk(), sblastmbufchk(). - Assert the socket buffer lock in SBLINKRECORD(). - Break out various sbappend() functions into sbappend_locked() (and variations on that name) that assert the socket buffer lock. The !_locked() variations unconditionally lock the socket buffer before calling their _locked counterparts. Internally, make sure to call _locked() support routines, etc, if already holding the socket buffer lock. - Break out sbinsertoob() into sbinsertoob_locked() that asserts the socket buffer lock. sbinsertoob() unconditionally locks the socket buffer before calling sbinsertoob_locked(). - Break out sbflush() into sbflush_locked() that asserts the socket buffer lock. sbflush() unconditionally locks the socket buffer before calling sbflush_locked(). Update panic strings for new function names. - Break out sbdrop() into sbdrop_locked() that asserts the socket buffer lock. sbdrop() unconditionally locks the socket buffer before calling sbdrop_locked(). - Break out sbdroprecord() into sbdroprecord_locked() that asserts the socket buffer lock. sbdroprecord() unconditionally locks the socket buffer before calling sbdroprecord_locked(). - sofree() now calls socantsendmore_locked() and re-acquires the socket buffer lock on return. It also now calls sbrelease_locked(). - sorflush() now calls socantrcvmore_locked() and re-acquires the socket buffer lock on return. Clean up/mess up other behavior in sorflush() relating to the temporary stack copy of the socket buffer used with dom_dispose by more properly initializing the temporary copy, and selectively bzeroing/copying more carefully to prevent WITNESS from getting confused by improperly initialized mutexes. Annotate why that's necessary, or at least, needed. - soisconnected() now calls sbdrop_locked() before unlocking the socket buffer to avoid locking overhead. Some parts of this change were: Submitted by: sam Sponsored by: FreeBSD Foundation Obtained from: BSD/OS
2004-06-21 00:20:43 +00:00
sbappend_locked(&so2->so_rcv, m);
In uipc_send() and uipc_rcvd(), store unp->unp_conn pointer in unp2 while working with the second unpcb to make the code more clear.
2006-07-22 18:41:42 +00:00
mbcnt = so2->so_rcv.sb_mbcnt - unp2->unp_mbcnt;
unp2->unp_mbcnt = so2->so_rcv.sb_mbcnt;
Reduce periods of simultaneous acquisition of various socket buffer locks and the unplock during uipc_rcvd() and uipc_send() by caching certain values from one structure while its locks are held, and applying them to a second structure while its locks are held. If done carefully, this should be correct, and will reduce the amount of work done with the global unp lock held. Tested by: kris (earlier version)
2006-07-11 21:49:54 +00:00
sbcc = so2->so_rcv.sb_cc;
sorwakeup_locked(so2);
SOCKBUF_LOCK(&so->so_snd);
In uipc_send() and uipc_rcvd(), store unp->unp_conn pointer in unp2 while working with the second unpcb to make the code more clear.
2006-07-22 18:41:42 +00:00
newhiwat = so->so_snd.sb_hiwat - (sbcc - unp2->unp_cc);
Remove uidinfo hash table lookup and maintenance out of chgproccnt() and chgsbsize(), which are called rather frequently and may be called from an interrupt context in the case of chgsbsize(). Instead, do the hash table lookup and maintenance when credentials are changed, which is a lot less frequent. Add pointers to the uidinfo structures to the ucred and pcred structures for fast access. Pass a pointer to the credential to chgproccnt() and chgsbsize() instead of passing the uid. Add a reference count to the uidinfo structure and use it to decide when to free the structure rather than freeing the structure when the resource consumption drops to zero. Move the resource tracking code from kern_proc.c to kern_resource.c. Move some duplicate code sequences in kern_prot.c to separate helper functions. Change KASSERTs in this code to unconditional tests and calls to panic().
2000-09-05 22:11:13 +00:00
(void)chgsbsize(so->so_cred->cr_uidinfo, &so->so_snd.sb_hiwat,
Remove any possibility of hiwat-related race conditions by changing the chgsbsize() call to use a "subject" pointer (&sb.sb_hiwat) and a u_long target to set it to. The whole thing is splnet(). This fixes a problem that jdp has been able to provoke.
2000-08-29 11:28:06 +00:00
newhiwat, RLIM_INFINITY);
Reduce periods of simultaneous acquisition of various socket buffer locks and the unplock during uipc_rcvd() and uipc_send() by caching certain values from one structure while its locks are held, and applying them to a second structure while its locks are held. If done carefully, this should be correct, and will reduce the amount of work done with the global unp lock held. Tested by: kris (earlier version)
2006-07-11 21:49:54 +00:00
so->so_snd.sb_mbmax -= mbcnt;
Add send buffer locking to uipc_send(). Without this locking a race can occur between a reader and a writer that results in a panic upon close, e.g., "panic: sbflush_locked: cc 4 || mb 0xffffff0052afa400 || mbcnt 0" Reviewed by: rwatson@ MFC after: 2 weeks
2004-12-22 20:28:46 +00:00
SOCKBUF_UNLOCK(&so->so_snd);
In uipc_send() and uipc_rcvd(), store unp->unp_conn pointer in unp2 while working with the second unpcb to make the code more clear.
2006-07-22 18:41:42 +00:00
unp2->unp_cc = sbcc;
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
UNP_PCB_UNLOCK(unp2);
Prefer NULL to 0 when testing and assigning pointer values.
2004-03-30 02:16:25 +00:00
m = NULL;
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
break;
default:
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
panic("uipc_send unknown socktype");
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
}
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
/*
Remove two simultaneous acquisitions of multiple unpcb locks from uipc_send in cases where only a global read lock is held by breaking them out and avoiding the unpcb lock acquire in the common case. This avoids deadlocks which manifested with X11, and should also marginally further improve performance. Reported by: sepotvin, brooks
2007-03-01 09:00:42 +00:00
* SEND_EOF is equivalent to a SEND followed by a SHUTDOWN.
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
*/
if (flags & PRUS_EOF) {
Remove two simultaneous acquisitions of multiple unpcb locks from uipc_send in cases where only a global read lock is held by breaking them out and avoiding the unpcb lock acquire in the common case. This avoids deadlocks which manifested with X11, and should also marginally further improve performance. Reported by: sepotvin, brooks
2007-03-01 09:00:42 +00:00
UNP_PCB_LOCK(unp);
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
socantsendmore(so);
unp_shutdown(unp);
Remove two simultaneous acquisitions of multiple unpcb locks from uipc_send in cases where only a global read lock is held by breaking them out and avoiding the unpcb lock acquire in the common case. This avoids deadlocks which manifested with X11, and should also marginally further improve performance. Reported by: sepotvin, brooks
2007-03-01 09:00:42 +00:00
UNP_PCB_UNLOCK(unp);
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
}
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
if ((nam != NULL) || (flags & PRUS_EOF))
UNP_GLOBAL_WUNLOCK();
else
UNP_GLOBAL_RUNLOCK();
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
Prefer NULL to 0 when testing and assigning pointer values.
2004-03-30 02:16:25 +00:00
if (control != NULL && error != 0)
Fix descriptor leak provoked by KKIS.05051999.003b exploit code. unp_internalize() takes a reference to the descriptor. If the send fails after unp_internalize(), the control mbuf would be freed ophaning the reference. Tested in -CURRENT by: Pierre Beyssac <beyssac@enst.fr>
1999-05-10 18:09:39 +00:00
unp_dispose(control);
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
release:
Prefer NULL to 0 when testing and assigning pointer values.
2004-03-30 02:16:25 +00:00
if (control != NULL)
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
m_freem(control);
Prefer NULL to 0 when testing and assigning pointer values.
2004-03-30 02:16:25 +00:00
if (m != NULL)
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
m_freem(m);
Mechanical whitespace cleanup; parenthesize return values; other minor style nits.
2004-01-11 19:48:19 +00:00
return (error);
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
}
static int
uipc_sense(struct socket *so, struct stat *sb)
{
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
struct unpcb *unp, *unp2;
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
struct socket *so2;
Always acquire the UNIX domain socket subsystem lock (UNP lock) before dereferencing sotounpcb() and checking its value, as so_pcb is protected by protocol locking, not subsystem locking. This prevents races during close() by one thread and use of ths socket in another. unp_bind() now assert the UNP lock, and uipc_bind() now acquires the lock around calls to unp_bind().
2004-08-16 04:41:03 +00:00
unp = sotounpcb(so);
Modify UNIX domain sockets to guarantee, and assume, that so_pcb is always defined for an in-use socket. This allows us to eliminate countless tests of whether so_pcb is non-NULL, eliminating dozens of error cases. For now, retain the call to sotryfree() in the uipc_abort() path, but this will eventually move to soabort(). These new assumptions should be largely correct, and will become more so as the socket/pcb reference model is fixed. Removing the notion that so_pcb can be non-NULL is a critical step towards further fine-graining of the UNIX domain socket locking, as the so_pcb reference no longer needs to be protected using locks, instead it is a property of the socket life cycle.
2006-03-17 13:52:57 +00:00
KASSERT(unp != NULL, ("uipc_sense: unp == NULL"));
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
sb->st_blksize = so->so_snd.sb_hiwat;
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
UNP_GLOBAL_RLOCK();
UNP_PCB_LOCK(unp);
unp2 = unp->unp_conn;
if (so->so_type == SOCK_STREAM && unp2 != NULL) {
so2 = unp2->unp_socket;
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
sb->st_blksize += so2->so_rcv.sb_cc;
}
Second half of the dev_t cleanup. The big lines are: NODEV -> NULL NOUDEV -> NODEV udev_t -> dev_t udev2dev() -> findcdev() Various minor adjustments including handling of userland access to kernel space struct cdev etc.
2004-06-17 17:16:53 +00:00
sb->st_dev = NODEV;
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
if (unp->unp_ino == 0)
Ensure that the made-up inode number for a Unix domain socket is persistent.
2002-12-25 07:59:39 +00:00
unp->unp_ino = (++unp_ino == 0) ? ++unp_ino : unp_ino;
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
sb->st_ino = unp->unp_ino;
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
UNP_PCB_UNLOCK(unp);
UNP_GLOBAL_RUNLOCK();
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
return (0);
}
static int
uipc_shutdown(struct socket *so)
{
Always acquire the UNIX domain socket subsystem lock (UNP lock) before dereferencing sotounpcb() and checking its value, as so_pcb is protected by protocol locking, not subsystem locking. This prevents races during close() by one thread and use of ths socket in another. unp_bind() now assert the UNP lock, and uipc_bind() now acquires the lock around calls to unp_bind().
2004-08-16 04:41:03 +00:00
struct unpcb *unp;
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
Always acquire the UNIX domain socket subsystem lock (UNP lock) before dereferencing sotounpcb() and checking its value, as so_pcb is protected by protocol locking, not subsystem locking. This prevents races during close() by one thread and use of ths socket in another. unp_bind() now assert the UNP lock, and uipc_bind() now acquires the lock around calls to unp_bind().
2004-08-16 04:41:03 +00:00
unp = sotounpcb(so);
Modify UNIX domain sockets to guarantee, and assume, that so_pcb is always defined for an in-use socket. This allows us to eliminate countless tests of whether so_pcb is non-NULL, eliminating dozens of error cases. For now, retain the call to sotryfree() in the uipc_abort() path, but this will eventually move to soabort(). These new assumptions should be largely correct, and will become more so as the socket/pcb reference model is fixed. Removing the notion that so_pcb can be non-NULL is a critical step towards further fine-graining of the UNIX domain socket locking, as the so_pcb reference no longer needs to be protected using locks, instead it is a property of the socket life cycle.
2006-03-17 13:52:57 +00:00
KASSERT(unp != NULL, ("uipc_shutdown: unp == NULL"));
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
UNP_GLOBAL_WLOCK();
UNP_PCB_LOCK(unp);
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
socantsendmore(so);
unp_shutdown(unp);
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
UNP_PCB_UNLOCK(unp);
UNP_GLOBAL_WUNLOCK();
Mechanical whitespace cleanup; parenthesize return values; other minor style nits.
2004-01-11 19:48:19 +00:00
return (0);
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
}
static int
Fix all areas of the system (or at least all those in LINT) to avoid storing socket addresses in mbufs. (Socket buffers are the one exception.) A number of kernel APIs needed to get fixed in order to make this happen. Also, fix three protocol families which kept PCBs in mbufs to not malloc them instead. Delete some old compatibility cruft while we're at it, and add some new routines in the in_cksum family.
1997-08-16 19:16:27 +00:00
uipc_sockaddr(struct socket *so, struct sockaddr **nam)
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
{
Always acquire the UNIX domain socket subsystem lock (UNP lock) before dereferencing sotounpcb() and checking its value, as so_pcb is protected by protocol locking, not subsystem locking. This prevents races during close() by one thread and use of ths socket in another. unp_bind() now assert the UNP lock, and uipc_bind() now acquires the lock around calls to unp_bind().
2004-08-16 04:41:03 +00:00
struct unpcb *unp;
Introduce a subsystem lock around UNIX domain sockets in order to protect global and allocated variables. This strategy is derived from work originally developed by BSDi for BSD/OS, and applied to FreeBSD by Sam Leffler: - Add unp_mtx, a global mutex which will protect all UNIX domain socket related variables, structures, etc. - Add UNP_LOCK(), UNP_UNLOCK(), UNP_LOCK_ASSERT() macros. - Acquire unp_mtx on entering most UNIX domain socket code, drop/re-acquire around calls into VFS, and release it on return. - Avoid performing sodupsockaddr() while holding the mutex, so in general move to allocating storage before acquiring the mutex to copy the data. - Make a stack copy of the xucred rather than copying out while holding unp_mtx. Copy the peer credential out after releasing the mutex. - Add additional assertions of vnode locks following VOP_CREATE(). A few notes: - Use of an sx lock for the file list mutex may cause problems with regard to unp_mtx when garbage collection passed file descriptors. - The locking in unp_pcblist() for sysctl monitoring is correct subject to the unpcb zone not returning memory for reuse by other subsystems (consistent with similar existing concerns). - Sam's version of this change, as with the BSD/OS version, made use of both a global lock and per-unpcb locks. However, in practice, the global lock covered all accesses, so I have simplified out the unpcb locks in the interest of getting this merged faster (reducing the overhead but not sacrificing granularity in most cases). We will want to explore possibilities for improving lock granularity in this code in the future. Submitted by: sam Sponsored by: FreeBSD Foundatiuon Obtained from: BSD/OS 5 snapshot provided by BSDi
2004-06-10 21:34:38 +00:00
const struct sockaddr *sa;
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
Modify UNIX domain sockets to guarantee, and assume, that so_pcb is always defined for an in-use socket. This allows us to eliminate countless tests of whether so_pcb is non-NULL, eliminating dozens of error cases. For now, retain the call to sotryfree() in the uipc_abort() path, but this will eventually move to soabort(). These new assumptions should be largely correct, and will become more so as the socket/pcb reference model is fixed. Removing the notion that so_pcb can be non-NULL is a critical step towards further fine-graining of the UNIX domain socket locking, as the so_pcb reference no longer needs to be protected using locks, instead it is a property of the socket life cycle.
2006-03-17 13:52:57 +00:00
unp = sotounpcb(so);
KASSERT(unp != NULL, ("uipc_sockaddr: unp == NULL"));
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
Introduce a subsystem lock around UNIX domain sockets in order to protect global and allocated variables. This strategy is derived from work originally developed by BSDi for BSD/OS, and applied to FreeBSD by Sam Leffler: - Add unp_mtx, a global mutex which will protect all UNIX domain socket related variables, structures, etc. - Add UNP_LOCK(), UNP_UNLOCK(), UNP_LOCK_ASSERT() macros. - Acquire unp_mtx on entering most UNIX domain socket code, drop/re-acquire around calls into VFS, and release it on return. - Avoid performing sodupsockaddr() while holding the mutex, so in general move to allocating storage before acquiring the mutex to copy the data. - Make a stack copy of the xucred rather than copying out while holding unp_mtx. Copy the peer credential out after releasing the mutex. - Add additional assertions of vnode locks following VOP_CREATE(). A few notes: - Use of an sx lock for the file list mutex may cause problems with regard to unp_mtx when garbage collection passed file descriptors. - The locking in unp_pcblist() for sysctl monitoring is correct subject to the unpcb zone not returning memory for reuse by other subsystems (consistent with similar existing concerns). - Sam's version of this change, as with the BSD/OS version, made use of both a global lock and per-unpcb locks. However, in practice, the global lock covered all accesses, so I have simplified out the unpcb locks in the interest of getting this merged faster (reducing the overhead but not sacrificing granularity in most cases). We will want to explore possibilities for improving lock granularity in this code in the future. Submitted by: sam Sponsored by: FreeBSD Foundatiuon Obtained from: BSD/OS 5 snapshot provided by BSDi
2004-06-10 21:34:38 +00:00
*nam = malloc(sizeof(struct sockaddr_un), M_SONAME, M_WAITOK);
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
UNP_PCB_LOCK(unp);
Prefer NULL to 0 when testing and assigning pointer values.
2004-03-30 02:16:25 +00:00
if (unp->unp_addr != NULL)
Introduce a subsystem lock around UNIX domain sockets in order to protect global and allocated variables. This strategy is derived from work originally developed by BSDi for BSD/OS, and applied to FreeBSD by Sam Leffler: - Add unp_mtx, a global mutex which will protect all UNIX domain socket related variables, structures, etc. - Add UNP_LOCK(), UNP_UNLOCK(), UNP_LOCK_ASSERT() macros. - Acquire unp_mtx on entering most UNIX domain socket code, drop/re-acquire around calls into VFS, and release it on return. - Avoid performing sodupsockaddr() while holding the mutex, so in general move to allocating storage before acquiring the mutex to copy the data. - Make a stack copy of the xucred rather than copying out while holding unp_mtx. Copy the peer credential out after releasing the mutex. - Add additional assertions of vnode locks following VOP_CREATE(). A few notes: - Use of an sx lock for the file list mutex may cause problems with regard to unp_mtx when garbage collection passed file descriptors. - The locking in unp_pcblist() for sysctl monitoring is correct subject to the unpcb zone not returning memory for reuse by other subsystems (consistent with similar existing concerns). - Sam's version of this change, as with the BSD/OS version, made use of both a global lock and per-unpcb locks. However, in practice, the global lock covered all accesses, so I have simplified out the unpcb locks in the interest of getting this merged faster (reducing the overhead but not sacrificing granularity in most cases). We will want to explore possibilities for improving lock granularity in this code in the future. Submitted by: sam Sponsored by: FreeBSD Foundatiuon Obtained from: BSD/OS 5 snapshot provided by BSDi
2004-06-10 21:34:38 +00:00
sa = (struct sockaddr *) unp->unp_addr;
Change uipc_sockaddr so that a sockaddr_un without a path is returned nam for an unbound socket instead of leaving nam untouched in that case. This way, the getsockname() output can be used to determine the address family of such sockets (AF_LOCAL). Reviewed by: iedowse Approved by: rwatson
2001-04-24 19:09:23 +00:00
else
Introduce a subsystem lock around UNIX domain sockets in order to protect global and allocated variables. This strategy is derived from work originally developed by BSDi for BSD/OS, and applied to FreeBSD by Sam Leffler: - Add unp_mtx, a global mutex which will protect all UNIX domain socket related variables, structures, etc. - Add UNP_LOCK(), UNP_UNLOCK(), UNP_LOCK_ASSERT() macros. - Acquire unp_mtx on entering most UNIX domain socket code, drop/re-acquire around calls into VFS, and release it on return. - Avoid performing sodupsockaddr() while holding the mutex, so in general move to allocating storage before acquiring the mutex to copy the data. - Make a stack copy of the xucred rather than copying out while holding unp_mtx. Copy the peer credential out after releasing the mutex. - Add additional assertions of vnode locks following VOP_CREATE(). A few notes: - Use of an sx lock for the file list mutex may cause problems with regard to unp_mtx when garbage collection passed file descriptors. - The locking in unp_pcblist() for sysctl monitoring is correct subject to the unpcb zone not returning memory for reuse by other subsystems (consistent with similar existing concerns). - Sam's version of this change, as with the BSD/OS version, made use of both a global lock and per-unpcb locks. However, in practice, the global lock covered all accesses, so I have simplified out the unpcb locks in the interest of getting this merged faster (reducing the overhead but not sacrificing granularity in most cases). We will want to explore possibilities for improving lock granularity in this code in the future. Submitted by: sam Sponsored by: FreeBSD Foundatiuon Obtained from: BSD/OS 5 snapshot provided by BSDi
2004-06-10 21:34:38 +00:00
sa = &sun_noname;
bcopy(sa, *nam, sa->sa_len);
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
UNP_PCB_UNLOCK(unp);
Mechanical whitespace cleanup; parenthesize return values; other minor style nits.
2004-01-11 19:48:19 +00:00
return (0);
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
}
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
struct pr_usrreqs uipc_usrreqs = {
Initialize struct pr_userreqs in new/sparse style and fill in common default elements in net_init_domain(). This makes it possible to grep these structures and see any bogosities.
2004-11-08 14:44:54 +00:00
.pru_abort = uipc_abort,
.pru_accept = uipc_accept,
.pru_attach = uipc_attach,
.pru_bind = uipc_bind,
.pru_connect = uipc_connect,
.pru_connect2 = uipc_connect2,
.pru_detach = uipc_detach,
.pru_disconnect = uipc_disconnect,
.pru_listen = uipc_listen,
.pru_peeraddr = uipc_peeraddr,
.pru_rcvd = uipc_rcvd,
.pru_send = uipc_send,
.pru_sense = uipc_sense,
.pru_shutdown = uipc_shutdown,
.pru_sockaddr = uipc_sockaddr,
Change semantics of socket close and detach. Add a new protocol switch function, pru_close, to notify protocols that the file descriptor or other consumer of a socket is closing the socket. pru_abort is now a notification of close also, and no longer detaches. pru_detach is no longer used to notify of close, and will be called during socket tear-down by sofree() when all references to a socket evaporate after an earlier call to abort or close the socket. This means detach is now an unconditional teardown of a socket, whereas previously sockets could persist after detach of the protocol retained a reference. This faciliates sharing mutexes between layers of the network stack as the mutex is required during the checking and removal of references at the head of sofree(). With this change, pru_detach can now assume that the mutex will no longer be required by the socket layer after completion, whereas before this was not necessarily true. Reviewed by: gnn
2006-07-21 17:11:15 +00:00
.pru_close = uipc_close,
The long-awaited mega-massive-network-code- cleanup. Part I. This commit includes the following changes: 1) Old-style (pr_usrreq()) protocols are no longer supported, the compatibility glue for them is deleted, and the kernel will panic on boot if any are compiled in. 2) Certain protocol entry points are modified to take a process structure, so they they can easily tell whether or not it is possible to sleep, and also to access credentials. 3) SS_PRIV is no more, and with it goes the SO_PRIVSTATE setsockopt() call. Protocols should use the process pointer they are now passed. 4) The PF_LOCAL and PF_ROUTE families have been updated to use the new style, as has the `raw' skeleton family. 5) PF_LOCAL sockets now obey the process's umask when creating a socket in the filesystem. As a result, LINT is now broken. I'm hoping that some enterprising hacker with a bit more time will either make the broken bits work (should be easy for netipx) or dike them out.
1997-04-27 20:01:29 +00:00
};
Implement a LOCAL_PEERCRED socket option which returns a `struct xucred` with the credentials of the connected peer. Obviously this only works (and makes sense) on SOCK_STREAM sockets. This works for both the connect(2) and listen(2) callers. There is precise documentation of the semantics in unix(4). Reviewed by: dwmalone (eyeballed)
2001-08-17 22:01:18 +00:00
int
style(9)-ize function headers, remove use of 'register'. MFC after: 3 days
2005-02-20 23:22:13 +00:00
uipc_ctloutput(struct socket *so, struct sockopt *sopt)
Implement a LOCAL_PEERCRED socket option which returns a `struct xucred` with the credentials of the connected peer. Obviously this only works (and makes sense) on SOCK_STREAM sockets. This works for both the connect(2) and listen(2) callers. There is precise documentation of the semantics in unix(4). Reviewed by: dwmalone (eyeballed)
2001-08-17 22:01:18 +00:00
{
Always acquire the UNIX domain socket subsystem lock (UNP lock) before dereferencing sotounpcb() and checking its value, as so_pcb is protected by protocol locking, not subsystem locking. This prevents races during close() by one thread and use of ths socket in another. unp_bind() now assert the UNP lock, and uipc_bind() now acquires the lock around calls to unp_bind().
2004-08-16 04:41:03 +00:00
struct unpcb *unp;
Introduce a subsystem lock around UNIX domain sockets in order to protect global and allocated variables. This strategy is derived from work originally developed by BSDi for BSD/OS, and applied to FreeBSD by Sam Leffler: - Add unp_mtx, a global mutex which will protect all UNIX domain socket related variables, structures, etc. - Add UNP_LOCK(), UNP_UNLOCK(), UNP_LOCK_ASSERT() macros. - Acquire unp_mtx on entering most UNIX domain socket code, drop/re-acquire around calls into VFS, and release it on return. - Avoid performing sodupsockaddr() while holding the mutex, so in general move to allocating storage before acquiring the mutex to copy the data. - Make a stack copy of the xucred rather than copying out while holding unp_mtx. Copy the peer credential out after releasing the mutex. - Add additional assertions of vnode locks following VOP_CREATE(). A few notes: - Use of an sx lock for the file list mutex may cause problems with regard to unp_mtx when garbage collection passed file descriptors. - The locking in unp_pcblist() for sysctl monitoring is correct subject to the unpcb zone not returning memory for reuse by other subsystems (consistent with similar existing concerns). - Sam's version of this change, as with the BSD/OS version, made use of both a global lock and per-unpcb locks. However, in practice, the global lock covered all accesses, so I have simplified out the unpcb locks in the interest of getting this merged faster (reducing the overhead but not sacrificing granularity in most cases). We will want to explore possibilities for improving lock granularity in this code in the future. Submitted by: sam Sponsored by: FreeBSD Foundatiuon Obtained from: BSD/OS 5 snapshot provided by BSDi
2004-06-10 21:34:38 +00:00
struct xucred xu;
Implement unix(4) socket options LOCAL_CREDS and LOCAL_CONNWAIT. - Add unp_addsockcred() (for LOCAL_CREDS). - Add an argument to unp_connect2() to differentiate between PRU_CONNECT and PRU_CONNECT2. (for LOCAL_CONNWAIT) Obtained from: NetBSD (with some changes)
2005-04-13 00:01:46 +00:00
int error, optval;
Check sopt_level in uipc_ctloutput() and return early if it is non-zero. This prevents unintended consequnces when an application calls things like setsockopt(x, SOL_SOCKET, SO_REUSEADDR, ...) on a Unix domain socket.
2005-04-20 02:57:56 +00:00
if (sopt->sopt_level != 0)
return (EINVAL);
Implement unix(4) socket options LOCAL_CREDS and LOCAL_CONNWAIT. - Add unp_addsockcred() (for LOCAL_CREDS). - Add an argument to unp_connect2() to differentiate between PRU_CONNECT and PRU_CONNECT2. (for LOCAL_CONNWAIT) Obtained from: NetBSD (with some changes)
2005-04-13 00:01:46 +00:00
unp = sotounpcb(so);
Modify UNIX domain sockets to guarantee, and assume, that so_pcb is always defined for an in-use socket. This allows us to eliminate countless tests of whether so_pcb is non-NULL, eliminating dozens of error cases. For now, retain the call to sotryfree() in the uipc_abort() path, but this will eventually move to soabort(). These new assumptions should be largely correct, and will become more so as the socket/pcb reference model is fixed. Removing the notion that so_pcb can be non-NULL is a critical step towards further fine-graining of the UNIX domain socket locking, as the so_pcb reference no longer needs to be protected using locks, instead it is a property of the socket life cycle.
2006-03-17 13:52:57 +00:00
KASSERT(unp != NULL, ("uipc_ctloutput: unp == NULL"));
Implement unix(4) socket options LOCAL_CREDS and LOCAL_CONNWAIT. - Add unp_addsockcred() (for LOCAL_CREDS). - Add an argument to unp_connect2() to differentiate between PRU_CONNECT and PRU_CONNECT2. (for LOCAL_CONNWAIT) Obtained from: NetBSD (with some changes)
2005-04-13 00:01:46 +00:00
error = 0;
Implement a LOCAL_PEERCRED socket option which returns a `struct xucred` with the credentials of the connected peer. Obviously this only works (and makes sense) on SOCK_STREAM sockets. This works for both the connect(2) and listen(2) callers. There is precise documentation of the semantics in unix(4). Reviewed by: dwmalone (eyeballed)
2001-08-17 22:01:18 +00:00
switch (sopt->sopt_dir) {
case SOPT_GET:
switch (sopt->sopt_name) {
case LOCAL_PEERCRED:
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
UNP_PCB_LOCK(unp);
Implement a LOCAL_PEERCRED socket option which returns a `struct xucred` with the credentials of the connected peer. Obviously this only works (and makes sense) on SOCK_STREAM sockets. This works for both the connect(2) and listen(2) callers. There is precise documentation of the semantics in unix(4). Reviewed by: dwmalone (eyeballed)
2001-08-17 22:01:18 +00:00
if (unp->unp_flags & UNP_HAVEPC)
Introduce a subsystem lock around UNIX domain sockets in order to protect global and allocated variables. This strategy is derived from work originally developed by BSDi for BSD/OS, and applied to FreeBSD by Sam Leffler: - Add unp_mtx, a global mutex which will protect all UNIX domain socket related variables, structures, etc. - Add UNP_LOCK(), UNP_UNLOCK(), UNP_LOCK_ASSERT() macros. - Acquire unp_mtx on entering most UNIX domain socket code, drop/re-acquire around calls into VFS, and release it on return. - Avoid performing sodupsockaddr() while holding the mutex, so in general move to allocating storage before acquiring the mutex to copy the data. - Make a stack copy of the xucred rather than copying out while holding unp_mtx. Copy the peer credential out after releasing the mutex. - Add additional assertions of vnode locks following VOP_CREATE(). A few notes: - Use of an sx lock for the file list mutex may cause problems with regard to unp_mtx when garbage collection passed file descriptors. - The locking in unp_pcblist() for sysctl monitoring is correct subject to the unpcb zone not returning memory for reuse by other subsystems (consistent with similar existing concerns). - Sam's version of this change, as with the BSD/OS version, made use of both a global lock and per-unpcb locks. However, in practice, the global lock covered all accesses, so I have simplified out the unpcb locks in the interest of getting this merged faster (reducing the overhead but not sacrificing granularity in most cases). We will want to explore possibilities for improving lock granularity in this code in the future. Submitted by: sam Sponsored by: FreeBSD Foundatiuon Obtained from: BSD/OS 5 snapshot provided by BSDi
2004-06-10 21:34:38 +00:00
xu = unp->unp_peercred;
Implement a LOCAL_PEERCRED socket option which returns a `struct xucred` with the credentials of the connected peer. Obviously this only works (and makes sense) on SOCK_STREAM sockets. This works for both the connect(2) and listen(2) callers. There is precise documentation of the semantics in unix(4). Reviewed by: dwmalone (eyeballed)
2001-08-17 22:01:18 +00:00
else {
if (so->so_type == SOCK_STREAM)
error = ENOTCONN;
else
error = EINVAL;
}
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
UNP_PCB_UNLOCK(unp);
Introduce a subsystem lock around UNIX domain sockets in order to protect global and allocated variables. This strategy is derived from work originally developed by BSDi for BSD/OS, and applied to FreeBSD by Sam Leffler: - Add unp_mtx, a global mutex which will protect all UNIX domain socket related variables, structures, etc. - Add UNP_LOCK(), UNP_UNLOCK(), UNP_LOCK_ASSERT() macros. - Acquire unp_mtx on entering most UNIX domain socket code, drop/re-acquire around calls into VFS, and release it on return. - Avoid performing sodupsockaddr() while holding the mutex, so in general move to allocating storage before acquiring the mutex to copy the data. - Make a stack copy of the xucred rather than copying out while holding unp_mtx. Copy the peer credential out after releasing the mutex. - Add additional assertions of vnode locks following VOP_CREATE(). A few notes: - Use of an sx lock for the file list mutex may cause problems with regard to unp_mtx when garbage collection passed file descriptors. - The locking in unp_pcblist() for sysctl monitoring is correct subject to the unpcb zone not returning memory for reuse by other subsystems (consistent with similar existing concerns). - Sam's version of this change, as with the BSD/OS version, made use of both a global lock and per-unpcb locks. However, in practice, the global lock covered all accesses, so I have simplified out the unpcb locks in the interest of getting this merged faster (reducing the overhead but not sacrificing granularity in most cases). We will want to explore possibilities for improving lock granularity in this code in the future. Submitted by: sam Sponsored by: FreeBSD Foundatiuon Obtained from: BSD/OS 5 snapshot provided by BSDi
2004-06-10 21:34:38 +00:00
if (error == 0)
error = sooptcopyout(sopt, &xu, sizeof(xu));
Implement a LOCAL_PEERCRED socket option which returns a `struct xucred` with the credentials of the connected peer. Obviously this only works (and makes sense) on SOCK_STREAM sockets. This works for both the connect(2) and listen(2) callers. There is precise documentation of the semantics in unix(4). Reviewed by: dwmalone (eyeballed)
2001-08-17 22:01:18 +00:00
break;
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
Implement unix(4) socket options LOCAL_CREDS and LOCAL_CONNWAIT. - Add unp_addsockcred() (for LOCAL_CREDS). - Add an argument to unp_connect2() to differentiate between PRU_CONNECT and PRU_CONNECT2. (for LOCAL_CONNWAIT) Obtained from: NetBSD (with some changes)
2005-04-13 00:01:46 +00:00
case LOCAL_CREDS:
Push UNIX domain socket locking further into uipc_ctloutput() in order to avoid holding the UNIX domain socket subsystem lock over soooptcopyin() and sooptcopyout(). This problem was introduced when LOCAL_CREDS, and LOCAL_CONNWAIT support were added. Reviewed by: mdodd
2007-02-06 14:31:37 +00:00
/* Unocked read. */
Implement unix(4) socket options LOCAL_CREDS and LOCAL_CONNWAIT. - Add unp_addsockcred() (for LOCAL_CREDS). - Add an argument to unp_connect2() to differentiate between PRU_CONNECT and PRU_CONNECT2. (for LOCAL_CONNWAIT) Obtained from: NetBSD (with some changes)
2005-04-13 00:01:46 +00:00
optval = unp->unp_flags & UNP_WANTCRED ? 1 : 0;
error = sooptcopyout(sopt, &optval, sizeof(optval));
break;
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
Implement unix(4) socket options LOCAL_CREDS and LOCAL_CONNWAIT. - Add unp_addsockcred() (for LOCAL_CREDS). - Add an argument to unp_connect2() to differentiate between PRU_CONNECT and PRU_CONNECT2. (for LOCAL_CONNWAIT) Obtained from: NetBSD (with some changes)
2005-04-13 00:01:46 +00:00
case LOCAL_CONNWAIT:
Push UNIX domain socket locking further into uipc_ctloutput() in order to avoid holding the UNIX domain socket subsystem lock over soooptcopyin() and sooptcopyout(). This problem was introduced when LOCAL_CREDS, and LOCAL_CONNWAIT support were added. Reviewed by: mdodd
2007-02-06 14:31:37 +00:00
/* Unocked read. */
Implement unix(4) socket options LOCAL_CREDS and LOCAL_CONNWAIT. - Add unp_addsockcred() (for LOCAL_CREDS). - Add an argument to unp_connect2() to differentiate between PRU_CONNECT and PRU_CONNECT2. (for LOCAL_CONNWAIT) Obtained from: NetBSD (with some changes)
2005-04-13 00:01:46 +00:00
optval = unp->unp_flags & UNP_CONNWAIT ? 1 : 0;
error = sooptcopyout(sopt, &optval, sizeof(optval));
break;
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
Implement a LOCAL_PEERCRED socket option which returns a `struct xucred` with the credentials of the connected peer. Obviously this only works (and makes sense) on SOCK_STREAM sockets. This works for both the connect(2) and listen(2) callers. There is precise documentation of the semantics in unix(4). Reviewed by: dwmalone (eyeballed)
2001-08-17 22:01:18 +00:00
default:
error = EOPNOTSUPP;
break;
}
break;
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
Implement a LOCAL_PEERCRED socket option which returns a `struct xucred` with the credentials of the connected peer. Obviously this only works (and makes sense) on SOCK_STREAM sockets. This works for both the connect(2) and listen(2) callers. There is precise documentation of the semantics in unix(4). Reviewed by: dwmalone (eyeballed)
2001-08-17 22:01:18 +00:00
case SOPT_SET:
Implement unix(4) socket options LOCAL_CREDS and LOCAL_CONNWAIT. - Add unp_addsockcred() (for LOCAL_CREDS). - Add an argument to unp_connect2() to differentiate between PRU_CONNECT and PRU_CONNECT2. (for LOCAL_CONNWAIT) Obtained from: NetBSD (with some changes)
2005-04-13 00:01:46 +00:00
switch (sopt->sopt_name) {
case LOCAL_CREDS:
case LOCAL_CONNWAIT:
error = sooptcopyin(sopt, &optval, sizeof(optval),
sizeof(optval));
if (error)
break;
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
#define OPTSET(bit) do { \
UNP_PCB_LOCK(unp); \
if (optval) \
unp->unp_flags |= bit; \
else \
unp->unp_flags &= ~bit; \
UNP_PCB_UNLOCK(unp); \
} while (0)
Implement unix(4) socket options LOCAL_CREDS and LOCAL_CONNWAIT. - Add unp_addsockcred() (for LOCAL_CREDS). - Add an argument to unp_connect2() to differentiate between PRU_CONNECT and PRU_CONNECT2. (for LOCAL_CONNWAIT) Obtained from: NetBSD (with some changes)
2005-04-13 00:01:46 +00:00
switch (sopt->sopt_name) {
case LOCAL_CREDS:
OPTSET(UNP_WANTCRED);
break;
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
Implement unix(4) socket options LOCAL_CREDS and LOCAL_CONNWAIT. - Add unp_addsockcred() (for LOCAL_CREDS). - Add an argument to unp_connect2() to differentiate between PRU_CONNECT and PRU_CONNECT2. (for LOCAL_CONNWAIT) Obtained from: NetBSD (with some changes)
2005-04-13 00:01:46 +00:00
case LOCAL_CONNWAIT:
OPTSET(UNP_CONNWAIT);
break;
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
Implement unix(4) socket options LOCAL_CREDS and LOCAL_CONNWAIT. - Add unp_addsockcred() (for LOCAL_CREDS). - Add an argument to unp_connect2() to differentiate between PRU_CONNECT and PRU_CONNECT2. (for LOCAL_CONNWAIT) Obtained from: NetBSD (with some changes)
2005-04-13 00:01:46 +00:00
default:
break;
}
break;
#undef OPTSET
default:
error = ENOPROTOOPT;
break;
}
Add missing break. Found by: marcus
2005-04-25 00:48:04 +00:00
break;
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
Implement a LOCAL_PEERCRED socket option which returns a `struct xucred` with the credentials of the connected peer. Obviously this only works (and makes sense) on SOCK_STREAM sockets. This works for both the connect(2) and listen(2) callers. There is precise documentation of the semantics in unix(4). Reviewed by: dwmalone (eyeballed)
2001-08-17 22:01:18 +00:00
default:
error = EOPNOTSUPP;
break;
}
return (error);
}
Mechanical whitespace cleanup; parenthesize return values; other minor style nits.
2004-01-11 19:48:19 +00:00
Another mega commit to staticize things.
1995-12-14 09:55:16 +00:00
static int
style(9)-ize function headers, remove use of 'register'. MFC after: 3 days
2005-02-20 23:22:13 +00:00
unp_connect(struct socket *so, struct sockaddr *nam, struct thread *td)
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
{
style(9)-ize function headers, remove use of 'register'. MFC after: 3 days
2005-02-20 23:22:13 +00:00
struct sockaddr_un *soun = (struct sockaddr_un *)nam;
struct vnode *vp;
struct socket *so2, *so3;
After completing a name lookup for a target UNIX domain socket to connect to, re-check that the local UNIX domain socket hasn't been closed while we slept, and if so, return EINVAL. This affects the system running both with and without Giant over the network stack, and recent ULE changes appear to cause it to trigger more frequently than previously under load. While here, improve catching of possibly closed UNIX domain sockets in one or two additional circumstances. I have a much larger set of related changes in Perforce, but they require more testing before they can be merged. One debugging printf is left in place to indicate when such a race takes place: this is typically triggered by a buggy application that simultaenously connect()'s and close()'s a UNIX domain socket file descriptor. I'll remove this at some point in the future, but am interested in seeing how frequently this is reported. In the case of Martin's reported problem, it appears to be a result of a non-thread safe syslog() implementation in the C library, which does not synchronize access to its logging file descriptor. Reported by: mbr
2004-08-14 03:43:49 +00:00
struct unpcb *unp, *unp2, *unp3;
Fix all areas of the system (or at least all those in LINT) to avoid storing socket addresses in mbufs. (Socket buffers are the one exception.) A number of kernel APIs needed to get fixed in order to make this happen. Also, fix three protocol families which kept PCBs in mbufs to not malloc them instead. Delete some old compatibility cruft while we're at it, and add some new routines in the in_cksum family.
1997-08-16 19:16:27 +00:00
int error, len;
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
struct nameidata nd;
Fix all areas of the system (or at least all those in LINT) to avoid storing socket addresses in mbufs. (Socket buffers are the one exception.) A number of kernel APIs needed to get fixed in order to make this happen. Also, fix three protocol families which kept PCBs in mbufs to not malloc them instead. Delete some old compatibility cruft while we're at it, and add some new routines in the in_cksum family.
1997-08-16 19:16:27 +00:00
char buf[SOCK_MAXADDRLEN];
Introduce a subsystem lock around UNIX domain sockets in order to protect global and allocated variables. This strategy is derived from work originally developed by BSDi for BSD/OS, and applied to FreeBSD by Sam Leffler: - Add unp_mtx, a global mutex which will protect all UNIX domain socket related variables, structures, etc. - Add UNP_LOCK(), UNP_UNLOCK(), UNP_LOCK_ASSERT() macros. - Acquire unp_mtx on entering most UNIX domain socket code, drop/re-acquire around calls into VFS, and release it on return. - Avoid performing sodupsockaddr() while holding the mutex, so in general move to allocating storage before acquiring the mutex to copy the data. - Make a stack copy of the xucred rather than copying out while holding unp_mtx. Copy the peer credential out after releasing the mutex. - Add additional assertions of vnode locks following VOP_CREATE(). A few notes: - Use of an sx lock for the file list mutex may cause problems with regard to unp_mtx when garbage collection passed file descriptors. - The locking in unp_pcblist() for sysctl monitoring is correct subject to the unpcb zone not returning memory for reuse by other subsystems (consistent with similar existing concerns). - Sam's version of this change, as with the BSD/OS version, made use of both a global lock and per-unpcb locks. However, in practice, the global lock covered all accesses, so I have simplified out the unpcb locks in the interest of getting this merged faster (reducing the overhead but not sacrificing granularity in most cases). We will want to explore possibilities for improving lock granularity in this code in the future. Submitted by: sam Sponsored by: FreeBSD Foundatiuon Obtained from: BSD/OS 5 snapshot provided by BSDi
2004-06-10 21:34:38 +00:00
struct sockaddr *sa;
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
UNP_GLOBAL_WLOCK_ASSERT();
UNP_GLOBAL_WUNLOCK();
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
Modify UNIX domain sockets to guarantee, and assume, that so_pcb is always defined for an in-use socket. This allows us to eliminate countless tests of whether so_pcb is non-NULL, eliminating dozens of error cases. For now, retain the call to sotryfree() in the uipc_abort() path, but this will eventually move to soabort(). These new assumptions should be largely correct, and will become more so as the socket/pcb reference model is fixed. Removing the notion that so_pcb can be non-NULL is a critical step towards further fine-graining of the UNIX domain socket locking, as the so_pcb reference no longer needs to be protected using locks, instead it is a property of the socket life cycle.
2006-03-17 13:52:57 +00:00
unp = sotounpcb(so);
KASSERT(unp != NULL, ("unp_connect: unp == NULL"));
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
Fix all areas of the system (or at least all those in LINT) to avoid storing socket addresses in mbufs. (Socket buffers are the one exception.) A number of kernel APIs needed to get fixed in order to make this happen. Also, fix three protocol families which kept PCBs in mbufs to not malloc them instead. Delete some old compatibility cruft while we're at it, and add some new routines in the in_cksum family.
1997-08-16 19:16:27 +00:00
len = nam->sa_len - offsetof(struct sockaddr_un, sun_path);
if (len <= 0)
Mechanical whitespace cleanup; parenthesize return values; other minor style nits.
2004-01-11 19:48:19 +00:00
return (EINVAL);
- Allocate only enough space for a temporary buffer to hold the path including the terminating NUL character from `struct sockaddr_un' rather than SOCK_MAXADDRLEN bytes. - Use strlcpy() instead of strncpy() to copy strings.
2002-10-17 15:52:42 +00:00
strlcpy(buf, soun->sun_path, len + 1);
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
UNP_PCB_LOCK(unp);
Add two new unpcb flags, UNP_BINDING and UNP_CONNECTING, which will be used to mark UNIX domain sockets as being in the process of binding or connecting. Use these to prevent simultaneous bind or connect operations by multiple threads or processes on the same socket at the same time, which closes race conditions present in the UNIX domain socket implementation since inception.
2006-07-23 12:01:14 +00:00
if (unp->unp_flags & UNP_CONNECTING) {
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
UNP_PCB_UNLOCK(unp);
Add two new unpcb flags, UNP_BINDING and UNP_CONNECTING, which will be used to mark UNIX domain sockets as being in the process of binding or connecting. Use these to prevent simultaneous bind or connect operations by multiple threads or processes on the same socket at the same time, which closes race conditions present in the UNIX domain socket implementation since inception.
2006-07-23 12:01:14 +00:00
return (EALREADY);
}
Set UNP_CONNECTING when committing to moving ahead in unp_connect(). This logic was lost when merging the remainder of these changes in 1.178.
2007-02-13 21:00:57 +00:00
unp->unp_flags |= UNP_CONNECTING;
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
UNP_PCB_UNLOCK(unp);
Introduce a subsystem lock around UNIX domain sockets in order to protect global and allocated variables. This strategy is derived from work originally developed by BSDi for BSD/OS, and applied to FreeBSD by Sam Leffler: - Add unp_mtx, a global mutex which will protect all UNIX domain socket related variables, structures, etc. - Add UNP_LOCK(), UNP_UNLOCK(), UNP_LOCK_ASSERT() macros. - Acquire unp_mtx on entering most UNIX domain socket code, drop/re-acquire around calls into VFS, and release it on return. - Avoid performing sodupsockaddr() while holding the mutex, so in general move to allocating storage before acquiring the mutex to copy the data. - Make a stack copy of the xucred rather than copying out while holding unp_mtx. Copy the peer credential out after releasing the mutex. - Add additional assertions of vnode locks following VOP_CREATE(). A few notes: - Use of an sx lock for the file list mutex may cause problems with regard to unp_mtx when garbage collection passed file descriptors. - The locking in unp_pcblist() for sysctl monitoring is correct subject to the unpcb zone not returning memory for reuse by other subsystems (consistent with similar existing concerns). - Sam's version of this change, as with the BSD/OS version, made use of both a global lock and per-unpcb locks. However, in practice, the global lock covered all accesses, so I have simplified out the unpcb locks in the interest of getting this merged faster (reducing the overhead but not sacrificing granularity in most cases). We will want to explore possibilities for improving lock granularity in this code in the future. Submitted by: sam Sponsored by: FreeBSD Foundatiuon Obtained from: BSD/OS 5 snapshot provided by BSDi
2004-06-10 21:34:38 +00:00
sa = malloc(sizeof(struct sockaddr_un), M_SONAME, M_WAITOK);
mtx_lock(&Giant);
KSE Milestone 2 Note ALL MODULES MUST BE RECOMPILED make the kernel aware that there are smaller units of scheduling than the process. (but only allow one thread per process at this time). This is functionally equivalent to teh previousl -current except that there is a thread associated with each process. Sorry john! (your next MFC will be a doosie!) Reviewed by: peter@freebsd.org, dillon@freebsd.org X-MFC after: ha ha ha ha
2001-09-12 08:38:13 +00:00
NDINIT(&nd, LOOKUP, FOLLOW | LOCKLEAF, UIO_SYSSPACE, buf, td);
All of this is cosmetic. prototypes, #includes, printfs and so on. Makes GCC a lot more silent.
1994-10-02 17:35:40 +00:00
error = namei(&nd);
if (error)
Introduce a subsystem lock around UNIX domain sockets in order to protect global and allocated variables. This strategy is derived from work originally developed by BSDi for BSD/OS, and applied to FreeBSD by Sam Leffler: - Add unp_mtx, a global mutex which will protect all UNIX domain socket related variables, structures, etc. - Add UNP_LOCK(), UNP_UNLOCK(), UNP_LOCK_ASSERT() macros. - Acquire unp_mtx on entering most UNIX domain socket code, drop/re-acquire around calls into VFS, and release it on return. - Avoid performing sodupsockaddr() while holding the mutex, so in general move to allocating storage before acquiring the mutex to copy the data. - Make a stack copy of the xucred rather than copying out while holding unp_mtx. Copy the peer credential out after releasing the mutex. - Add additional assertions of vnode locks following VOP_CREATE(). A few notes: - Use of an sx lock for the file list mutex may cause problems with regard to unp_mtx when garbage collection passed file descriptors. - The locking in unp_pcblist() for sysctl monitoring is correct subject to the unpcb zone not returning memory for reuse by other subsystems (consistent with similar existing concerns). - Sam's version of this change, as with the BSD/OS version, made use of both a global lock and per-unpcb locks. However, in practice, the global lock covered all accesses, so I have simplified out the unpcb locks in the interest of getting this merged faster (reducing the overhead but not sacrificing granularity in most cases). We will want to explore possibilities for improving lock granularity in this code in the future. Submitted by: sam Sponsored by: FreeBSD Foundatiuon Obtained from: BSD/OS 5 snapshot provided by BSDi
2004-06-10 21:34:38 +00:00
vp = NULL;
else
vp = nd.ni_vp;
ASSERT_VOP_LOCKED(vp, "unp_connect");
Introduce NDFREE (and remove VOP_ABORTOP)
1999-12-15 23:02:35 +00:00
NDFREE(&nd, NDF_ONLY_PNBUF);
Introduce a subsystem lock around UNIX domain sockets in order to protect global and allocated variables. This strategy is derived from work originally developed by BSDi for BSD/OS, and applied to FreeBSD by Sam Leffler: - Add unp_mtx, a global mutex which will protect all UNIX domain socket related variables, structures, etc. - Add UNP_LOCK(), UNP_UNLOCK(), UNP_LOCK_ASSERT() macros. - Acquire unp_mtx on entering most UNIX domain socket code, drop/re-acquire around calls into VFS, and release it on return. - Avoid performing sodupsockaddr() while holding the mutex, so in general move to allocating storage before acquiring the mutex to copy the data. - Make a stack copy of the xucred rather than copying out while holding unp_mtx. Copy the peer credential out after releasing the mutex. - Add additional assertions of vnode locks following VOP_CREATE(). A few notes: - Use of an sx lock for the file list mutex may cause problems with regard to unp_mtx when garbage collection passed file descriptors. - The locking in unp_pcblist() for sysctl monitoring is correct subject to the unpcb zone not returning memory for reuse by other subsystems (consistent with similar existing concerns). - Sam's version of this change, as with the BSD/OS version, made use of both a global lock and per-unpcb locks. However, in practice, the global lock covered all accesses, so I have simplified out the unpcb locks in the interest of getting this merged faster (reducing the overhead but not sacrificing granularity in most cases). We will want to explore possibilities for improving lock granularity in this code in the future. Submitted by: sam Sponsored by: FreeBSD Foundatiuon Obtained from: BSD/OS 5 snapshot provided by BSDi
2004-06-10 21:34:38 +00:00
if (error)
goto bad;
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
if (vp->v_type != VSOCK) {
error = ENOTSOCK;
goto bad;
}
Add an additional MAC check to the UNIX domain socket connect path: check that the subject has read/write access to the vnode using the vnode MAC check. MFC after: 3 weeks Submitted by: Spencer Minear <spencer_minear at securecomputing dot com> Obtained from: TrustedBSD Project
2007-02-22 09:37:44 +00:00
#ifdef MAC
error = mac_check_vnode_open(td->td_ucred, vp, VWRITE | VREAD);
if (error)
goto bad;
#endif
Simple p_ucred -> td_ucred changes to start using the per-thread ucred reference.
2002-02-27 18:32:23 +00:00
error = VOP_ACCESS(vp, VWRITE, td->td_ucred, td);
All of this is cosmetic. prototypes, #includes, printfs and so on. Makes GCC a lot more silent.
1994-10-02 17:35:40 +00:00
if (error)
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
goto bad;
Drop Giant and acquire the UNIX domain socket subsystem lock a bit earlier in unp_connect() so that vp->v_socket can't change between our copying its value to a local variable and later use of that variable. This may have been responsible for a panic during shutdown that I experienced where simultaneous closing of a listen socket by rpcbind and a new connection being made to rpcbind by mountd.
2004-07-18 01:29:43 +00:00
mtx_unlock(&Giant);
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
After completing a name lookup for a target UNIX domain socket to connect to, re-check that the local UNIX domain socket hasn't been closed while we slept, and if so, return EINVAL. This affects the system running both with and without Giant over the network stack, and recent ULE changes appear to cause it to trigger more frequently than previously under load. While here, improve catching of possibly closed UNIX domain sockets in one or two additional circumstances. I have a much larger set of related changes in Perforce, but they require more testing before they can be merged. One debugging printf is left in place to indicate when such a race takes place: this is typically triggered by a buggy application that simultaenously connect()'s and close()'s a UNIX domain socket file descriptor. I'll remove this at some point in the future, but am interested in seeing how frequently this is reported. In the case of Martin's reported problem, it appears to be a result of a non-thread safe syslog() implementation in the C library, which does not synchronize access to its logging file descriptor. Reported by: mbr
2004-08-14 03:43:49 +00:00
unp = sotounpcb(so);
Modify UNIX domain sockets to guarantee, and assume, that so_pcb is always defined for an in-use socket. This allows us to eliminate countless tests of whether so_pcb is non-NULL, eliminating dozens of error cases. For now, retain the call to sotryfree() in the uipc_abort() path, but this will eventually move to soabort(). These new assumptions should be largely correct, and will become more so as the socket/pcb reference model is fixed. Removing the notion that so_pcb can be non-NULL is a critical step towards further fine-graining of the UNIX domain socket locking, as the so_pcb reference no longer needs to be protected using locks, instead it is a property of the socket life cycle.
2006-03-17 13:52:57 +00:00
KASSERT(unp != NULL, ("unp_connect: unp == NULL"));
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
/*
* Lock global lock for two reasons: make sure v_socket is stable,
* and to protect simultaneous locking of multiple pcbs.
*/
UNP_GLOBAL_WLOCK();
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
so2 = vp->v_socket;
Prefer NULL to 0 when testing and assigning pointer values.
2004-03-30 02:16:25 +00:00
if (so2 == NULL) {
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
error = ECONNREFUSED;
Drop Giant and acquire the UNIX domain socket subsystem lock a bit earlier in unp_connect() so that vp->v_socket can't change between our copying its value to a local variable and later use of that variable. This may have been responsible for a panic during shutdown that I experienced where simultaneous closing of a listen socket by rpcbind and a new connection being made to rpcbind by mountd.
2004-07-18 01:29:43 +00:00
goto bad2;
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
}
if (so->so_type != so2->so_type) {
error = EPROTOTYPE;
Drop Giant and acquire the UNIX domain socket subsystem lock a bit earlier in unp_connect() so that vp->v_socket can't change between our copying its value to a local variable and later use of that variable. This may have been responsible for a panic during shutdown that I experienced where simultaneous closing of a listen socket by rpcbind and a new connection being made to rpcbind by mountd.
2004-07-18 01:29:43 +00:00
goto bad2;
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
}
if (so->so_proto->pr_flags & PR_CONNREQUIRED) {
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
if (so2->so_options & SO_ACCEPTCONN) {
/*
* We can't drop the global lock here or 'so2' may
* become invalid, meaning that we will later recurse
* back into the UNIX domain socket code while
* holding the global lock.
*/
Introduce a subsystem lock around UNIX domain sockets in order to protect global and allocated variables. This strategy is derived from work originally developed by BSDi for BSD/OS, and applied to FreeBSD by Sam Leffler: - Add unp_mtx, a global mutex which will protect all UNIX domain socket related variables, structures, etc. - Add UNP_LOCK(), UNP_UNLOCK(), UNP_LOCK_ASSERT() macros. - Acquire unp_mtx on entering most UNIX domain socket code, drop/re-acquire around calls into VFS, and release it on return. - Avoid performing sodupsockaddr() while holding the mutex, so in general move to allocating storage before acquiring the mutex to copy the data. - Make a stack copy of the xucred rather than copying out while holding unp_mtx. Copy the peer credential out after releasing the mutex. - Add additional assertions of vnode locks following VOP_CREATE(). A few notes: - Use of an sx lock for the file list mutex may cause problems with regard to unp_mtx when garbage collection passed file descriptors. - The locking in unp_pcblist() for sysctl monitoring is correct subject to the unpcb zone not returning memory for reuse by other subsystems (consistent with similar existing concerns). - Sam's version of this change, as with the BSD/OS version, made use of both a global lock and per-unpcb locks. However, in practice, the global lock covered all accesses, so I have simplified out the unpcb locks in the interest of getting this merged faster (reducing the overhead but not sacrificing granularity in most cases). We will want to explore possibilities for improving lock granularity in this code in the future. Submitted by: sam Sponsored by: FreeBSD Foundatiuon Obtained from: BSD/OS 5 snapshot provided by BSDi
2004-06-10 21:34:38 +00:00
so3 = sonewconn(so2, 0);
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
} else
Introduce a subsystem lock around UNIX domain sockets in order to protect global and allocated variables. This strategy is derived from work originally developed by BSDi for BSD/OS, and applied to FreeBSD by Sam Leffler: - Add unp_mtx, a global mutex which will protect all UNIX domain socket related variables, structures, etc. - Add UNP_LOCK(), UNP_UNLOCK(), UNP_LOCK_ASSERT() macros. - Acquire unp_mtx on entering most UNIX domain socket code, drop/re-acquire around calls into VFS, and release it on return. - Avoid performing sodupsockaddr() while holding the mutex, so in general move to allocating storage before acquiring the mutex to copy the data. - Make a stack copy of the xucred rather than copying out while holding unp_mtx. Copy the peer credential out after releasing the mutex. - Add additional assertions of vnode locks following VOP_CREATE(). A few notes: - Use of an sx lock for the file list mutex may cause problems with regard to unp_mtx when garbage collection passed file descriptors. - The locking in unp_pcblist() for sysctl monitoring is correct subject to the unpcb zone not returning memory for reuse by other subsystems (consistent with similar existing concerns). - Sam's version of this change, as with the BSD/OS version, made use of both a global lock and per-unpcb locks. However, in practice, the global lock covered all accesses, so I have simplified out the unpcb locks in the interest of getting this merged faster (reducing the overhead but not sacrificing granularity in most cases). We will want to explore possibilities for improving lock granularity in this code in the future. Submitted by: sam Sponsored by: FreeBSD Foundatiuon Obtained from: BSD/OS 5 snapshot provided by BSDi
2004-06-10 21:34:38 +00:00
so3 = NULL;
if (so3 == NULL) {
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
error = ECONNREFUSED;
Introduce a subsystem lock around UNIX domain sockets in order to protect global and allocated variables. This strategy is derived from work originally developed by BSDi for BSD/OS, and applied to FreeBSD by Sam Leffler: - Add unp_mtx, a global mutex which will protect all UNIX domain socket related variables, structures, etc. - Add UNP_LOCK(), UNP_UNLOCK(), UNP_LOCK_ASSERT() macros. - Acquire unp_mtx on entering most UNIX domain socket code, drop/re-acquire around calls into VFS, and release it on return. - Avoid performing sodupsockaddr() while holding the mutex, so in general move to allocating storage before acquiring the mutex to copy the data. - Make a stack copy of the xucred rather than copying out while holding unp_mtx. Copy the peer credential out after releasing the mutex. - Add additional assertions of vnode locks following VOP_CREATE(). A few notes: - Use of an sx lock for the file list mutex may cause problems with regard to unp_mtx when garbage collection passed file descriptors. - The locking in unp_pcblist() for sysctl monitoring is correct subject to the unpcb zone not returning memory for reuse by other subsystems (consistent with similar existing concerns). - Sam's version of this change, as with the BSD/OS version, made use of both a global lock and per-unpcb locks. However, in practice, the global lock covered all accesses, so I have simplified out the unpcb locks in the interest of getting this merged faster (reducing the overhead but not sacrificing granularity in most cases). We will want to explore possibilities for improving lock granularity in this code in the future. Submitted by: sam Sponsored by: FreeBSD Foundatiuon Obtained from: BSD/OS 5 snapshot provided by BSDi
2004-06-10 21:34:38 +00:00
goto bad2;
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
}
Implement a LOCAL_PEERCRED socket option which returns a `struct xucred` with the credentials of the connected peer. Obviously this only works (and makes sense) on SOCK_STREAM sockets. This works for both the connect(2) and listen(2) callers. There is precise documentation of the semantics in unix(4). Reviewed by: dwmalone (eyeballed)
2001-08-17 22:01:18 +00:00
unp = sotounpcb(so);
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
unp2 = sotounpcb(so2);
unp3 = sotounpcb(so3);
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
UNP_PCB_LOCK(unp);
UNP_PCB_LOCK(unp2);
UNP_PCB_LOCK(unp3);
Introduce a subsystem lock around UNIX domain sockets in order to protect global and allocated variables. This strategy is derived from work originally developed by BSDi for BSD/OS, and applied to FreeBSD by Sam Leffler: - Add unp_mtx, a global mutex which will protect all UNIX domain socket related variables, structures, etc. - Add UNP_LOCK(), UNP_UNLOCK(), UNP_LOCK_ASSERT() macros. - Acquire unp_mtx on entering most UNIX domain socket code, drop/re-acquire around calls into VFS, and release it on return. - Avoid performing sodupsockaddr() while holding the mutex, so in general move to allocating storage before acquiring the mutex to copy the data. - Make a stack copy of the xucred rather than copying out while holding unp_mtx. Copy the peer credential out after releasing the mutex. - Add additional assertions of vnode locks following VOP_CREATE(). A few notes: - Use of an sx lock for the file list mutex may cause problems with regard to unp_mtx when garbage collection passed file descriptors. - The locking in unp_pcblist() for sysctl monitoring is correct subject to the unpcb zone not returning memory for reuse by other subsystems (consistent with similar existing concerns). - Sam's version of this change, as with the BSD/OS version, made use of both a global lock and per-unpcb locks. However, in practice, the global lock covered all accesses, so I have simplified out the unpcb locks in the interest of getting this merged faster (reducing the overhead but not sacrificing granularity in most cases). We will want to explore possibilities for improving lock granularity in this code in the future. Submitted by: sam Sponsored by: FreeBSD Foundatiuon Obtained from: BSD/OS 5 snapshot provided by BSDi
2004-06-10 21:34:38 +00:00
if (unp2->unp_addr != NULL) {
bcopy(unp2->unp_addr, sa, unp2->unp_addr->sun_len);
unp3->unp_addr = (struct sockaddr_un *) sa;
sa = NULL;
}
Implement a LOCAL_PEERCRED socket option which returns a `struct xucred` with the credentials of the connected peer. Obviously this only works (and makes sense) on SOCK_STREAM sockets. This works for both the connect(2) and listen(2) callers. There is precise documentation of the semantics in unix(4). Reviewed by: dwmalone (eyeballed)
2001-08-17 22:01:18 +00:00
/*
* unp_peercred management:
*
Re-wrap and other minor formatting and punctuation fixes for UNIX domain socket comments.
2006-07-22 17:24:55 +00:00
* The connecter's (client's) credentials are copied from its
* process structure at the time of connect() (which is now).
Implement a LOCAL_PEERCRED socket option which returns a `struct xucred` with the credentials of the connected peer. Obviously this only works (and makes sense) on SOCK_STREAM sockets. This works for both the connect(2) and listen(2) callers. There is precise documentation of the semantics in unix(4). Reviewed by: dwmalone (eyeballed)
2001-08-17 22:01:18 +00:00
*/
Simple p_ucred -> td_ucred changes to start using the per-thread ucred reference.
2002-02-27 18:32:23 +00:00
cru2x(td->td_ucred, &unp3->unp_peercred);
Implement a LOCAL_PEERCRED socket option which returns a `struct xucred` with the credentials of the connected peer. Obviously this only works (and makes sense) on SOCK_STREAM sockets. This works for both the connect(2) and listen(2) callers. There is precise documentation of the semantics in unix(4). Reviewed by: dwmalone (eyeballed)
2001-08-17 22:01:18 +00:00
unp3->unp_flags |= UNP_HAVEPC;
/*
Re-wrap and other minor formatting and punctuation fixes for UNIX domain socket comments.
2006-07-22 17:24:55 +00:00
* The receiver's (server's) credentials are copied from the
* unp_peercred member of socket on which the former called
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
* listen(); uipc_listen() cached that process's credentials
Re-wrap and other minor formatting and punctuation fixes for UNIX domain socket comments.
2006-07-22 17:24:55 +00:00
* at that time so we can use them now.
Implement a LOCAL_PEERCRED socket option which returns a `struct xucred` with the credentials of the connected peer. Obviously this only works (and makes sense) on SOCK_STREAM sockets. This works for both the connect(2) and listen(2) callers. There is precise documentation of the semantics in unix(4). Reviewed by: dwmalone (eyeballed)
2001-08-17 22:01:18 +00:00
*/
KASSERT(unp2->unp_flags & UNP_HAVEPCCACHED,
("unp_connect: listener without cached peercred"));
memcpy(&unp->unp_peercred, &unp2->unp_peercred,
sizeof(unp->unp_peercred));
unp->unp_flags |= UNP_HAVEPC;
Inherit LOCAL_CREDS option from listen socket for sockets returned by accept(2). PR: kern/90644 Submitted by: Andrey Simonenko OK'ed by: mdodd Tested by: NetBSD regress/sys/kern/unfdpass/unfdpass.c MFC after: 1 month
2006-04-24 19:09:33 +00:00
if (unp2->unp_flags & UNP_WANTCRED)
unp3->unp_flags |= UNP_WANTCRED;
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
UNP_PCB_UNLOCK(unp3);
UNP_PCB_UNLOCK(unp2);
UNP_PCB_UNLOCK(unp);
Introduce support for Mandatory Access Control and extensible kernel access control. Invoke the necessary MAC entry points to maintain labels on sockets. In particular, invoke entry points during socket allocation and destruction, as well as creation by a process or during an accept-scenario (sonewconn). For UNIX domain sockets, also assign a peer label. As the socket code isn't locked down yet, locking interactions are not yet clear. Various protocol stack socket operations (such as peer label assignment for IPv4) will follow. Obtained from: TrustedBSD Project Sponsored by: DARPA, NAI Labs
2002-07-31 03:03:22 +00:00
#ifdef MAC
Socket MAC labels so_label and so_peerlabel are now protected by SOCK_LOCK(so): - Hold socket lock over calls to MAC entry points reading or manipulating socket labels. - Assert socket lock in MAC entry point implementations. - When externalizing the socket label, first make a thread-local copy while holding the socket lock, then release the socket lock to externalize to userspace.
2004-06-13 02:50:07 +00:00
SOCK_LOCK(so);
Introduce support for Mandatory Access Control and extensible kernel access control. Invoke the necessary MAC entry points to maintain labels on sockets. In particular, invoke entry points during socket allocation and destruction, as well as creation by a process or during an accept-scenario (sonewconn). For UNIX domain sockets, also assign a peer label. As the socket code isn't locked down yet, locking interactions are not yet clear. Various protocol stack socket operations (such as peer label assignment for IPv4) will follow. Obtained from: TrustedBSD Project Sponsored by: DARPA, NAI Labs
2002-07-31 03:03:22 +00:00
mac_set_socket_peer_from_socket(so, so3);
mac_set_socket_peer_from_socket(so3, so);
Socket MAC labels so_label and so_peerlabel are now protected by SOCK_LOCK(so): - Hold socket lock over calls to MAC entry points reading or manipulating socket labels. - Assert socket lock in MAC entry point implementations. - When externalizing the socket label, first make a thread-local copy while holding the socket lock, then release the socket lock to externalize to userspace.
2004-06-13 02:50:07 +00:00
SOCK_UNLOCK(so);
Introduce support for Mandatory Access Control and extensible kernel access control. Invoke the necessary MAC entry points to maintain labels on sockets. In particular, invoke entry points during socket allocation and destruction, as well as creation by a process or during an accept-scenario (sonewconn). For UNIX domain sockets, also assign a peer label. As the socket code isn't locked down yet, locking interactions are not yet clear. Various protocol stack socket operations (such as peer label assignment for IPv4) will follow. Obtained from: TrustedBSD Project Sponsored by: DARPA, NAI Labs
2002-07-31 03:03:22 +00:00
#endif
Implement a LOCAL_PEERCRED socket option which returns a `struct xucred` with the credentials of the connected peer. Obviously this only works (and makes sense) on SOCK_STREAM sockets. This works for both the connect(2) and listen(2) callers. There is precise documentation of the semantics in unix(4). Reviewed by: dwmalone (eyeballed)
2001-08-17 22:01:18 +00:00
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
so2 = so3;
}
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
unp = sotounpcb(so);
KASSERT(unp != NULL, ("unp_connect: unp == NULL"));
unp2 = sotounpcb(so2);
KASSERT(unp2 != NULL, ("unp_connect: unp2 == NULL"));
UNP_PCB_LOCK(unp);
UNP_PCB_LOCK(unp2);
Implement unix(4) socket options LOCAL_CREDS and LOCAL_CONNWAIT. - Add unp_addsockcred() (for LOCAL_CREDS). - Add an argument to unp_connect2() to differentiate between PRU_CONNECT and PRU_CONNECT2. (for LOCAL_CONNWAIT) Obtained from: NetBSD (with some changes)
2005-04-13 00:01:46 +00:00
error = unp_connect2(so, so2, PRU_CONNECT);
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
UNP_PCB_UNLOCK(unp2);
UNP_PCB_UNLOCK(unp);
Introduce a subsystem lock around UNIX domain sockets in order to protect global and allocated variables. This strategy is derived from work originally developed by BSDi for BSD/OS, and applied to FreeBSD by Sam Leffler: - Add unp_mtx, a global mutex which will protect all UNIX domain socket related variables, structures, etc. - Add UNP_LOCK(), UNP_UNLOCK(), UNP_LOCK_ASSERT() macros. - Acquire unp_mtx on entering most UNIX domain socket code, drop/re-acquire around calls into VFS, and release it on return. - Avoid performing sodupsockaddr() while holding the mutex, so in general move to allocating storage before acquiring the mutex to copy the data. - Make a stack copy of the xucred rather than copying out while holding unp_mtx. Copy the peer credential out after releasing the mutex. - Add additional assertions of vnode locks following VOP_CREATE(). A few notes: - Use of an sx lock for the file list mutex may cause problems with regard to unp_mtx when garbage collection passed file descriptors. - The locking in unp_pcblist() for sysctl monitoring is correct subject to the unpcb zone not returning memory for reuse by other subsystems (consistent with similar existing concerns). - Sam's version of this change, as with the BSD/OS version, made use of both a global lock and per-unpcb locks. However, in practice, the global lock covered all accesses, so I have simplified out the unpcb locks in the interest of getting this merged faster (reducing the overhead but not sacrificing granularity in most cases). We will want to explore possibilities for improving lock granularity in this code in the future. Submitted by: sam Sponsored by: FreeBSD Foundatiuon Obtained from: BSD/OS 5 snapshot provided by BSDi
2004-06-10 21:34:38 +00:00
bad2:
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
UNP_GLOBAL_WUNLOCK();
Introduce a subsystem lock around UNIX domain sockets in order to protect global and allocated variables. This strategy is derived from work originally developed by BSDi for BSD/OS, and applied to FreeBSD by Sam Leffler: - Add unp_mtx, a global mutex which will protect all UNIX domain socket related variables, structures, etc. - Add UNP_LOCK(), UNP_UNLOCK(), UNP_LOCK_ASSERT() macros. - Acquire unp_mtx on entering most UNIX domain socket code, drop/re-acquire around calls into VFS, and release it on return. - Avoid performing sodupsockaddr() while holding the mutex, so in general move to allocating storage before acquiring the mutex to copy the data. - Make a stack copy of the xucred rather than copying out while holding unp_mtx. Copy the peer credential out after releasing the mutex. - Add additional assertions of vnode locks following VOP_CREATE(). A few notes: - Use of an sx lock for the file list mutex may cause problems with regard to unp_mtx when garbage collection passed file descriptors. - The locking in unp_pcblist() for sysctl monitoring is correct subject to the unpcb zone not returning memory for reuse by other subsystems (consistent with similar existing concerns). - Sam's version of this change, as with the BSD/OS version, made use of both a global lock and per-unpcb locks. However, in practice, the global lock covered all accesses, so I have simplified out the unpcb locks in the interest of getting this merged faster (reducing the overhead but not sacrificing granularity in most cases). We will want to explore possibilities for improving lock granularity in this code in the future. Submitted by: sam Sponsored by: FreeBSD Foundatiuon Obtained from: BSD/OS 5 snapshot provided by BSDi
2004-06-10 21:34:38 +00:00
mtx_lock(&Giant);
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
bad:
Introduce a subsystem lock around UNIX domain sockets in order to protect global and allocated variables. This strategy is derived from work originally developed by BSDi for BSD/OS, and applied to FreeBSD by Sam Leffler: - Add unp_mtx, a global mutex which will protect all UNIX domain socket related variables, structures, etc. - Add UNP_LOCK(), UNP_UNLOCK(), UNP_LOCK_ASSERT() macros. - Acquire unp_mtx on entering most UNIX domain socket code, drop/re-acquire around calls into VFS, and release it on return. - Avoid performing sodupsockaddr() while holding the mutex, so in general move to allocating storage before acquiring the mutex to copy the data. - Make a stack copy of the xucred rather than copying out while holding unp_mtx. Copy the peer credential out after releasing the mutex. - Add additional assertions of vnode locks following VOP_CREATE(). A few notes: - Use of an sx lock for the file list mutex may cause problems with regard to unp_mtx when garbage collection passed file descriptors. - The locking in unp_pcblist() for sysctl monitoring is correct subject to the unpcb zone not returning memory for reuse by other subsystems (consistent with similar existing concerns). - Sam's version of this change, as with the BSD/OS version, made use of both a global lock and per-unpcb locks. However, in practice, the global lock covered all accesses, so I have simplified out the unpcb locks in the interest of getting this merged faster (reducing the overhead but not sacrificing granularity in most cases). We will want to explore possibilities for improving lock granularity in this code in the future. Submitted by: sam Sponsored by: FreeBSD Foundatiuon Obtained from: BSD/OS 5 snapshot provided by BSDi
2004-06-10 21:34:38 +00:00
mtx_assert(&Giant, MA_OWNED);
if (vp != NULL)
vput(vp);
mtx_unlock(&Giant);
free(sa, M_SONAME);
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
UNP_GLOBAL_WLOCK();
UNP_PCB_LOCK(unp);
Add two new unpcb flags, UNP_BINDING and UNP_CONNECTING, which will be used to mark UNIX domain sockets as being in the process of binding or connecting. Use these to prevent simultaneous bind or connect operations by multiple threads or processes on the same socket at the same time, which closes race conditions present in the UNIX domain socket implementation since inception.
2006-07-23 12:01:14 +00:00
unp->unp_flags &= ~UNP_CONNECTING;
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
UNP_PCB_UNLOCK(unp);
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
return (error);
}
Export uipc_connect2() from uipc_usrreq.c instead of unp_connect2(), and consume that interface in portalfs and fifofs instead. In the new world order, unp_connect2() assumes that the unpcb mutex is held, whereas uipc_connect2() validates that the passed sockets are UNIX domain sockets, then grabs the mutex. NB: the portalfs and fifofs code gets down and dirty with UNIX domain sockets. Maybe this is a bad thing.
2004-03-31 01:41:30 +00:00
static int
Implement unix(4) socket options LOCAL_CREDS and LOCAL_CONNWAIT. - Add unp_addsockcred() (for LOCAL_CREDS). - Add an argument to unp_connect2() to differentiate between PRU_CONNECT and PRU_CONNECT2. (for LOCAL_CONNWAIT) Obtained from: NetBSD (with some changes)
2005-04-13 00:01:46 +00:00
unp_connect2(struct socket *so, struct socket *so2, int req)
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
{
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
struct unpcb *unp;
style(9)-ize function headers, remove use of 'register'. MFC after: 3 days
2005-02-20 23:22:13 +00:00
struct unpcb *unp2;
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
unp = sotounpcb(so);
KASSERT(unp != NULL, ("unp_connect2: unp == NULL"));
unp2 = sotounpcb(so2);
KASSERT(unp2 != NULL, ("unp_connect2: unp2 == NULL"));
UNP_GLOBAL_WLOCK_ASSERT();
UNP_PCB_LOCK_ASSERT(unp);
UNP_PCB_LOCK_ASSERT(unp2);
Introduce a subsystem lock around UNIX domain sockets in order to protect global and allocated variables. This strategy is derived from work originally developed by BSDi for BSD/OS, and applied to FreeBSD by Sam Leffler: - Add unp_mtx, a global mutex which will protect all UNIX domain socket related variables, structures, etc. - Add UNP_LOCK(), UNP_UNLOCK(), UNP_LOCK_ASSERT() macros. - Acquire unp_mtx on entering most UNIX domain socket code, drop/re-acquire around calls into VFS, and release it on return. - Avoid performing sodupsockaddr() while holding the mutex, so in general move to allocating storage before acquiring the mutex to copy the data. - Make a stack copy of the xucred rather than copying out while holding unp_mtx. Copy the peer credential out after releasing the mutex. - Add additional assertions of vnode locks following VOP_CREATE(). A few notes: - Use of an sx lock for the file list mutex may cause problems with regard to unp_mtx when garbage collection passed file descriptors. - The locking in unp_pcblist() for sysctl monitoring is correct subject to the unpcb zone not returning memory for reuse by other subsystems (consistent with similar existing concerns). - Sam's version of this change, as with the BSD/OS version, made use of both a global lock and per-unpcb locks. However, in practice, the global lock covered all accesses, so I have simplified out the unpcb locks in the interest of getting this merged faster (reducing the overhead but not sacrificing granularity in most cases). We will want to explore possibilities for improving lock granularity in this code in the future. Submitted by: sam Sponsored by: FreeBSD Foundatiuon Obtained from: BSD/OS 5 snapshot provided by BSDi
2004-06-10 21:34:38 +00:00
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
if (so2->so_type != so->so_type)
return (EPROTOTYPE);
unp->unp_conn = unp2;
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
switch (so->so_type) {
case SOCK_DGRAM:
Convert socket structures to be type-stable and add a version number. Define a parameter which indicates the maximum number of sockets in a system, and use this to size the zone allocators used for sockets and for certain PCBs. Convert PF_LOCAL PCB structures to be type-stable and add a version number. Define an external format for infomation about socket structures and use it in several places. Define a mechanism to get all PF_LOCAL and PF_INET PCB lists through sysctl(3) without blocking network interrupts for an unreasonable length of time. This probably still has some bugs and/or race conditions, but it seems to work well enough on my machines. It is now possible for `netstat' to get almost all of its information via the sysctl(3) interface rather than reading kmem (changes to follow).
1998-05-15 20:11:40 +00:00
LIST_INSERT_HEAD(&unp2->unp_refs, unp, unp_reflink);
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
soisconnected(so);
break;
case SOCK_STREAM:
unp2->unp_conn = unp;
Implement unix(4) socket options LOCAL_CREDS and LOCAL_CONNWAIT. - Add unp_addsockcred() (for LOCAL_CREDS). - Add an argument to unp_connect2() to differentiate between PRU_CONNECT and PRU_CONNECT2. (for LOCAL_CONNWAIT) Obtained from: NetBSD (with some changes)
2005-04-13 00:01:46 +00:00
if (req == PRU_CONNECT &&
((unp->unp_flags | unp2->unp_flags) & UNP_CONNWAIT))
soisconnecting(so);
else
soisconnected(so);
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
soisconnected(so2);
break;
default:
panic("unp_connect2");
}
return (0);
}
Another mega commit to staticize things.
1995-12-14 09:55:16 +00:00
static void
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
unp_disconnect(struct unpcb *unp, struct unpcb *unp2)
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
{
Hold SOCK_LOCK(so) when frobbing so_state when disconnecting a connected UNIX domain datagram socket.
2004-06-20 21:29:56 +00:00
struct socket *so;
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
KASSERT(unp2 != NULL, ("unp_disconnect: unp2 == NULL"));
UNP_GLOBAL_WLOCK_ASSERT();
UNP_PCB_LOCK_ASSERT(unp);
UNP_PCB_LOCK_ASSERT(unp2);
Introduce a subsystem lock around UNIX domain sockets in order to protect global and allocated variables. This strategy is derived from work originally developed by BSDi for BSD/OS, and applied to FreeBSD by Sam Leffler: - Add unp_mtx, a global mutex which will protect all UNIX domain socket related variables, structures, etc. - Add UNP_LOCK(), UNP_UNLOCK(), UNP_LOCK_ASSERT() macros. - Acquire unp_mtx on entering most UNIX domain socket code, drop/re-acquire around calls into VFS, and release it on return. - Avoid performing sodupsockaddr() while holding the mutex, so in general move to allocating storage before acquiring the mutex to copy the data. - Make a stack copy of the xucred rather than copying out while holding unp_mtx. Copy the peer credential out after releasing the mutex. - Add additional assertions of vnode locks following VOP_CREATE(). A few notes: - Use of an sx lock for the file list mutex may cause problems with regard to unp_mtx when garbage collection passed file descriptors. - The locking in unp_pcblist() for sysctl monitoring is correct subject to the unpcb zone not returning memory for reuse by other subsystems (consistent with similar existing concerns). - Sam's version of this change, as with the BSD/OS version, made use of both a global lock and per-unpcb locks. However, in practice, the global lock covered all accesses, so I have simplified out the unpcb locks in the interest of getting this merged faster (reducing the overhead but not sacrificing granularity in most cases). We will want to explore possibilities for improving lock granularity in this code in the future. Submitted by: sam Sponsored by: FreeBSD Foundatiuon Obtained from: BSD/OS 5 snapshot provided by BSDi
2004-06-10 21:34:38 +00:00
Prefer NULL to 0 when testing and assigning pointer values.
2004-03-30 02:16:25 +00:00
unp->unp_conn = NULL;
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
switch (unp->unp_socket->so_type) {
case SOCK_DGRAM:
Convert socket structures to be type-stable and add a version number. Define a parameter which indicates the maximum number of sockets in a system, and use this to size the zone allocators used for sockets and for certain PCBs. Convert PF_LOCAL PCB structures to be type-stable and add a version number. Define an external format for infomation about socket structures and use it in several places. Define a mechanism to get all PF_LOCAL and PF_INET PCB lists through sysctl(3) without blocking network interrupts for an unreasonable length of time. This probably still has some bugs and/or race conditions, but it seems to work well enough on my machines. It is now possible for `netstat' to get almost all of its information via the sysctl(3) interface rather than reading kmem (changes to follow).
1998-05-15 20:11:40 +00:00
LIST_REMOVE(unp, unp_reflink);
Hold SOCK_LOCK(so) when frobbing so_state when disconnecting a connected UNIX domain datagram socket.
2004-06-20 21:29:56 +00:00
so = unp->unp_socket;
SOCK_LOCK(so);
so->so_state &= ~SS_ISCONNECTED;
SOCK_UNLOCK(so);
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
break;
case SOCK_STREAM:
soisdisconnected(unp->unp_socket);
Prefer NULL to 0 when testing and assigning pointer values.
2004-03-30 02:16:25 +00:00
unp2->unp_conn = NULL;
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
soisdisconnected(unp2->unp_socket);
break;
}
}
Introduce a subsystem lock around UNIX domain sockets in order to protect global and allocated variables. This strategy is derived from work originally developed by BSDi for BSD/OS, and applied to FreeBSD by Sam Leffler: - Add unp_mtx, a global mutex which will protect all UNIX domain socket related variables, structures, etc. - Add UNP_LOCK(), UNP_UNLOCK(), UNP_LOCK_ASSERT() macros. - Acquire unp_mtx on entering most UNIX domain socket code, drop/re-acquire around calls into VFS, and release it on return. - Avoid performing sodupsockaddr() while holding the mutex, so in general move to allocating storage before acquiring the mutex to copy the data. - Make a stack copy of the xucred rather than copying out while holding unp_mtx. Copy the peer credential out after releasing the mutex. - Add additional assertions of vnode locks following VOP_CREATE(). A few notes: - Use of an sx lock for the file list mutex may cause problems with regard to unp_mtx when garbage collection passed file descriptors. - The locking in unp_pcblist() for sysctl monitoring is correct subject to the unpcb zone not returning memory for reuse by other subsystems (consistent with similar existing concerns). - Sam's version of this change, as with the BSD/OS version, made use of both a global lock and per-unpcb locks. However, in practice, the global lock covered all accesses, so I have simplified out the unpcb locks in the interest of getting this merged faster (reducing the overhead but not sacrificing granularity in most cases). We will want to explore possibilities for improving lock granularity in this code in the future. Submitted by: sam Sponsored by: FreeBSD Foundatiuon Obtained from: BSD/OS 5 snapshot provided by BSDi
2004-06-10 21:34:38 +00:00
/*
Re-wrap and other minor formatting and punctuation fixes for UNIX domain socket comments.
2006-07-22 17:24:55 +00:00
* unp_pcblist() assumes that UNIX domain socket memory is never reclaimed by
* the zone (UMA_ZONE_NOFREE), and as such potentially stale pointers are
* safe to reference. It first scans the list of struct unpcb's to generate
* a pointer list, then it rescans its list one entry at a time to
Introduce a subsystem lock around UNIX domain sockets in order to protect global and allocated variables. This strategy is derived from work originally developed by BSDi for BSD/OS, and applied to FreeBSD by Sam Leffler: - Add unp_mtx, a global mutex which will protect all UNIX domain socket related variables, structures, etc. - Add UNP_LOCK(), UNP_UNLOCK(), UNP_LOCK_ASSERT() macros. - Acquire unp_mtx on entering most UNIX domain socket code, drop/re-acquire around calls into VFS, and release it on return. - Avoid performing sodupsockaddr() while holding the mutex, so in general move to allocating storage before acquiring the mutex to copy the data. - Make a stack copy of the xucred rather than copying out while holding unp_mtx. Copy the peer credential out after releasing the mutex. - Add additional assertions of vnode locks following VOP_CREATE(). A few notes: - Use of an sx lock for the file list mutex may cause problems with regard to unp_mtx when garbage collection passed file descriptors. - The locking in unp_pcblist() for sysctl monitoring is correct subject to the unpcb zone not returning memory for reuse by other subsystems (consistent with similar existing concerns). - Sam's version of this change, as with the BSD/OS version, made use of both a global lock and per-unpcb locks. However, in practice, the global lock covered all accesses, so I have simplified out the unpcb locks in the interest of getting this merged faster (reducing the overhead but not sacrificing granularity in most cases). We will want to explore possibilities for improving lock granularity in this code in the future. Submitted by: sam Sponsored by: FreeBSD Foundatiuon Obtained from: BSD/OS 5 snapshot provided by BSDi
2004-06-10 21:34:38 +00:00
* externalize and copyout. It checks the generation number to see if a
* struct unpcb has been reused, and will skip it if so.
*/
Convert socket structures to be type-stable and add a version number. Define a parameter which indicates the maximum number of sockets in a system, and use this to size the zone allocators used for sockets and for certain PCBs. Convert PF_LOCAL PCB structures to be type-stable and add a version number. Define an external format for infomation about socket structures and use it in several places. Define a mechanism to get all PF_LOCAL and PF_INET PCB lists through sysctl(3) without blocking network interrupts for an unreasonable length of time. This probably still has some bugs and/or race conditions, but it seems to work well enough on my machines. It is now possible for `netstat' to get almost all of its information via the sysctl(3) interface rather than reading kmem (changes to follow).
1998-05-15 20:11:40 +00:00
static int
Previous commit changing SYSCTL_HANDLER_ARGS violated KNF. Pointed out by: bde
2000-07-04 11:25:35 +00:00
unp_pcblist(SYSCTL_HANDLER_ARGS)
Convert socket structures to be type-stable and add a version number. Define a parameter which indicates the maximum number of sockets in a system, and use this to size the zone allocators used for sockets and for certain PCBs. Convert PF_LOCAL PCB structures to be type-stable and add a version number. Define an external format for infomation about socket structures and use it in several places. Define a mechanism to get all PF_LOCAL and PF_INET PCB lists through sysctl(3) without blocking network interrupts for an unreasonable length of time. This probably still has some bugs and/or race conditions, but it seems to work well enough on my machines. It is now possible for `netstat' to get almost all of its information via the sysctl(3) interface rather than reading kmem (changes to follow).
1998-05-15 20:11:40 +00:00
{
Nitpicking and dusting performed on a train. Removes trivial warnings about unused variables, labels and other lint.
1998-10-25 17:44:59 +00:00
int error, i, n;
- Close a race between enumerating UNIX domain socket pcb structures via sysctl and socket teardown by adding a reference count to the UNIX domain pcb object and fixing the sysctl that enumerates unpcbs to grab a reference on each unpcb while it builds the list to copy out to userland. - Close a race between UNIX domain pcb garbage collection (unp_gc()) and file descriptor teardown (fdrop()) by adding a new garbage collection flag FWAIT. unp_gc() sets FWAIT while it walks the message buffers in a UNIX domain socket looking for nested file descriptor references and clears the flag when it is finished. fdrop() checks to see if the flag is set on a file descriptor whose refcount just dropped to 0 and waits for unp_gc() to clear the flag before completely destroying the file descriptor. MFC after: 1 week Reviewed by: rwatson Submitted by: ups Hopefully makes the panics go away: mx1
2007-01-05 19:59:46 +00:00
int freeunp;
Convert socket structures to be type-stable and add a version number. Define a parameter which indicates the maximum number of sockets in a system, and use this to size the zone allocators used for sockets and for certain PCBs. Convert PF_LOCAL PCB structures to be type-stable and add a version number. Define an external format for infomation about socket structures and use it in several places. Define a mechanism to get all PF_LOCAL and PF_INET PCB lists through sysctl(3) without blocking network interrupts for an unreasonable length of time. This probably still has some bugs and/or race conditions, but it seems to work well enough on my machines. It is now possible for `netstat' to get almost all of its information via the sysctl(3) interface rather than reading kmem (changes to follow).
1998-05-15 20:11:40 +00:00
struct unpcb *unp, **unp_list;
unp_gen_t gencnt;
Don't alocate a 400 byte buffer on the stack, Nor 800 bytes of structures.. MFC after: 2 weeks
2001-08-18 02:53:50 +00:00
struct xunpgen *xug;
Convert socket structures to be type-stable and add a version number. Define a parameter which indicates the maximum number of sockets in a system, and use this to size the zone allocators used for sockets and for certain PCBs. Convert PF_LOCAL PCB structures to be type-stable and add a version number. Define an external format for infomation about socket structures and use it in several places. Define a mechanism to get all PF_LOCAL and PF_INET PCB lists through sysctl(3) without blocking network interrupts for an unreasonable length of time. This probably still has some bugs and/or race conditions, but it seems to work well enough on my machines. It is now possible for `netstat' to get almost all of its information via the sysctl(3) interface rather than reading kmem (changes to follow).
1998-05-15 20:11:40 +00:00
struct unp_head *head;
Don't alocate a 400 byte buffer on the stack, Nor 800 bytes of structures.. MFC after: 2 weeks
2001-08-18 02:53:50 +00:00
struct xunpcb *xu;
Convert socket structures to be type-stable and add a version number. Define a parameter which indicates the maximum number of sockets in a system, and use this to size the zone allocators used for sockets and for certain PCBs. Convert PF_LOCAL PCB structures to be type-stable and add a version number. Define an external format for infomation about socket structures and use it in several places. Define a mechanism to get all PF_LOCAL and PF_INET PCB lists through sysctl(3) without blocking network interrupts for an unreasonable length of time. This probably still has some bugs and/or race conditions, but it seems to work well enough on my machines. It is now possible for `netstat' to get almost all of its information via the sysctl(3) interface rather than reading kmem (changes to follow).
1998-05-15 20:11:40 +00:00
Cast pointers to uintptr_t/intptr_t instead of to u_long/long, respectively. Most of the longs should probably have been u_longs, but this changes is just to prevent warnings about casts between pointers and integers of different sizes, not to fix poorly chosen types.
1998-07-15 02:32:35 +00:00
head = ((intptr_t)arg1 == SOCK_DGRAM ? &unp_dhead : &unp_shead);
Convert socket structures to be type-stable and add a version number. Define a parameter which indicates the maximum number of sockets in a system, and use this to size the zone allocators used for sockets and for certain PCBs. Convert PF_LOCAL PCB structures to be type-stable and add a version number. Define an external format for infomation about socket structures and use it in several places. Define a mechanism to get all PF_LOCAL and PF_INET PCB lists through sysctl(3) without blocking network interrupts for an unreasonable length of time. This probably still has some bugs and/or race conditions, but it seems to work well enough on my machines. It is now possible for `netstat' to get almost all of its information via the sysctl(3) interface rather than reading kmem (changes to follow).
1998-05-15 20:11:40 +00:00
/*
* The process of preparing the PCB list is too time-consuming and
* resource-intensive to repeat twice on every request.
*/
Prefer NULL to 0 when testing and assigning pointer values.
2004-03-30 02:16:25 +00:00
if (req->oldptr == NULL) {
Convert socket structures to be type-stable and add a version number. Define a parameter which indicates the maximum number of sockets in a system, and use this to size the zone allocators used for sockets and for certain PCBs. Convert PF_LOCAL PCB structures to be type-stable and add a version number. Define an external format for infomation about socket structures and use it in several places. Define a mechanism to get all PF_LOCAL and PF_INET PCB lists through sysctl(3) without blocking network interrupts for an unreasonable length of time. This probably still has some bugs and/or race conditions, but it seems to work well enough on my machines. It is now possible for `netstat' to get almost all of its information via the sysctl(3) interface rather than reading kmem (changes to follow).
1998-05-15 20:11:40 +00:00
n = unp_count;
Don't alocate a 400 byte buffer on the stack, Nor 800 bytes of structures.. MFC after: 2 weeks
2001-08-18 02:53:50 +00:00
req->oldidx = 2 * (sizeof *xug)
Convert socket structures to be type-stable and add a version number. Define a parameter which indicates the maximum number of sockets in a system, and use this to size the zone allocators used for sockets and for certain PCBs. Convert PF_LOCAL PCB structures to be type-stable and add a version number. Define an external format for infomation about socket structures and use it in several places. Define a mechanism to get all PF_LOCAL and PF_INET PCB lists through sysctl(3) without blocking network interrupts for an unreasonable length of time. This probably still has some bugs and/or race conditions, but it seems to work well enough on my machines. It is now possible for `netstat' to get almost all of its information via the sysctl(3) interface rather than reading kmem (changes to follow).
1998-05-15 20:11:40 +00:00
+ (n + n/8) * sizeof(struct xunpcb);
Mechanical whitespace cleanup; parenthesize return values; other minor style nits.
2004-01-11 19:48:19 +00:00
return (0);
Convert socket structures to be type-stable and add a version number. Define a parameter which indicates the maximum number of sockets in a system, and use this to size the zone allocators used for sockets and for certain PCBs. Convert PF_LOCAL PCB structures to be type-stable and add a version number. Define an external format for infomation about socket structures and use it in several places. Define a mechanism to get all PF_LOCAL and PF_INET PCB lists through sysctl(3) without blocking network interrupts for an unreasonable length of time. This probably still has some bugs and/or race conditions, but it seems to work well enough on my machines. It is now possible for `netstat' to get almost all of its information via the sysctl(3) interface rather than reading kmem (changes to follow).
1998-05-15 20:11:40 +00:00
}
Prefer NULL to 0 when testing and assigning pointer values.
2004-03-30 02:16:25 +00:00
if (req->newptr != NULL)
Mechanical whitespace cleanup; parenthesize return values; other minor style nits.
2004-01-11 19:48:19 +00:00
return (EPERM);
Convert socket structures to be type-stable and add a version number. Define a parameter which indicates the maximum number of sockets in a system, and use this to size the zone allocators used for sockets and for certain PCBs. Convert PF_LOCAL PCB structures to be type-stable and add a version number. Define an external format for infomation about socket structures and use it in several places. Define a mechanism to get all PF_LOCAL and PF_INET PCB lists through sysctl(3) without blocking network interrupts for an unreasonable length of time. This probably still has some bugs and/or race conditions, but it seems to work well enough on my machines. It is now possible for `netstat' to get almost all of its information via the sysctl(3) interface rather than reading kmem (changes to follow).
1998-05-15 20:11:40 +00:00
/*
* OK, now we're committed to doing something.
*/
Back out M_* changes, per decision of the TRB. Approved by: trb
2003-02-19 05:47:46 +00:00
xug = malloc(sizeof(*xug), M_TEMP, M_WAITOK);
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
UNP_GLOBAL_RLOCK();
Convert socket structures to be type-stable and add a version number. Define a parameter which indicates the maximum number of sockets in a system, and use this to size the zone allocators used for sockets and for certain PCBs. Convert PF_LOCAL PCB structures to be type-stable and add a version number. Define an external format for infomation about socket structures and use it in several places. Define a mechanism to get all PF_LOCAL and PF_INET PCB lists through sysctl(3) without blocking network interrupts for an unreasonable length of time. This probably still has some bugs and/or race conditions, but it seems to work well enough on my machines. It is now possible for `netstat' to get almost all of its information via the sysctl(3) interface rather than reading kmem (changes to follow).
1998-05-15 20:11:40 +00:00
gencnt = unp_gencnt;
n = unp_count;
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
UNP_GLOBAL_RUNLOCK();
Convert socket structures to be type-stable and add a version number. Define a parameter which indicates the maximum number of sockets in a system, and use this to size the zone allocators used for sockets and for certain PCBs. Convert PF_LOCAL PCB structures to be type-stable and add a version number. Define an external format for infomation about socket structures and use it in several places. Define a mechanism to get all PF_LOCAL and PF_INET PCB lists through sysctl(3) without blocking network interrupts for an unreasonable length of time. This probably still has some bugs and/or race conditions, but it seems to work well enough on my machines. It is now possible for `netstat' to get almost all of its information via the sysctl(3) interface rather than reading kmem (changes to follow).
1998-05-15 20:11:40 +00:00
Don't alocate a 400 byte buffer on the stack, Nor 800 bytes of structures.. MFC after: 2 weeks
2001-08-18 02:53:50 +00:00
xug->xug_len = sizeof *xug;
xug->xug_count = n;
xug->xug_gen = gencnt;
xug->xug_sogen = so_gencnt;
error = SYSCTL_OUT(req, xug, sizeof *xug);
if (error) {
free(xug, M_TEMP);
Mechanical whitespace cleanup; parenthesize return values; other minor style nits.
2004-01-11 19:48:19 +00:00
return (error);
Don't alocate a 400 byte buffer on the stack, Nor 800 bytes of structures.. MFC after: 2 weeks
2001-08-18 02:53:50 +00:00
}
Convert socket structures to be type-stable and add a version number. Define a parameter which indicates the maximum number of sockets in a system, and use this to size the zone allocators used for sockets and for certain PCBs. Convert PF_LOCAL PCB structures to be type-stable and add a version number. Define an external format for infomation about socket structures and use it in several places. Define a mechanism to get all PF_LOCAL and PF_INET PCB lists through sysctl(3) without blocking network interrupts for an unreasonable length of time. This probably still has some bugs and/or race conditions, but it seems to work well enough on my machines. It is now possible for `netstat' to get almost all of its information via the sysctl(3) interface rather than reading kmem (changes to follow).
1998-05-15 20:11:40 +00:00
Back out M_* changes, per decision of the TRB. Approved by: trb
2003-02-19 05:47:46 +00:00
unp_list = malloc(n * sizeof *unp_list, M_TEMP, M_WAITOK);
Mechanical whitespace cleanup; parenthesize return values; other minor style nits.
2004-01-11 19:48:19 +00:00
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
/*
* XXXRW: Note, this code relies very explicitly in pcb's being type
* stable.
*/
UNP_GLOBAL_RLOCK();
This is a partial commit of the patch from PR 14914: Alot of the code in sys/kern directly accesses the *Q_HEAD and *Q_ENTRY structures for list operations. This patch makes all list operations in sys/kern use the queue(3) macros, rather than directly accessing the *Q_{HEAD,ENTRY} structures. This batch of changes compile to the same object files. Reviewed by: phk Submitted by: Jake Burkholder <jake@checker.org> PR: 14914
1999-11-16 10:56:05 +00:00
for (unp = LIST_FIRST(head), i = 0; unp && i < n;
unp = LIST_NEXT(unp, unp_link)) {
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
UNP_PCB_LOCK(unp);
- Combine kern.ps_showallprocs and kern.ipc.showallsockets into a single kern.security.seeotheruids_permitted, describes as: "Unprivileged processes may see subjects/objects with different real uid" NOTE: kern.ps_showallprocs exists in -STABLE, and therefore there is an API change. kern.ipc.showallsockets does not. - Check kern.security.seeotheruids_permitted in cr_cansee(). - Replace visibility calls to socheckuid() with cr_cansee() (retain the change to socheckuid() in ipfw, where it is used for rule-matching). - Remove prison_unpcb() and make use of cr_cansee() against the UNIX domain socket credential instead of comparing root vnodes for the UDS and the process. This allows multiple jails to share the same chroot() and not see each others UNIX domain sockets. - Remove unused socheckproc(). Now that cr_cansee() is used universally for socket visibility, a variety of policies are more consistently enforced, including uid-based restrictions and jail-based restrictions. This also better-supports the introduction of additional MAC models. Reviewed by: ps, billf Obtained from: TrustedBSD Project
2001-10-09 21:40:30 +00:00
if (unp->unp_gencnt <= gencnt) {
Simple p_ucred -> td_ucred changes to start using the per-thread ucred reference.
2002-02-27 18:32:23 +00:00
if (cr_cansee(req->td->td_ucred,
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
unp->unp_socket->so_cred)) {
UNP_PCB_UNLOCK(unp);
Only allow users to see their own socket connections if kern.ipc.showallsockets is set to 0. Submitted by: billf (with modifications by me) Inspired by: Dave McKay (aka pm aka Packet Magnet) Reviewed by: peter MFC after: 2 weeks
2001-10-05 07:06:32 +00:00
continue;
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
}
Convert socket structures to be type-stable and add a version number. Define a parameter which indicates the maximum number of sockets in a system, and use this to size the zone allocators used for sockets and for certain PCBs. Convert PF_LOCAL PCB structures to be type-stable and add a version number. Define an external format for infomation about socket structures and use it in several places. Define a mechanism to get all PF_LOCAL and PF_INET PCB lists through sysctl(3) without blocking network interrupts for an unreasonable length of time. This probably still has some bugs and/or race conditions, but it seems to work well enough on my machines. It is now possible for `netstat' to get almost all of its information via the sysctl(3) interface rather than reading kmem (changes to follow).
1998-05-15 20:11:40 +00:00
unp_list[i++] = unp;
- Close a race between enumerating UNIX domain socket pcb structures via sysctl and socket teardown by adding a reference count to the UNIX domain pcb object and fixing the sysctl that enumerates unpcbs to grab a reference on each unpcb while it builds the list to copy out to userland. - Close a race between UNIX domain pcb garbage collection (unp_gc()) and file descriptor teardown (fdrop()) by adding a new garbage collection flag FWAIT. unp_gc() sets FWAIT while it walks the message buffers in a UNIX domain socket looking for nested file descriptor references and clears the flag when it is finished. fdrop() checks to see if the flag is set on a file descriptor whose refcount just dropped to 0 and waits for unp_gc() to clear the flag before completely destroying the file descriptor. MFC after: 1 week Reviewed by: rwatson Submitted by: ups Hopefully makes the panics go away: mx1
2007-01-05 19:59:46 +00:00
unp->unp_refcount++;
Only allow users to see their own socket connections if kern.ipc.showallsockets is set to 0. Submitted by: billf (with modifications by me) Inspired by: Dave McKay (aka pm aka Packet Magnet) Reviewed by: peter MFC after: 2 weeks
2001-10-05 07:06:32 +00:00
}
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
UNP_PCB_UNLOCK(unp);
Convert socket structures to be type-stable and add a version number. Define a parameter which indicates the maximum number of sockets in a system, and use this to size the zone allocators used for sockets and for certain PCBs. Convert PF_LOCAL PCB structures to be type-stable and add a version number. Define an external format for infomation about socket structures and use it in several places. Define a mechanism to get all PF_LOCAL and PF_INET PCB lists through sysctl(3) without blocking network interrupts for an unreasonable length of time. This probably still has some bugs and/or race conditions, but it seems to work well enough on my machines. It is now possible for `netstat' to get almost all of its information via the sysctl(3) interface rather than reading kmem (changes to follow).
1998-05-15 20:11:40 +00:00
}
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
UNP_GLOBAL_RUNLOCK();
Re-wrap and other minor formatting and punctuation fixes for UNIX domain socket comments.
2006-07-22 17:24:55 +00:00
n = i; /* In case we lost some during malloc. */
Convert socket structures to be type-stable and add a version number. Define a parameter which indicates the maximum number of sockets in a system, and use this to size the zone allocators used for sockets and for certain PCBs. Convert PF_LOCAL PCB structures to be type-stable and add a version number. Define an external format for infomation about socket structures and use it in several places. Define a mechanism to get all PF_LOCAL and PF_INET PCB lists through sysctl(3) without blocking network interrupts for an unreasonable length of time. This probably still has some bugs and/or race conditions, but it seems to work well enough on my machines. It is now possible for `netstat' to get almost all of its information via the sysctl(3) interface rather than reading kmem (changes to follow).
1998-05-15 20:11:40 +00:00
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
/*
* XXXRW: The logic below asumes that it is OK to lock a mutex in
* an unpcb that may have been freed.
*/
Convert socket structures to be type-stable and add a version number. Define a parameter which indicates the maximum number of sockets in a system, and use this to size the zone allocators used for sockets and for certain PCBs. Convert PF_LOCAL PCB structures to be type-stable and add a version number. Define an external format for infomation about socket structures and use it in several places. Define a mechanism to get all PF_LOCAL and PF_INET PCB lists through sysctl(3) without blocking network interrupts for an unreasonable length of time. This probably still has some bugs and/or race conditions, but it seems to work well enough on my machines. It is now possible for `netstat' to get almost all of its information via the sysctl(3) interface rather than reading kmem (changes to follow).
1998-05-15 20:11:40 +00:00
error = 0;
Fix two issues which were missed in FreeBSD-SA-05:08.kmem. Reported by: Uwe Doering
2005-05-07 00:41:36 +00:00
xu = malloc(sizeof(*xu), M_TEMP, M_WAITOK | M_ZERO);
Convert socket structures to be type-stable and add a version number. Define a parameter which indicates the maximum number of sockets in a system, and use this to size the zone allocators used for sockets and for certain PCBs. Convert PF_LOCAL PCB structures to be type-stable and add a version number. Define an external format for infomation about socket structures and use it in several places. Define a mechanism to get all PF_LOCAL and PF_INET PCB lists through sysctl(3) without blocking network interrupts for an unreasonable length of time. This probably still has some bugs and/or race conditions, but it seems to work well enough on my machines. It is now possible for `netstat' to get almost all of its information via the sysctl(3) interface rather than reading kmem (changes to follow).
1998-05-15 20:11:40 +00:00
for (i = 0; i < n; i++) {
unp = unp_list[i];
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
UNP_PCB_LOCK(unp);
- Close a race between enumerating UNIX domain socket pcb structures via sysctl and socket teardown by adding a reference count to the UNIX domain pcb object and fixing the sysctl that enumerates unpcbs to grab a reference on each unpcb while it builds the list to copy out to userland. - Close a race between UNIX domain pcb garbage collection (unp_gc()) and file descriptor teardown (fdrop()) by adding a new garbage collection flag FWAIT. unp_gc() sets FWAIT while it walks the message buffers in a UNIX domain socket looking for nested file descriptor references and clears the flag when it is finished. fdrop() checks to see if the flag is set on a file descriptor whose refcount just dropped to 0 and waits for unp_gc() to clear the flag before completely destroying the file descriptor. MFC after: 1 week Reviewed by: rwatson Submitted by: ups Hopefully makes the panics go away: mx1
2007-01-05 19:59:46 +00:00
unp->unp_refcount--;
if (unp->unp_refcount != 0 && unp->unp_gencnt <= gencnt) {
Don't alocate a 400 byte buffer on the stack, Nor 800 bytes of structures.. MFC after: 2 weeks
2001-08-18 02:53:50 +00:00
xu->xu_len = sizeof *xu;
xu->xu_unpp = unp;
Convert socket structures to be type-stable and add a version number. Define a parameter which indicates the maximum number of sockets in a system, and use this to size the zone allocators used for sockets and for certain PCBs. Convert PF_LOCAL PCB structures to be type-stable and add a version number. Define an external format for infomation about socket structures and use it in several places. Define a mechanism to get all PF_LOCAL and PF_INET PCB lists through sysctl(3) without blocking network interrupts for an unreasonable length of time. This probably still has some bugs and/or race conditions, but it seems to work well enough on my machines. It is now possible for `netstat' to get almost all of its information via the sysctl(3) interface rather than reading kmem (changes to follow).
1998-05-15 20:11:40 +00:00
/*
* XXX - need more locking here to protect against
* connect/disconnect races for SMP.
*/
Prefer NULL to 0 when testing and assigning pointer values.
2004-03-30 02:16:25 +00:00
if (unp->unp_addr != NULL)
Mechanical whitespace cleanup; parenthesize return values; other minor style nits.
2004-01-11 19:48:19 +00:00
bcopy(unp->unp_addr, &xu->xu_addr,
Convert socket structures to be type-stable and add a version number. Define a parameter which indicates the maximum number of sockets in a system, and use this to size the zone allocators used for sockets and for certain PCBs. Convert PF_LOCAL PCB structures to be type-stable and add a version number. Define an external format for infomation about socket structures and use it in several places. Define a mechanism to get all PF_LOCAL and PF_INET PCB lists through sysctl(3) without blocking network interrupts for an unreasonable length of time. This probably still has some bugs and/or race conditions, but it seems to work well enough on my machines. It is now possible for `netstat' to get almost all of its information via the sysctl(3) interface rather than reading kmem (changes to follow).
1998-05-15 20:11:40 +00:00
unp->unp_addr->sun_len);
Prefer NULL to 0 when testing and assigning pointer values.
2004-03-30 02:16:25 +00:00
if (unp->unp_conn != NULL &&
unp->unp_conn->unp_addr != NULL)
Convert socket structures to be type-stable and add a version number. Define a parameter which indicates the maximum number of sockets in a system, and use this to size the zone allocators used for sockets and for certain PCBs. Convert PF_LOCAL PCB structures to be type-stable and add a version number. Define an external format for infomation about socket structures and use it in several places. Define a mechanism to get all PF_LOCAL and PF_INET PCB lists through sysctl(3) without blocking network interrupts for an unreasonable length of time. This probably still has some bugs and/or race conditions, but it seems to work well enough on my machines. It is now possible for `netstat' to get almost all of its information via the sysctl(3) interface rather than reading kmem (changes to follow).
1998-05-15 20:11:40 +00:00
bcopy(unp->unp_conn->unp_addr,
Don't alocate a 400 byte buffer on the stack, Nor 800 bytes of structures.. MFC after: 2 weeks
2001-08-18 02:53:50 +00:00
&xu->xu_caddr,
Convert socket structures to be type-stable and add a version number. Define a parameter which indicates the maximum number of sockets in a system, and use this to size the zone allocators used for sockets and for certain PCBs. Convert PF_LOCAL PCB structures to be type-stable and add a version number. Define an external format for infomation about socket structures and use it in several places. Define a mechanism to get all PF_LOCAL and PF_INET PCB lists through sysctl(3) without blocking network interrupts for an unreasonable length of time. This probably still has some bugs and/or race conditions, but it seems to work well enough on my machines. It is now possible for `netstat' to get almost all of its information via the sysctl(3) interface rather than reading kmem (changes to follow).
1998-05-15 20:11:40 +00:00
unp->unp_conn->unp_addr->sun_len);
Don't alocate a 400 byte buffer on the stack, Nor 800 bytes of structures.. MFC after: 2 weeks
2001-08-18 02:53:50 +00:00
bcopy(unp, &xu->xu_unp, sizeof *unp);
sotoxsocket(unp->unp_socket, &xu->xu_socket);
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
UNP_PCB_UNLOCK(unp);
Don't alocate a 400 byte buffer on the stack, Nor 800 bytes of structures.. MFC after: 2 weeks
2001-08-18 02:53:50 +00:00
error = SYSCTL_OUT(req, xu, sizeof *xu);
- Close a race between enumerating UNIX domain socket pcb structures via sysctl and socket teardown by adding a reference count to the UNIX domain pcb object and fixing the sysctl that enumerates unpcbs to grab a reference on each unpcb while it builds the list to copy out to userland. - Close a race between UNIX domain pcb garbage collection (unp_gc()) and file descriptor teardown (fdrop()) by adding a new garbage collection flag FWAIT. unp_gc() sets FWAIT while it walks the message buffers in a UNIX domain socket looking for nested file descriptor references and clears the flag when it is finished. fdrop() checks to see if the flag is set on a file descriptor whose refcount just dropped to 0 and waits for unp_gc() to clear the flag before completely destroying the file descriptor. MFC after: 1 week Reviewed by: rwatson Submitted by: ups Hopefully makes the panics go away: mx1
2007-01-05 19:59:46 +00:00
} else {
freeunp = (unp->unp_refcount == 0);
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
UNP_PCB_UNLOCK(unp);
if (freeunp) {
UNP_PCB_LOCK_DESTROY(unp);
- Close a race between enumerating UNIX domain socket pcb structures via sysctl and socket teardown by adding a reference count to the UNIX domain pcb object and fixing the sysctl that enumerates unpcbs to grab a reference on each unpcb while it builds the list to copy out to userland. - Close a race between UNIX domain pcb garbage collection (unp_gc()) and file descriptor teardown (fdrop()) by adding a new garbage collection flag FWAIT. unp_gc() sets FWAIT while it walks the message buffers in a UNIX domain socket looking for nested file descriptor references and clears the flag when it is finished. fdrop() checks to see if the flag is set on a file descriptor whose refcount just dropped to 0 and waits for unp_gc() to clear the flag before completely destroying the file descriptor. MFC after: 1 week Reviewed by: rwatson Submitted by: ups Hopefully makes the panics go away: mx1
2007-01-05 19:59:46 +00:00
uma_zfree(unp_zone, unp);
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
}
Convert socket structures to be type-stable and add a version number. Define a parameter which indicates the maximum number of sockets in a system, and use this to size the zone allocators used for sockets and for certain PCBs. Convert PF_LOCAL PCB structures to be type-stable and add a version number. Define an external format for infomation about socket structures and use it in several places. Define a mechanism to get all PF_LOCAL and PF_INET PCB lists through sysctl(3) without blocking network interrupts for an unreasonable length of time. This probably still has some bugs and/or race conditions, but it seems to work well enough on my machines. It is now possible for `netstat' to get almost all of its information via the sysctl(3) interface rather than reading kmem (changes to follow).
1998-05-15 20:11:40 +00:00
}
}
Don't alocate a 400 byte buffer on the stack, Nor 800 bytes of structures.. MFC after: 2 weeks
2001-08-18 02:53:50 +00:00
free(xu, M_TEMP);
Convert socket structures to be type-stable and add a version number. Define a parameter which indicates the maximum number of sockets in a system, and use this to size the zone allocators used for sockets and for certain PCBs. Convert PF_LOCAL PCB structures to be type-stable and add a version number. Define an external format for infomation about socket structures and use it in several places. Define a mechanism to get all PF_LOCAL and PF_INET PCB lists through sysctl(3) without blocking network interrupts for an unreasonable length of time. This probably still has some bugs and/or race conditions, but it seems to work well enough on my machines. It is now possible for `netstat' to get almost all of its information via the sysctl(3) interface rather than reading kmem (changes to follow).
1998-05-15 20:11:40 +00:00
if (!error) {
/*
Re-wrap and other minor formatting and punctuation fixes for UNIX domain socket comments.
2006-07-22 17:24:55 +00:00
* Give the user an updated idea of our state. If the
* generation differs from what we told her before, she knows
* that something happened while we were processing this
* request, and it might be necessary to retry.
Convert socket structures to be type-stable and add a version number. Define a parameter which indicates the maximum number of sockets in a system, and use this to size the zone allocators used for sockets and for certain PCBs. Convert PF_LOCAL PCB structures to be type-stable and add a version number. Define an external format for infomation about socket structures and use it in several places. Define a mechanism to get all PF_LOCAL and PF_INET PCB lists through sysctl(3) without blocking network interrupts for an unreasonable length of time. This probably still has some bugs and/or race conditions, but it seems to work well enough on my machines. It is now possible for `netstat' to get almost all of its information via the sysctl(3) interface rather than reading kmem (changes to follow).
1998-05-15 20:11:40 +00:00
*/
Don't alocate a 400 byte buffer on the stack, Nor 800 bytes of structures.. MFC after: 2 weeks
2001-08-18 02:53:50 +00:00
xug->xug_gen = unp_gencnt;
xug->xug_sogen = so_gencnt;
xug->xug_count = unp_count;
error = SYSCTL_OUT(req, xug, sizeof *xug);
Convert socket structures to be type-stable and add a version number. Define a parameter which indicates the maximum number of sockets in a system, and use this to size the zone allocators used for sockets and for certain PCBs. Convert PF_LOCAL PCB structures to be type-stable and add a version number. Define an external format for infomation about socket structures and use it in several places. Define a mechanism to get all PF_LOCAL and PF_INET PCB lists through sysctl(3) without blocking network interrupts for an unreasonable length of time. This probably still has some bugs and/or race conditions, but it seems to work well enough on my machines. It is now possible for `netstat' to get almost all of its information via the sysctl(3) interface rather than reading kmem (changes to follow).
1998-05-15 20:11:40 +00:00
}
free(unp_list, M_TEMP);
Don't alocate a 400 byte buffer on the stack, Nor 800 bytes of structures.. MFC after: 2 weeks
2001-08-18 02:53:50 +00:00
free(xug, M_TEMP);
Mechanical whitespace cleanup; parenthesize return values; other minor style nits.
2004-01-11 19:48:19 +00:00
return (error);
Convert socket structures to be type-stable and add a version number. Define a parameter which indicates the maximum number of sockets in a system, and use this to size the zone allocators used for sockets and for certain PCBs. Convert PF_LOCAL PCB structures to be type-stable and add a version number. Define an external format for infomation about socket structures and use it in several places. Define a mechanism to get all PF_LOCAL and PF_INET PCB lists through sysctl(3) without blocking network interrupts for an unreasonable length of time. This probably still has some bugs and/or race conditions, but it seems to work well enough on my machines. It is now possible for `netstat' to get almost all of its information via the sysctl(3) interface rather than reading kmem (changes to follow).
1998-05-15 20:11:40 +00:00
}
Mechanical whitespace cleanup; parenthesize return values; other minor style nits.
2004-01-11 19:48:19 +00:00
SYSCTL_PROC(_net_local_dgram, OID_AUTO, pcblist, CTLFLAG_RD,
Convert socket structures to be type-stable and add a version number. Define a parameter which indicates the maximum number of sockets in a system, and use this to size the zone allocators used for sockets and for certain PCBs. Convert PF_LOCAL PCB structures to be type-stable and add a version number. Define an external format for infomation about socket structures and use it in several places. Define a mechanism to get all PF_LOCAL and PF_INET PCB lists through sysctl(3) without blocking network interrupts for an unreasonable length of time. This probably still has some bugs and/or race conditions, but it seems to work well enough on my machines. It is now possible for `netstat' to get almost all of its information via the sysctl(3) interface rather than reading kmem (changes to follow).
1998-05-15 20:11:40 +00:00
(caddr_t)(long)SOCK_DGRAM, 0, unp_pcblist, "S,xunpcb",
"List of active local datagram sockets");
Mechanical whitespace cleanup; parenthesize return values; other minor style nits.
2004-01-11 19:48:19 +00:00
SYSCTL_PROC(_net_local_stream, OID_AUTO, pcblist, CTLFLAG_RD,
Convert socket structures to be type-stable and add a version number. Define a parameter which indicates the maximum number of sockets in a system, and use this to size the zone allocators used for sockets and for certain PCBs. Convert PF_LOCAL PCB structures to be type-stable and add a version number. Define an external format for infomation about socket structures and use it in several places. Define a mechanism to get all PF_LOCAL and PF_INET PCB lists through sysctl(3) without blocking network interrupts for an unreasonable length of time. This probably still has some bugs and/or race conditions, but it seems to work well enough on my machines. It is now possible for `netstat' to get almost all of its information via the sysctl(3) interface rather than reading kmem (changes to follow).
1998-05-15 20:11:40 +00:00
(caddr_t)(long)SOCK_STREAM, 0, unp_pcblist, "S,xunpcb",
"List of active local stream sockets");
Another mega commit to staticize things.
1995-12-14 09:55:16 +00:00
static void
style(9)-ize function headers, remove use of 'register'. MFC after: 3 days
2005-02-20 23:22:13 +00:00
unp_shutdown(struct unpcb *unp)
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
{
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
struct unpcb *unp2;
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
struct socket *so;
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
UNP_GLOBAL_WLOCK_ASSERT();
UNP_PCB_LOCK_ASSERT(unp);
Introduce a subsystem lock around UNIX domain sockets in order to protect global and allocated variables. This strategy is derived from work originally developed by BSDi for BSD/OS, and applied to FreeBSD by Sam Leffler: - Add unp_mtx, a global mutex which will protect all UNIX domain socket related variables, structures, etc. - Add UNP_LOCK(), UNP_UNLOCK(), UNP_LOCK_ASSERT() macros. - Acquire unp_mtx on entering most UNIX domain socket code, drop/re-acquire around calls into VFS, and release it on return. - Avoid performing sodupsockaddr() while holding the mutex, so in general move to allocating storage before acquiring the mutex to copy the data. - Make a stack copy of the xucred rather than copying out while holding unp_mtx. Copy the peer credential out after releasing the mutex. - Add additional assertions of vnode locks following VOP_CREATE(). A few notes: - Use of an sx lock for the file list mutex may cause problems with regard to unp_mtx when garbage collection passed file descriptors. - The locking in unp_pcblist() for sysctl monitoring is correct subject to the unpcb zone not returning memory for reuse by other subsystems (consistent with similar existing concerns). - Sam's version of this change, as with the BSD/OS version, made use of both a global lock and per-unpcb locks. However, in practice, the global lock covered all accesses, so I have simplified out the unpcb locks in the interest of getting this merged faster (reducing the overhead but not sacrificing granularity in most cases). We will want to explore possibilities for improving lock granularity in this code in the future. Submitted by: sam Sponsored by: FreeBSD Foundatiuon Obtained from: BSD/OS 5 snapshot provided by BSDi
2004-06-10 21:34:38 +00:00
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
unp2 = unp->unp_conn;
if (unp->unp_socket->so_type == SOCK_STREAM && unp2 != NULL) {
so = unp2->unp_socket;
if (so != NULL)
socantrcvmore(so);
}
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
}
Another mega commit to staticize things.
1995-12-14 09:55:16 +00:00
static void
style(9)-ize function headers, remove use of 'register'. MFC after: 3 days
2005-02-20 23:22:13 +00:00
unp_drop(struct unpcb *unp, int errno)
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
{
struct socket *so = unp->unp_socket;
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
struct unpcb *unp2;
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
UNP_GLOBAL_WLOCK_ASSERT();
UNP_PCB_LOCK_ASSERT(unp);
Introduce a subsystem lock around UNIX domain sockets in order to protect global and allocated variables. This strategy is derived from work originally developed by BSDi for BSD/OS, and applied to FreeBSD by Sam Leffler: - Add unp_mtx, a global mutex which will protect all UNIX domain socket related variables, structures, etc. - Add UNP_LOCK(), UNP_UNLOCK(), UNP_LOCK_ASSERT() macros. - Acquire unp_mtx on entering most UNIX domain socket code, drop/re-acquire around calls into VFS, and release it on return. - Avoid performing sodupsockaddr() while holding the mutex, so in general move to allocating storage before acquiring the mutex to copy the data. - Make a stack copy of the xucred rather than copying out while holding unp_mtx. Copy the peer credential out after releasing the mutex. - Add additional assertions of vnode locks following VOP_CREATE(). A few notes: - Use of an sx lock for the file list mutex may cause problems with regard to unp_mtx when garbage collection passed file descriptors. - The locking in unp_pcblist() for sysctl monitoring is correct subject to the unpcb zone not returning memory for reuse by other subsystems (consistent with similar existing concerns). - Sam's version of this change, as with the BSD/OS version, made use of both a global lock and per-unpcb locks. However, in practice, the global lock covered all accesses, so I have simplified out the unpcb locks in the interest of getting this merged faster (reducing the overhead but not sacrificing granularity in most cases). We will want to explore possibilities for improving lock granularity in this code in the future. Submitted by: sam Sponsored by: FreeBSD Foundatiuon Obtained from: BSD/OS 5 snapshot provided by BSDi
2004-06-10 21:34:38 +00:00
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
so->so_error = errno;
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
unp2 = unp->unp_conn;
if (unp2 == NULL)
return;
UNP_PCB_LOCK(unp2);
unp_disconnect(unp, unp2);
UNP_PCB_UNLOCK(unp2);
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
}
Hopefully improve control message passing over Unix domain sockets. 1) Allow the sending of more than one control message at a time over a unix domain socket. This should cover the PR 29499. 2) This requires that unp_{ex,in}ternalize and unp_scan understand mbufs with more than one control message at a time. 3) Internalize and externalize used to work on the mbuf in-place. This made life quite complicated and the code for sizeof(int) < sizeof(file *) could end up doing the wrong thing. The patch always create a new mbuf/cluster now. This resulted in the change of the prototype for the domain externalise function. 4) You can now send SCM_TIMESTAMP messages. 5) Always use CMSG_DATA(cm) to determine the start where the data in unp_{ex,in}ternalize. It was using ((struct cmsghdr *)cm + 1) in some places, which gives the wrong alignment on the alpha. (NetBSD made this fix some time ago). This results in an ABI change for discriptor passing and creds passing on the alpha. (Probably on the IA64 and Spare ports too). 6) Fix userland programs to use CMSG_* macros too. 7) Be more careful about freeing mbufs containing (file *)s. This is made possible by the prototype change of externalise. PR: 29499 MFC after: 6 weeks
2001-10-04 13:11:48 +00:00
static void
style(9)-ize function headers, remove use of 'register'. MFC after: 3 days
2005-02-20 23:22:13 +00:00
unp_freerights(struct file **rp, int fdcount)
Hopefully improve control message passing over Unix domain sockets. 1) Allow the sending of more than one control message at a time over a unix domain socket. This should cover the PR 29499. 2) This requires that unp_{ex,in}ternalize and unp_scan understand mbufs with more than one control message at a time. 3) Internalize and externalize used to work on the mbuf in-place. This made life quite complicated and the code for sizeof(int) < sizeof(file *) could end up doing the wrong thing. The patch always create a new mbuf/cluster now. This resulted in the change of the prototype for the domain externalise function. 4) You can now send SCM_TIMESTAMP messages. 5) Always use CMSG_DATA(cm) to determine the start where the data in unp_{ex,in}ternalize. It was using ((struct cmsghdr *)cm + 1) in some places, which gives the wrong alignment on the alpha. (NetBSD made this fix some time ago). This results in an ABI change for discriptor passing and creds passing on the alpha. (Probably on the IA64 and Spare ports too). 6) Fix userland programs to use CMSG_* macros too. 7) Be more careful about freeing mbufs containing (file *)s. This is made possible by the prototype change of externalise. PR: 29499 MFC after: 6 weeks
2001-10-04 13:11:48 +00:00
{
int i;
struct file *fp;
for (i = 0; i < fdcount; i++) {
/*
Re-wrap and other minor formatting and punctuation fixes for UNIX domain socket comments.
2006-07-22 17:24:55 +00:00
* Zero the pointer before calling unp_discard since it may
* end up in unp_gc()..
XXX a comment in uipc_usrreq.c that requires updating.
2006-01-13 00:00:32 +00:00
*
* XXXRW: This is less true than it used to be.
Hopefully improve control message passing over Unix domain sockets. 1) Allow the sending of more than one control message at a time over a unix domain socket. This should cover the PR 29499. 2) This requires that unp_{ex,in}ternalize and unp_scan understand mbufs with more than one control message at a time. 3) Internalize and externalize used to work on the mbuf in-place. This made life quite complicated and the code for sizeof(int) < sizeof(file *) could end up doing the wrong thing. The patch always create a new mbuf/cluster now. This resulted in the change of the prototype for the domain externalise function. 4) You can now send SCM_TIMESTAMP messages. 5) Always use CMSG_DATA(cm) to determine the start where the data in unp_{ex,in}ternalize. It was using ((struct cmsghdr *)cm + 1) in some places, which gives the wrong alignment on the alpha. (NetBSD made this fix some time ago). This results in an ABI change for discriptor passing and creds passing on the alpha. (Probably on the IA64 and Spare ports too). 6) Fix userland programs to use CMSG_* macros too. 7) Be more careful about freeing mbufs containing (file *)s. This is made possible by the prototype change of externalise. PR: 29499 MFC after: 6 weeks
2001-10-04 13:11:48 +00:00
*/
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
fp = *rp;
*rp++ = NULL;
Hopefully improve control message passing over Unix domain sockets. 1) Allow the sending of more than one control message at a time over a unix domain socket. This should cover the PR 29499. 2) This requires that unp_{ex,in}ternalize and unp_scan understand mbufs with more than one control message at a time. 3) Internalize and externalize used to work on the mbuf in-place. This made life quite complicated and the code for sizeof(int) < sizeof(file *) could end up doing the wrong thing. The patch always create a new mbuf/cluster now. This resulted in the change of the prototype for the domain externalise function. 4) You can now send SCM_TIMESTAMP messages. 5) Always use CMSG_DATA(cm) to determine the start where the data in unp_{ex,in}ternalize. It was using ((struct cmsghdr *)cm + 1) in some places, which gives the wrong alignment on the alpha. (NetBSD made this fix some time ago). This results in an ABI change for discriptor passing and creds passing on the alpha. (Probably on the IA64 and Spare ports too). 6) Fix userland programs to use CMSG_* macros too. 7) Be more careful about freeing mbufs containing (file *)s. This is made possible by the prototype change of externalise. PR: 29499 MFC after: 6 weeks
2001-10-04 13:11:48 +00:00
unp_discard(fp);
}
}
The big 4.4BSD Lite to FreeBSD 2.0.0 (Development) patch. Reviewed by: Rodney W. Grimes Submitted by: John Dyson and David Greenman
1994-05-25 09:21:21 +00:00
int
style(9)-ize function headers, remove use of 'register'. MFC after: 3 days
2005-02-20 23:22:13 +00:00
unp_externalize(struct mbuf *control, struct mbuf **controlp)
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
{
KSE Milestone 2 Note ALL MODULES MUST BE RECOMPILED make the kernel aware that there are smaller units of scheduling than the process. (but only allow one thread per process at this time). This is functionally equivalent to teh previousl -current except that there is a thread associated with each process. Sorry john! (your next MFC will be a doosie!) Reviewed by: peter@freebsd.org, dillon@freebsd.org X-MFC after: ha ha ha ha
2001-09-12 08:38:13 +00:00
struct thread *td = curthread; /* XXX */
Hopefully improve control message passing over Unix domain sockets. 1) Allow the sending of more than one control message at a time over a unix domain socket. This should cover the PR 29499. 2) This requires that unp_{ex,in}ternalize and unp_scan understand mbufs with more than one control message at a time. 3) Internalize and externalize used to work on the mbuf in-place. This made life quite complicated and the code for sizeof(int) < sizeof(file *) could end up doing the wrong thing. The patch always create a new mbuf/cluster now. This resulted in the change of the prototype for the domain externalise function. 4) You can now send SCM_TIMESTAMP messages. 5) Always use CMSG_DATA(cm) to determine the start where the data in unp_{ex,in}ternalize. It was using ((struct cmsghdr *)cm + 1) in some places, which gives the wrong alignment on the alpha. (NetBSD made this fix some time ago). This results in an ABI change for discriptor passing and creds passing on the alpha. (Probably on the IA64 and Spare ports too). 6) Fix userland programs to use CMSG_* macros too. 7) Be more careful about freeing mbufs containing (file *)s. This is made possible by the prototype change of externalise. PR: 29499 MFC after: 6 weeks
2001-10-04 13:11:48 +00:00
struct cmsghdr *cm = mtod(control, struct cmsghdr *);
int i;
int *fdp;
struct file **rp;
struct file *fp;
void *data;
socklen_t clen = control->m_len, datalen;
int error, newfds;
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
int f;
Hopefully improve control message passing over Unix domain sockets. 1) Allow the sending of more than one control message at a time over a unix domain socket. This should cover the PR 29499. 2) This requires that unp_{ex,in}ternalize and unp_scan understand mbufs with more than one control message at a time. 3) Internalize and externalize used to work on the mbuf in-place. This made life quite complicated and the code for sizeof(int) < sizeof(file *) could end up doing the wrong thing. The patch always create a new mbuf/cluster now. This resulted in the change of the prototype for the domain externalise function. 4) You can now send SCM_TIMESTAMP messages. 5) Always use CMSG_DATA(cm) to determine the start where the data in unp_{ex,in}ternalize. It was using ((struct cmsghdr *)cm + 1) in some places, which gives the wrong alignment on the alpha. (NetBSD made this fix some time ago). This results in an ABI change for discriptor passing and creds passing on the alpha. (Probably on the IA64 and Spare ports too). 6) Fix userland programs to use CMSG_* macros too. 7) Be more careful about freeing mbufs containing (file *)s. This is made possible by the prototype change of externalise. PR: 29499 MFC after: 6 weeks
2001-10-04 13:11:48 +00:00
u_int newlen;
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
UNP_GLOBAL_UNLOCK_ASSERT();
Add UNP_UNLOCK_ASSERT() to asser that the UNIX domain socket subsystem lock is not held. Rather than annotating that the lock is released after calls to unp_detach() with a comment, annotate with an assertion. Assert that the UNIX domain socket subsystem lock is not held when unp_externalize() and unp_internalize() are called.
2004-08-19 01:45:16 +00:00
Hopefully improve control message passing over Unix domain sockets. 1) Allow the sending of more than one control message at a time over a unix domain socket. This should cover the PR 29499. 2) This requires that unp_{ex,in}ternalize and unp_scan understand mbufs with more than one control message at a time. 3) Internalize and externalize used to work on the mbuf in-place. This made life quite complicated and the code for sizeof(int) < sizeof(file *) could end up doing the wrong thing. The patch always create a new mbuf/cluster now. This resulted in the change of the prototype for the domain externalise function. 4) You can now send SCM_TIMESTAMP messages. 5) Always use CMSG_DATA(cm) to determine the start where the data in unp_{ex,in}ternalize. It was using ((struct cmsghdr *)cm + 1) in some places, which gives the wrong alignment on the alpha. (NetBSD made this fix some time ago). This results in an ABI change for discriptor passing and creds passing on the alpha. (Probably on the IA64 and Spare ports too). 6) Fix userland programs to use CMSG_* macros too. 7) Be more careful about freeing mbufs containing (file *)s. This is made possible by the prototype change of externalise. PR: 29499 MFC after: 6 weeks
2001-10-04 13:11:48 +00:00
error = 0;
if (controlp != NULL) /* controlp == NULL => free control messages */
*controlp = NULL;
while (cm != NULL) {
if (sizeof(*cm) > clen || cm->cmsg_len > clen) {
error = EINVAL;
break;
}
data = CMSG_DATA(cm);
datalen = (caddr_t)cm + cm->cmsg_len - (caddr_t)data;
if (cm->cmsg_level == SOL_SOCKET
&& cm->cmsg_type == SCM_RIGHTS) {
newfds = datalen / sizeof(struct file *);
rp = data;
s/discriptors/descriptors/
2003-03-23 19:41:34 +00:00
/* If we're not outputting the descriptors free them. */
Hopefully improve control message passing over Unix domain sockets. 1) Allow the sending of more than one control message at a time over a unix domain socket. This should cover the PR 29499. 2) This requires that unp_{ex,in}ternalize and unp_scan understand mbufs with more than one control message at a time. 3) Internalize and externalize used to work on the mbuf in-place. This made life quite complicated and the code for sizeof(int) < sizeof(file *) could end up doing the wrong thing. The patch always create a new mbuf/cluster now. This resulted in the change of the prototype for the domain externalise function. 4) You can now send SCM_TIMESTAMP messages. 5) Always use CMSG_DATA(cm) to determine the start where the data in unp_{ex,in}ternalize. It was using ((struct cmsghdr *)cm + 1) in some places, which gives the wrong alignment on the alpha. (NetBSD made this fix some time ago). This results in an ABI change for discriptor passing and creds passing on the alpha. (Probably on the IA64 and Spare ports too). 6) Fix userland programs to use CMSG_* macros too. 7) Be more careful about freeing mbufs containing (file *)s. This is made possible by the prototype change of externalise. PR: 29499 MFC after: 6 weeks
2001-10-04 13:11:48 +00:00
if (error || controlp == NULL) {
unp_freerights(rp, newfds);
goto next;
}
Replace custom file descriptor array sleep lock constructed using a mutex and flags with an sxlock. This leads to a significant and measurable performance improvement as a result of access to shared locking for frequent lookup operations, reduced general overhead, and reduced overhead in the event of contention. All of these are imported for threaded applications where simultaneous access to a shared file descriptor array occurs frequently. Kris has reported 2x-4x transaction rate improvements on 8-core MySQL benchmarks; smaller improvements can be expected for many workloads as a result of reduced overhead. - Generally eliminate the distinction between "fast" and regular acquisisition of the filedesc lock; the plan is that they will now all be fast. Change all locking instances to either shared or exclusive locks. - Correct a bug (pointed out by kib) in fdfree() where previously msleep() was called without the mutex held; sx_sleep() is now always called with the sxlock held exclusively. - Universally hold the struct file lock over changes to struct file, rather than the filedesc lock or no lock. Always update the f_ops field last. A further memory barrier is required here in the future (discussed with jhb). - Improve locking and reference management in linux_at(), which fails to properly acquire vnode references before using vnode pointers. Annotate improper use of vn_fullpath(), which will be replaced at a future date. In fcntl(), we conservatively acquire an exclusive lock, even though in some cases a shared lock may be sufficient, which should be revisited. The dropping of the filedesc lock in fdgrowtable() is no longer required as the sxlock can be held over the sleep operation; we should consider removing that (pointed out by attilio). Tested by: kris Discussed with: jhb, kris, attilio, jeff
2007-04-04 09:11:34 +00:00
FILEDESC_XLOCK(td->td_proc->p_fd);
Hopefully improve control message passing over Unix domain sockets. 1) Allow the sending of more than one control message at a time over a unix domain socket. This should cover the PR 29499. 2) This requires that unp_{ex,in}ternalize and unp_scan understand mbufs with more than one control message at a time. 3) Internalize and externalize used to work on the mbuf in-place. This made life quite complicated and the code for sizeof(int) < sizeof(file *) could end up doing the wrong thing. The patch always create a new mbuf/cluster now. This resulted in the change of the prototype for the domain externalise function. 4) You can now send SCM_TIMESTAMP messages. 5) Always use CMSG_DATA(cm) to determine the start where the data in unp_{ex,in}ternalize. It was using ((struct cmsghdr *)cm + 1) in some places, which gives the wrong alignment on the alpha. (NetBSD made this fix some time ago). This results in an ABI change for discriptor passing and creds passing on the alpha. (Probably on the IA64 and Spare ports too). 6) Fix userland programs to use CMSG_* macros too. 7) Be more careful about freeing mbufs containing (file *)s. This is made possible by the prototype change of externalise. PR: 29499 MFC after: 6 weeks
2001-10-04 13:11:48 +00:00
/* if the new FD's will not fit free them. */
if (!fdavail(td, newfds)) {
Replace custom file descriptor array sleep lock constructed using a mutex and flags with an sxlock. This leads to a significant and measurable performance improvement as a result of access to shared locking for frequent lookup operations, reduced general overhead, and reduced overhead in the event of contention. All of these are imported for threaded applications where simultaneous access to a shared file descriptor array occurs frequently. Kris has reported 2x-4x transaction rate improvements on 8-core MySQL benchmarks; smaller improvements can be expected for many workloads as a result of reduced overhead. - Generally eliminate the distinction between "fast" and regular acquisisition of the filedesc lock; the plan is that they will now all be fast. Change all locking instances to either shared or exclusive locks. - Correct a bug (pointed out by kib) in fdfree() where previously msleep() was called without the mutex held; sx_sleep() is now always called with the sxlock held exclusively. - Universally hold the struct file lock over changes to struct file, rather than the filedesc lock or no lock. Always update the f_ops field last. A further memory barrier is required here in the future (discussed with jhb). - Improve locking and reference management in linux_at(), which fails to properly acquire vnode references before using vnode pointers. Annotate improper use of vn_fullpath(), which will be replaced at a future date. In fcntl(), we conservatively acquire an exclusive lock, even though in some cases a shared lock may be sufficient, which should be revisited. The dropping of the filedesc lock in fdgrowtable() is no longer required as the sxlock can be held over the sleep operation; we should consider removing that (pointed out by attilio). Tested by: kris Discussed with: jhb, kris, attilio, jeff
2007-04-04 09:11:34 +00:00
FILEDESC_XUNLOCK(td->td_proc->p_fd);
Hopefully improve control message passing over Unix domain sockets. 1) Allow the sending of more than one control message at a time over a unix domain socket. This should cover the PR 29499. 2) This requires that unp_{ex,in}ternalize and unp_scan understand mbufs with more than one control message at a time. 3) Internalize and externalize used to work on the mbuf in-place. This made life quite complicated and the code for sizeof(int) < sizeof(file *) could end up doing the wrong thing. The patch always create a new mbuf/cluster now. This resulted in the change of the prototype for the domain externalise function. 4) You can now send SCM_TIMESTAMP messages. 5) Always use CMSG_DATA(cm) to determine the start where the data in unp_{ex,in}ternalize. It was using ((struct cmsghdr *)cm + 1) in some places, which gives the wrong alignment on the alpha. (NetBSD made this fix some time ago). This results in an ABI change for discriptor passing and creds passing on the alpha. (Probably on the IA64 and Spare ports too). 6) Fix userland programs to use CMSG_* macros too. 7) Be more careful about freeing mbufs containing (file *)s. This is made possible by the prototype change of externalise. PR: 29499 MFC after: 6 weeks
2001-10-04 13:11:48 +00:00
error = EMSGSIZE;
unp_freerights(rp, newfds);
goto next;
}
Enable SCM_RIGHTS on alpha. Allocate necessary buffer as conversion between int and struct file *. Approved by: jkh Submitted by: brian Reviewed by: bde, brian, peter
2000-03-09 15:15:27 +00:00
/*
Re-wrap and other minor formatting and punctuation fixes for UNIX domain socket comments.
2006-07-22 17:24:55 +00:00
* Now change each pointer to an fd in the global
* table to an integer that is the index to the local
* fd table entry that we set up to point to the
* global one we are transferring.
Enable SCM_RIGHTS on alpha. Allocate necessary buffer as conversion between int and struct file *. Approved by: jkh Submitted by: brian Reviewed by: bde, brian, peter
2000-03-09 15:15:27 +00:00
*/
Hopefully improve control message passing over Unix domain sockets. 1) Allow the sending of more than one control message at a time over a unix domain socket. This should cover the PR 29499. 2) This requires that unp_{ex,in}ternalize and unp_scan understand mbufs with more than one control message at a time. 3) Internalize and externalize used to work on the mbuf in-place. This made life quite complicated and the code for sizeof(int) < sizeof(file *) could end up doing the wrong thing. The patch always create a new mbuf/cluster now. This resulted in the change of the prototype for the domain externalise function. 4) You can now send SCM_TIMESTAMP messages. 5) Always use CMSG_DATA(cm) to determine the start where the data in unp_{ex,in}ternalize. It was using ((struct cmsghdr *)cm + 1) in some places, which gives the wrong alignment on the alpha. (NetBSD made this fix some time ago). This results in an ABI change for discriptor passing and creds passing on the alpha. (Probably on the IA64 and Spare ports too). 6) Fix userland programs to use CMSG_* macros too. 7) Be more careful about freeing mbufs containing (file *)s. This is made possible by the prototype change of externalise. PR: 29499 MFC after: 6 weeks
2001-10-04 13:11:48 +00:00
newlen = newfds * sizeof(int);
*controlp = sbcreatecontrol(NULL, newlen,
SCM_RIGHTS, SOL_SOCKET);
if (*controlp == NULL) {
Replace custom file descriptor array sleep lock constructed using a mutex and flags with an sxlock. This leads to a significant and measurable performance improvement as a result of access to shared locking for frequent lookup operations, reduced general overhead, and reduced overhead in the event of contention. All of these are imported for threaded applications where simultaneous access to a shared file descriptor array occurs frequently. Kris has reported 2x-4x transaction rate improvements on 8-core MySQL benchmarks; smaller improvements can be expected for many workloads as a result of reduced overhead. - Generally eliminate the distinction between "fast" and regular acquisisition of the filedesc lock; the plan is that they will now all be fast. Change all locking instances to either shared or exclusive locks. - Correct a bug (pointed out by kib) in fdfree() where previously msleep() was called without the mutex held; sx_sleep() is now always called with the sxlock held exclusively. - Universally hold the struct file lock over changes to struct file, rather than the filedesc lock or no lock. Always update the f_ops field last. A further memory barrier is required here in the future (discussed with jhb). - Improve locking and reference management in linux_at(), which fails to properly acquire vnode references before using vnode pointers. Annotate improper use of vn_fullpath(), which will be replaced at a future date. In fcntl(), we conservatively acquire an exclusive lock, even though in some cases a shared lock may be sufficient, which should be revisited. The dropping of the filedesc lock in fdgrowtable() is no longer required as the sxlock can be held over the sleep operation; we should consider removing that (pointed out by attilio). Tested by: kris Discussed with: jhb, kris, attilio, jeff
2007-04-04 09:11:34 +00:00
FILEDESC_XUNLOCK(td->td_proc->p_fd);
Hopefully improve control message passing over Unix domain sockets. 1) Allow the sending of more than one control message at a time over a unix domain socket. This should cover the PR 29499. 2) This requires that unp_{ex,in}ternalize and unp_scan understand mbufs with more than one control message at a time. 3) Internalize and externalize used to work on the mbuf in-place. This made life quite complicated and the code for sizeof(int) < sizeof(file *) could end up doing the wrong thing. The patch always create a new mbuf/cluster now. This resulted in the change of the prototype for the domain externalise function. 4) You can now send SCM_TIMESTAMP messages. 5) Always use CMSG_DATA(cm) to determine the start where the data in unp_{ex,in}ternalize. It was using ((struct cmsghdr *)cm + 1) in some places, which gives the wrong alignment on the alpha. (NetBSD made this fix some time ago). This results in an ABI change for discriptor passing and creds passing on the alpha. (Probably on the IA64 and Spare ports too). 6) Fix userland programs to use CMSG_* macros too. 7) Be more careful about freeing mbufs containing (file *)s. This is made possible by the prototype change of externalise. PR: 29499 MFC after: 6 weeks
2001-10-04 13:11:48 +00:00
error = E2BIG;
unp_freerights(rp, newfds);
goto next;
}
fdp = (int *)
CMSG_DATA(mtod(*controlp, struct cmsghdr *));
for (i = 0; i < newfds; i++) {
Restore correct semantics for F_DUPFD fcntl. This should fix the errors people have been getting with configure scripts.
2004-01-17 00:59:04 +00:00
if (fdalloc(td, 0, &f))
Hopefully improve control message passing over Unix domain sockets. 1) Allow the sending of more than one control message at a time over a unix domain socket. This should cover the PR 29499. 2) This requires that unp_{ex,in}ternalize and unp_scan understand mbufs with more than one control message at a time. 3) Internalize and externalize used to work on the mbuf in-place. This made life quite complicated and the code for sizeof(int) < sizeof(file *) could end up doing the wrong thing. The patch always create a new mbuf/cluster now. This resulted in the change of the prototype for the domain externalise function. 4) You can now send SCM_TIMESTAMP messages. 5) Always use CMSG_DATA(cm) to determine the start where the data in unp_{ex,in}ternalize. It was using ((struct cmsghdr *)cm + 1) in some places, which gives the wrong alignment on the alpha. (NetBSD made this fix some time ago). This results in an ABI change for discriptor passing and creds passing on the alpha. (Probably on the IA64 and Spare ports too). 6) Fix userland programs to use CMSG_* macros too. 7) Be more careful about freeing mbufs containing (file *)s. This is made possible by the prototype change of externalise. PR: 29499 MFC after: 6 weeks
2001-10-04 13:11:48 +00:00
panic("unp_externalize fdalloc failed");
fp = *rp++;
td->td_proc->p_fd->fd_ofiles[f] = fp;
SMP Lock struct file, filedesc and the global file list. Seigo Tanimura (tanimura) posted the initial delta. I've polished it quite a bit reducing the need for locking and adapting it for KSE. Locks: 1 mutex in each filedesc protects all the fields. protects "struct file" initialization, while a struct file is being changed from &badfileops -> &pipeops or something the filedesc should be locked. 1 mutex in each struct file protects the refcount fields. doesn't protect anything else. the flags used for garbage collection have been moved to f_gcflag which was the FILLER short, this doesn't need locking because the garbage collection is a single threaded container. could likely be made to use a pool mutex. 1 sx lock for the global filelist. struct file * fhold(struct file *fp); /* increments reference count on a file */ struct file * fhold_locked(struct file *fp); /* like fhold but expects file to locked */ struct file * ffind_hold(struct thread *, int fd); /* finds the struct file in thread, adds one reference and returns it unlocked */ struct file * ffind_lock(struct thread *, int fd); /* ffind_hold, but returns file locked */ I still have to smp-safe the fget cruft, I'll get to that asap.
2002-01-13 11:58:06 +00:00
FILE_LOCK(fp);
Hopefully improve control message passing over Unix domain sockets. 1) Allow the sending of more than one control message at a time over a unix domain socket. This should cover the PR 29499. 2) This requires that unp_{ex,in}ternalize and unp_scan understand mbufs with more than one control message at a time. 3) Internalize and externalize used to work on the mbuf in-place. This made life quite complicated and the code for sizeof(int) < sizeof(file *) could end up doing the wrong thing. The patch always create a new mbuf/cluster now. This resulted in the change of the prototype for the domain externalise function. 4) You can now send SCM_TIMESTAMP messages. 5) Always use CMSG_DATA(cm) to determine the start where the data in unp_{ex,in}ternalize. It was using ((struct cmsghdr *)cm + 1) in some places, which gives the wrong alignment on the alpha. (NetBSD made this fix some time ago). This results in an ABI change for discriptor passing and creds passing on the alpha. (Probably on the IA64 and Spare ports too). 6) Fix userland programs to use CMSG_* macros too. 7) Be more careful about freeing mbufs containing (file *)s. This is made possible by the prototype change of externalise. PR: 29499 MFC after: 6 weeks
2001-10-04 13:11:48 +00:00
fp->f_msgcount--;
SMP Lock struct file, filedesc and the global file list. Seigo Tanimura (tanimura) posted the initial delta. I've polished it quite a bit reducing the need for locking and adapting it for KSE. Locks: 1 mutex in each filedesc protects all the fields. protects "struct file" initialization, while a struct file is being changed from &badfileops -> &pipeops or something the filedesc should be locked. 1 mutex in each struct file protects the refcount fields. doesn't protect anything else. the flags used for garbage collection have been moved to f_gcflag which was the FILLER short, this doesn't need locking because the garbage collection is a single threaded container. could likely be made to use a pool mutex. 1 sx lock for the global filelist. struct file * fhold(struct file *fp); /* increments reference count on a file */ struct file * fhold_locked(struct file *fp); /* like fhold but expects file to locked */ struct file * ffind_hold(struct thread *, int fd); /* finds the struct file in thread, adds one reference and returns it unlocked */ struct file * ffind_lock(struct thread *, int fd); /* ffind_hold, but returns file locked */ I still have to smp-safe the fget cruft, I'll get to that asap.
2002-01-13 11:58:06 +00:00
FILE_UNLOCK(fp);
Hopefully improve control message passing over Unix domain sockets. 1) Allow the sending of more than one control message at a time over a unix domain socket. This should cover the PR 29499. 2) This requires that unp_{ex,in}ternalize and unp_scan understand mbufs with more than one control message at a time. 3) Internalize and externalize used to work on the mbuf in-place. This made life quite complicated and the code for sizeof(int) < sizeof(file *) could end up doing the wrong thing. The patch always create a new mbuf/cluster now. This resulted in the change of the prototype for the domain externalise function. 4) You can now send SCM_TIMESTAMP messages. 5) Always use CMSG_DATA(cm) to determine the start where the data in unp_{ex,in}ternalize. It was using ((struct cmsghdr *)cm + 1) in some places, which gives the wrong alignment on the alpha. (NetBSD made this fix some time ago). This results in an ABI change for discriptor passing and creds passing on the alpha. (Probably on the IA64 and Spare ports too). 6) Fix userland programs to use CMSG_* macros too. 7) Be more careful about freeing mbufs containing (file *)s. This is made possible by the prototype change of externalise. PR: 29499 MFC after: 6 weeks
2001-10-04 13:11:48 +00:00
unp_rights--;
*fdp++ = f;
}
Replace custom file descriptor array sleep lock constructed using a mutex and flags with an sxlock. This leads to a significant and measurable performance improvement as a result of access to shared locking for frequent lookup operations, reduced general overhead, and reduced overhead in the event of contention. All of these are imported for threaded applications where simultaneous access to a shared file descriptor array occurs frequently. Kris has reported 2x-4x transaction rate improvements on 8-core MySQL benchmarks; smaller improvements can be expected for many workloads as a result of reduced overhead. - Generally eliminate the distinction between "fast" and regular acquisisition of the filedesc lock; the plan is that they will now all be fast. Change all locking instances to either shared or exclusive locks. - Correct a bug (pointed out by kib) in fdfree() where previously msleep() was called without the mutex held; sx_sleep() is now always called with the sxlock held exclusively. - Universally hold the struct file lock over changes to struct file, rather than the filedesc lock or no lock. Always update the f_ops field last. A further memory barrier is required here in the future (discussed with jhb). - Improve locking and reference management in linux_at(), which fails to properly acquire vnode references before using vnode pointers. Annotate improper use of vn_fullpath(), which will be replaced at a future date. In fcntl(), we conservatively acquire an exclusive lock, even though in some cases a shared lock may be sufficient, which should be revisited. The dropping of the filedesc lock in fdgrowtable() is no longer required as the sxlock can be held over the sleep operation; we should consider removing that (pointed out by attilio). Tested by: kris Discussed with: jhb, kris, attilio, jeff
2007-04-04 09:11:34 +00:00
FILEDESC_XUNLOCK(td->td_proc->p_fd);
Re-wrap and other minor formatting and punctuation fixes for UNIX domain socket comments.
2006-07-22 17:24:55 +00:00
} else {
/* We can just copy anything else across. */
Hopefully improve control message passing over Unix domain sockets. 1) Allow the sending of more than one control message at a time over a unix domain socket. This should cover the PR 29499. 2) This requires that unp_{ex,in}ternalize and unp_scan understand mbufs with more than one control message at a time. 3) Internalize and externalize used to work on the mbuf in-place. This made life quite complicated and the code for sizeof(int) < sizeof(file *) could end up doing the wrong thing. The patch always create a new mbuf/cluster now. This resulted in the change of the prototype for the domain externalise function. 4) You can now send SCM_TIMESTAMP messages. 5) Always use CMSG_DATA(cm) to determine the start where the data in unp_{ex,in}ternalize. It was using ((struct cmsghdr *)cm + 1) in some places, which gives the wrong alignment on the alpha. (NetBSD made this fix some time ago). This results in an ABI change for discriptor passing and creds passing on the alpha. (Probably on the IA64 and Spare ports too). 6) Fix userland programs to use CMSG_* macros too. 7) Be more careful about freeing mbufs containing (file *)s. This is made possible by the prototype change of externalise. PR: 29499 MFC after: 6 weeks
2001-10-04 13:11:48 +00:00
if (error || controlp == NULL)
goto next;
*controlp = sbcreatecontrol(NULL, datalen,
cm->cmsg_type, cm->cmsg_level);
if (*controlp == NULL) {
error = ENOBUFS;
goto next;
}
bcopy(data,
CMSG_DATA(mtod(*controlp, struct cmsghdr *)),
datalen);
Enable SCM_RIGHTS on alpha. Allocate necessary buffer as conversion between int and struct file *. Approved by: jkh Submitted by: brian Reviewed by: bde, brian, peter
2000-03-09 15:15:27 +00:00
}
Hopefully improve control message passing over Unix domain sockets. 1) Allow the sending of more than one control message at a time over a unix domain socket. This should cover the PR 29499. 2) This requires that unp_{ex,in}ternalize and unp_scan understand mbufs with more than one control message at a time. 3) Internalize and externalize used to work on the mbuf in-place. This made life quite complicated and the code for sizeof(int) < sizeof(file *) could end up doing the wrong thing. The patch always create a new mbuf/cluster now. This resulted in the change of the prototype for the domain externalise function. 4) You can now send SCM_TIMESTAMP messages. 5) Always use CMSG_DATA(cm) to determine the start where the data in unp_{ex,in}ternalize. It was using ((struct cmsghdr *)cm + 1) in some places, which gives the wrong alignment on the alpha. (NetBSD made this fix some time ago). This results in an ABI change for discriptor passing and creds passing on the alpha. (Probably on the IA64 and Spare ports too). 6) Fix userland programs to use CMSG_* macros too. 7) Be more careful about freeing mbufs containing (file *)s. This is made possible by the prototype change of externalise. PR: 29499 MFC after: 6 weeks
2001-10-04 13:11:48 +00:00
controlp = &(*controlp)->m_next;
next:
if (CMSG_SPACE(datalen) < clen) {
clen -= CMSG_SPACE(datalen);
cm = (struct cmsghdr *)
((caddr_t)cm + CMSG_SPACE(datalen));
} else {
clen = 0;
cm = NULL;
Enable SCM_RIGHTS on alpha. Allocate necessary buffer as conversion between int and struct file *. Approved by: jkh Submitted by: brian Reviewed by: bde, brian, peter
2000-03-09 15:15:27 +00:00
}
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
}
Enable SCM_RIGHTS on alpha. Allocate necessary buffer as conversion between int and struct file *. Approved by: jkh Submitted by: brian Reviewed by: bde, brian, peter
2000-03-09 15:15:27 +00:00
Hopefully improve control message passing over Unix domain sockets. 1) Allow the sending of more than one control message at a time over a unix domain socket. This should cover the PR 29499. 2) This requires that unp_{ex,in}ternalize and unp_scan understand mbufs with more than one control message at a time. 3) Internalize and externalize used to work on the mbuf in-place. This made life quite complicated and the code for sizeof(int) < sizeof(file *) could end up doing the wrong thing. The patch always create a new mbuf/cluster now. This resulted in the change of the prototype for the domain externalise function. 4) You can now send SCM_TIMESTAMP messages. 5) Always use CMSG_DATA(cm) to determine the start where the data in unp_{ex,in}ternalize. It was using ((struct cmsghdr *)cm + 1) in some places, which gives the wrong alignment on the alpha. (NetBSD made this fix some time ago). This results in an ABI change for discriptor passing and creds passing on the alpha. (Probably on the IA64 and Spare ports too). 6) Fix userland programs to use CMSG_* macros too. 7) Be more careful about freeing mbufs containing (file *)s. This is made possible by the prototype change of externalise. PR: 29499 MFC after: 6 weeks
2001-10-04 13:11:48 +00:00
m_freem(control);
return (error);
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
}
Allow for nmbclusters and maxsockets to be increased via sysctl. An eventhandler is used to update all the various zones that depend on these values.
2006-04-21 09:25:40 +00:00
static void
unp_zone_change(void *tag)
{
uma_zone_set_max(unp_zone, maxsockets);
}
Convert socket structures to be type-stable and add a version number. Define a parameter which indicates the maximum number of sockets in a system, and use this to size the zone allocators used for sockets and for certain PCBs. Convert PF_LOCAL PCB structures to be type-stable and add a version number. Define an external format for infomation about socket structures and use it in several places. Define a mechanism to get all PF_LOCAL and PF_INET PCB lists through sysctl(3) without blocking network interrupts for an unreasonable length of time. This probably still has some bugs and/or race conditions, but it seems to work well enough on my machines. It is now possible for `netstat' to get almost all of its information via the sysctl(3) interface rather than reading kmem (changes to follow).
1998-05-15 20:11:40 +00:00
void
unp_init(void)
{
Re-wrap and other minor formatting and punctuation fixes for UNIX domain socket comments.
2006-07-22 17:24:55 +00:00
Remove references to vm_zone.h and switch over to the new uma API.
2002-03-20 04:11:52 +00:00
unp_zone = uma_zcreate("unpcb", sizeof(struct unpcb), NULL, NULL,
- Close a race between enumerating UNIX domain socket pcb structures via sysctl and socket teardown by adding a reference count to the UNIX domain pcb object and fixing the sysctl that enumerates unpcbs to grab a reference on each unpcb while it builds the list to copy out to userland. - Close a race between UNIX domain pcb garbage collection (unp_gc()) and file descriptor teardown (fdrop()) by adding a new garbage collection flag FWAIT. unp_gc() sets FWAIT while it walks the message buffers in a UNIX domain socket looking for nested file descriptor references and clears the flag when it is finished. fdrop() checks to see if the flag is set on a file descriptor whose refcount just dropped to 0 and waits for unp_gc() to clear the flag before completely destroying the file descriptor. MFC after: 1 week Reviewed by: rwatson Submitted by: ups Hopefully makes the panics go away: mx1
2007-01-05 19:59:46 +00:00
NULL, NULL, UMA_ALIGN_PTR, 0);
Prefer NULL to 0 when testing and assigning pointer values.
2004-03-30 02:16:25 +00:00
if (unp_zone == NULL)
Convert socket structures to be type-stable and add a version number. Define a parameter which indicates the maximum number of sockets in a system, and use this to size the zone allocators used for sockets and for certain PCBs. Convert PF_LOCAL PCB structures to be type-stable and add a version number. Define an external format for infomation about socket structures and use it in several places. Define a mechanism to get all PF_LOCAL and PF_INET PCB lists through sysctl(3) without blocking network interrupts for an unreasonable length of time. This probably still has some bugs and/or race conditions, but it seems to work well enough on my machines. It is now possible for `netstat' to get almost all of its information via the sysctl(3) interface rather than reading kmem (changes to follow).
1998-05-15 20:11:40 +00:00
panic("unp_init");
Allow for nmbclusters and maxsockets to be increased via sysctl. An eventhandler is used to update all the various zones that depend on these values.
2006-04-21 09:25:40 +00:00
uma_zone_set_max(unp_zone, maxsockets);
EVENTHANDLER_REGISTER(maxsockets_change, unp_zone_change,
NULL, EVENTHANDLER_PRI_ANY);
Convert socket structures to be type-stable and add a version number. Define a parameter which indicates the maximum number of sockets in a system, and use this to size the zone allocators used for sockets and for certain PCBs. Convert PF_LOCAL PCB structures to be type-stable and add a version number. Define an external format for infomation about socket structures and use it in several places. Define a mechanism to get all PF_LOCAL and PF_INET PCB lists through sysctl(3) without blocking network interrupts for an unreasonable length of time. This probably still has some bugs and/or race conditions, but it seems to work well enough on my machines. It is now possible for `netstat' to get almost all of its information via the sysctl(3) interface rather than reading kmem (changes to follow).
1998-05-15 20:11:40 +00:00
LIST_INIT(&unp_dhead);
LIST_INIT(&unp_shead);
Correct a number of serious and closely related bugs in the UNIX domain socket file descriptor garbage collection code, which is intended to detect and clear cycles of orphaned file descriptors that are "in-flight" in a socket when that socket is closed before they are received. The algorithm present was both run at poor times (resulting in recursion and reentrance), and also buggy in the presence of parallelism. In order to fix these problems, make the following changes: - When there are in-flight sockets and a UNIX domain socket is destroyed, asynchronously schedule the garbage collector, rather than running it synchronously in the current context. This avoids lock order issues when the garbage collection code reenters the UNIX domain socket code, avoiding lock order reversals, deadlocks, etc. Run the code asynchronously in a task queue. - In the garbage collector, when skipping file descriptors that have entered a closing state (i.e., have f_count == 0), re-test the FDEFER flag, and decrement unp_defer. As file descriptors can now transition to a closed state, while the garbage collector is running, it is no longer the case that unp_defer will remain an accurate count of deferred sockets in the mark portion of the GC algorithm. Otherwise, the garbage collector will loop waiting waiting for unp_defer to reach zero, which it will never do as it is skipping file descriptors that were marked in an earlier pass, but now closed. - Acquire the UNIX domain socket subsystem lock in unp_discard() when modifying the unp_rights counter, or a read/write race is risked with other threads also manipulating the counter. While here: - Remove #if 0'd code regarding acquiring the socket buffer sleep lock in the garbage collector, this is not required as we are able to use the socket buffer receive lock to protect scanning the receive buffer for in-flight file descriptors on the socket buffer. - Annotate that the description of the garbage collector implementation is increasingly inaccurate and needs to be updated. - Add counters of the number of deferred garbage collections and recycled file descriptors. This will be removed and is here temporarily for debugging purposes. With these changes in place, the unp_passfd regression test now appears to be passed consistently on UP and SMP systems for extended runs, whereas before it hung quickly or panicked, depending on which bug was triggered. Reported by: Philip Kizer <pckizer at nostrum dot com> MFC after: 2 weeks
2005-11-10 16:06:04 +00:00
TASK_INIT(&unp_gc_task, 0, unp_gc, NULL);
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
UNP_GLOBAL_LOCK_INIT();
Convert socket structures to be type-stable and add a version number. Define a parameter which indicates the maximum number of sockets in a system, and use this to size the zone allocators used for sockets and for certain PCBs. Convert PF_LOCAL PCB structures to be type-stable and add a version number. Define an external format for infomation about socket structures and use it in several places. Define a mechanism to get all PF_LOCAL and PF_INET PCB lists through sysctl(3) without blocking network interrupts for an unreasonable length of time. This probably still has some bugs and/or race conditions, but it seems to work well enough on my machines. It is now possible for `netstat' to get almost all of its information via the sysctl(3) interface rather than reading kmem (changes to follow).
1998-05-15 20:11:40 +00:00
}
Another mega commit to staticize things.
1995-12-14 09:55:16 +00:00
static int
style(9)-ize function headers, remove use of 'register'. MFC after: 3 days
2005-02-20 23:22:13 +00:00
unp_internalize(struct mbuf **controlp, struct thread *td)
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
{
Hopefully improve control message passing over Unix domain sockets. 1) Allow the sending of more than one control message at a time over a unix domain socket. This should cover the PR 29499. 2) This requires that unp_{ex,in}ternalize and unp_scan understand mbufs with more than one control message at a time. 3) Internalize and externalize used to work on the mbuf in-place. This made life quite complicated and the code for sizeof(int) < sizeof(file *) could end up doing the wrong thing. The patch always create a new mbuf/cluster now. This resulted in the change of the prototype for the domain externalise function. 4) You can now send SCM_TIMESTAMP messages. 5) Always use CMSG_DATA(cm) to determine the start where the data in unp_{ex,in}ternalize. It was using ((struct cmsghdr *)cm + 1) in some places, which gives the wrong alignment on the alpha. (NetBSD made this fix some time ago). This results in an ABI change for discriptor passing and creds passing on the alpha. (Probably on the IA64 and Spare ports too). 6) Fix userland programs to use CMSG_* macros too. 7) Be more careful about freeing mbufs containing (file *)s. This is made possible by the prototype change of externalise. PR: 29499 MFC after: 6 weeks
2001-10-04 13:11:48 +00:00
struct mbuf *control = *controlp;
KSE Milestone 2 Note ALL MODULES MUST BE RECOMPILED make the kernel aware that there are smaller units of scheduling than the process. (but only allow one thread per process at this time). This is functionally equivalent to teh previousl -current except that there is a thread associated with each process. Sorry john! (your next MFC will be a doosie!) Reviewed by: peter@freebsd.org, dillon@freebsd.org X-MFC after: ha ha ha ha
2001-09-12 08:38:13 +00:00
struct proc *p = td->td_proc;
Enable SCM_RIGHTS on alpha. Allocate necessary buffer as conversion between int and struct file *. Approved by: jkh Submitted by: brian Reviewed by: bde, brian, peter
2000-03-09 15:15:27 +00:00
struct filedesc *fdescp = p->p_fd;
Hopefully improve control message passing over Unix domain sockets. 1) Allow the sending of more than one control message at a time over a unix domain socket. This should cover the PR 29499. 2) This requires that unp_{ex,in}ternalize and unp_scan understand mbufs with more than one control message at a time. 3) Internalize and externalize used to work on the mbuf in-place. This made life quite complicated and the code for sizeof(int) < sizeof(file *) could end up doing the wrong thing. The patch always create a new mbuf/cluster now. This resulted in the change of the prototype for the domain externalise function. 4) You can now send SCM_TIMESTAMP messages. 5) Always use CMSG_DATA(cm) to determine the start where the data in unp_{ex,in}ternalize. It was using ((struct cmsghdr *)cm + 1) in some places, which gives the wrong alignment on the alpha. (NetBSD made this fix some time ago). This results in an ABI change for discriptor passing and creds passing on the alpha. (Probably on the IA64 and Spare ports too). 6) Fix userland programs to use CMSG_* macros too. 7) Be more careful about freeing mbufs containing (file *)s. This is made possible by the prototype change of externalise. PR: 29499 MFC after: 6 weeks
2001-10-04 13:11:48 +00:00
struct cmsghdr *cm = mtod(control, struct cmsghdr *);
struct cmsgcred *cmcred;
struct file **rp;
struct file *fp;
struct timeval *tv;
int i, fd, *fdp;
void *data;
socklen_t clen = control->m_len, datalen;
int error, oldfds;
Enable SCM_RIGHTS on alpha. Allocate necessary buffer as conversion between int and struct file *. Approved by: jkh Submitted by: brian Reviewed by: bde, brian, peter
2000-03-09 15:15:27 +00:00
u_int newlen;
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
UNP_GLOBAL_UNLOCK_ASSERT();
Add UNP_UNLOCK_ASSERT() to asser that the UNIX domain socket subsystem lock is not held. Rather than annotating that the lock is released after calls to unp_detach() with a comment, annotate with an assertion. Assert that the UNIX domain socket subsystem lock is not held when unp_externalize() and unp_internalize() are called.
2004-08-19 01:45:16 +00:00
Hopefully improve control message passing over Unix domain sockets. 1) Allow the sending of more than one control message at a time over a unix domain socket. This should cover the PR 29499. 2) This requires that unp_{ex,in}ternalize and unp_scan understand mbufs with more than one control message at a time. 3) Internalize and externalize used to work on the mbuf in-place. This made life quite complicated and the code for sizeof(int) < sizeof(file *) could end up doing the wrong thing. The patch always create a new mbuf/cluster now. This resulted in the change of the prototype for the domain externalise function. 4) You can now send SCM_TIMESTAMP messages. 5) Always use CMSG_DATA(cm) to determine the start where the data in unp_{ex,in}ternalize. It was using ((struct cmsghdr *)cm + 1) in some places, which gives the wrong alignment on the alpha. (NetBSD made this fix some time ago). This results in an ABI change for discriptor passing and creds passing on the alpha. (Probably on the IA64 and Spare ports too). 6) Fix userland programs to use CMSG_* macros too. 7) Be more careful about freeing mbufs containing (file *)s. This is made possible by the prototype change of externalise. PR: 29499 MFC after: 6 weeks
2001-10-04 13:11:48 +00:00
error = 0;
*controlp = NULL;
Add support to sendmsg()/recvmsg() for passing credentials between processes using AF_LOCAL sockets. This hack is going to be used with Secure RPC to duplicate a feature of STREAMS which has no real counterpart in sockets (with STREAMS/TLI, you can apparently use t_getinfo() to learn UID of a local process on the other side of a transport endpoint). What happens is this: the client sets up a sendmsg() call with ancillary data using the SCM_CREDS socket-level control message type. It does not need to fill in the structure. When the kernel notices the data, unp_internalize() fills in the cmesgcred structure with the sending process' credentials (UID, EUID, GID, and ancillary groups). This data is later delivered to the receiving process. The receiver can then perform the follwing tests: - Did the client send ancillary data? o Yes, proceed. o No, refuse to authenticate the client. - The the client send data of type SCM_CREDS? o Yes, proceed. o No, refuse to authenticate the client. - Is the cmsgcred structure the right size? o Yes, proceed. o No, signal a possible error. The receiver can now inspect the credential information and use it to authenticate the client.
1997-03-21 16:12:32 +00:00
Hopefully improve control message passing over Unix domain sockets. 1) Allow the sending of more than one control message at a time over a unix domain socket. This should cover the PR 29499. 2) This requires that unp_{ex,in}ternalize and unp_scan understand mbufs with more than one control message at a time. 3) Internalize and externalize used to work on the mbuf in-place. This made life quite complicated and the code for sizeof(int) < sizeof(file *) could end up doing the wrong thing. The patch always create a new mbuf/cluster now. This resulted in the change of the prototype for the domain externalise function. 4) You can now send SCM_TIMESTAMP messages. 5) Always use CMSG_DATA(cm) to determine the start where the data in unp_{ex,in}ternalize. It was using ((struct cmsghdr *)cm + 1) in some places, which gives the wrong alignment on the alpha. (NetBSD made this fix some time ago). This results in an ABI change for discriptor passing and creds passing on the alpha. (Probably on the IA64 and Spare ports too). 6) Fix userland programs to use CMSG_* macros too. 7) Be more careful about freeing mbufs containing (file *)s. This is made possible by the prototype change of externalise. PR: 29499 MFC after: 6 weeks
2001-10-04 13:11:48 +00:00
while (cm != NULL) {
if (sizeof(*cm) > clen || cm->cmsg_level != SOL_SOCKET
|| cm->cmsg_len > clen) {
error = EINVAL;
goto out;
}
data = CMSG_DATA(cm);
datalen = (caddr_t)cm + cm->cmsg_len - (caddr_t)data;
switch (cm->cmsg_type) {
/*
* Fill in credential information.
*/
case SCM_CREDS:
*controlp = sbcreatecontrol(NULL, sizeof(*cmcred),
SCM_CREDS, SOL_SOCKET);
if (*controlp == NULL) {
error = ENOBUFS;
goto out;
}
cmcred = (struct cmsgcred *)
CMSG_DATA(mtod(*controlp, struct cmsghdr *));
cmcred->cmcred_pid = p->p_pid;
Simple p_ucred -> td_ucred changes to start using the per-thread ucred reference.
2002-02-27 18:32:23 +00:00
cmcred->cmcred_uid = td->td_ucred->cr_ruid;
cmcred->cmcred_gid = td->td_ucred->cr_rgid;
cmcred->cmcred_euid = td->td_ucred->cr_uid;
cmcred->cmcred_ngroups = MIN(td->td_ucred->cr_ngroups,
Add support to sendmsg()/recvmsg() for passing credentials between processes using AF_LOCAL sockets. This hack is going to be used with Secure RPC to duplicate a feature of STREAMS which has no real counterpart in sockets (with STREAMS/TLI, you can apparently use t_getinfo() to learn UID of a local process on the other side of a transport endpoint). What happens is this: the client sets up a sendmsg() call with ancillary data using the SCM_CREDS socket-level control message type. It does not need to fill in the structure. When the kernel notices the data, unp_internalize() fills in the cmesgcred structure with the sending process' credentials (UID, EUID, GID, and ancillary groups). This data is later delivered to the receiving process. The receiver can then perform the follwing tests: - Did the client send ancillary data? o Yes, proceed. o No, refuse to authenticate the client. - The the client send data of type SCM_CREDS? o Yes, proceed. o No, refuse to authenticate the client. - Is the cmsgcred structure the right size? o Yes, proceed. o No, signal a possible error. The receiver can now inspect the credential information and use it to authenticate the client.
1997-03-21 16:12:32 +00:00
CMGROUP_MAX);
Hopefully improve control message passing over Unix domain sockets. 1) Allow the sending of more than one control message at a time over a unix domain socket. This should cover the PR 29499. 2) This requires that unp_{ex,in}ternalize and unp_scan understand mbufs with more than one control message at a time. 3) Internalize and externalize used to work on the mbuf in-place. This made life quite complicated and the code for sizeof(int) < sizeof(file *) could end up doing the wrong thing. The patch always create a new mbuf/cluster now. This resulted in the change of the prototype for the domain externalise function. 4) You can now send SCM_TIMESTAMP messages. 5) Always use CMSG_DATA(cm) to determine the start where the data in unp_{ex,in}ternalize. It was using ((struct cmsghdr *)cm + 1) in some places, which gives the wrong alignment on the alpha. (NetBSD made this fix some time ago). This results in an ABI change for discriptor passing and creds passing on the alpha. (Probably on the IA64 and Spare ports too). 6) Fix userland programs to use CMSG_* macros too. 7) Be more careful about freeing mbufs containing (file *)s. This is made possible by the prototype change of externalise. PR: 29499 MFC after: 6 weeks
2001-10-04 13:11:48 +00:00
for (i = 0; i < cmcred->cmcred_ngroups; i++)
cmcred->cmcred_groups[i] =
Simple p_ucred -> td_ucred changes to start using the per-thread ucred reference.
2002-02-27 18:32:23 +00:00
td->td_ucred->cr_groups[i];
Hopefully improve control message passing over Unix domain sockets. 1) Allow the sending of more than one control message at a time over a unix domain socket. This should cover the PR 29499. 2) This requires that unp_{ex,in}ternalize and unp_scan understand mbufs with more than one control message at a time. 3) Internalize and externalize used to work on the mbuf in-place. This made life quite complicated and the code for sizeof(int) < sizeof(file *) could end up doing the wrong thing. The patch always create a new mbuf/cluster now. This resulted in the change of the prototype for the domain externalise function. 4) You can now send SCM_TIMESTAMP messages. 5) Always use CMSG_DATA(cm) to determine the start where the data in unp_{ex,in}ternalize. It was using ((struct cmsghdr *)cm + 1) in some places, which gives the wrong alignment on the alpha. (NetBSD made this fix some time ago). This results in an ABI change for discriptor passing and creds passing on the alpha. (Probably on the IA64 and Spare ports too). 6) Fix userland programs to use CMSG_* macros too. 7) Be more careful about freeing mbufs containing (file *)s. This is made possible by the prototype change of externalise. PR: 29499 MFC after: 6 weeks
2001-10-04 13:11:48 +00:00
break;
Add support to sendmsg()/recvmsg() for passing credentials between processes using AF_LOCAL sockets. This hack is going to be used with Secure RPC to duplicate a feature of STREAMS which has no real counterpart in sockets (with STREAMS/TLI, you can apparently use t_getinfo() to learn UID of a local process on the other side of a transport endpoint). What happens is this: the client sets up a sendmsg() call with ancillary data using the SCM_CREDS socket-level control message type. It does not need to fill in the structure. When the kernel notices the data, unp_internalize() fills in the cmesgcred structure with the sending process' credentials (UID, EUID, GID, and ancillary groups). This data is later delivered to the receiving process. The receiver can then perform the follwing tests: - Did the client send ancillary data? o Yes, proceed. o No, refuse to authenticate the client. - The the client send data of type SCM_CREDS? o Yes, proceed. o No, refuse to authenticate the client. - Is the cmsgcred structure the right size? o Yes, proceed. o No, signal a possible error. The receiver can now inspect the credential information and use it to authenticate the client.
1997-03-21 16:12:32 +00:00
Hopefully improve control message passing over Unix domain sockets. 1) Allow the sending of more than one control message at a time over a unix domain socket. This should cover the PR 29499. 2) This requires that unp_{ex,in}ternalize and unp_scan understand mbufs with more than one control message at a time. 3) Internalize and externalize used to work on the mbuf in-place. This made life quite complicated and the code for sizeof(int) < sizeof(file *) could end up doing the wrong thing. The patch always create a new mbuf/cluster now. This resulted in the change of the prototype for the domain externalise function. 4) You can now send SCM_TIMESTAMP messages. 5) Always use CMSG_DATA(cm) to determine the start where the data in unp_{ex,in}ternalize. It was using ((struct cmsghdr *)cm + 1) in some places, which gives the wrong alignment on the alpha. (NetBSD made this fix some time ago). This results in an ABI change for discriptor passing and creds passing on the alpha. (Probably on the IA64 and Spare ports too). 6) Fix userland programs to use CMSG_* macros too. 7) Be more careful about freeing mbufs containing (file *)s. This is made possible by the prototype change of externalise. PR: 29499 MFC after: 6 weeks
2001-10-04 13:11:48 +00:00
case SCM_RIGHTS:
oldfds = datalen / sizeof (int);
/*
Re-wrap and other minor formatting and punctuation fixes for UNIX domain socket comments.
2006-07-22 17:24:55 +00:00
* Check that all the FDs passed in refer to legal
* files. If not, reject the entire operation.
Hopefully improve control message passing over Unix domain sockets. 1) Allow the sending of more than one control message at a time over a unix domain socket. This should cover the PR 29499. 2) This requires that unp_{ex,in}ternalize and unp_scan understand mbufs with more than one control message at a time. 3) Internalize and externalize used to work on the mbuf in-place. This made life quite complicated and the code for sizeof(int) < sizeof(file *) could end up doing the wrong thing. The patch always create a new mbuf/cluster now. This resulted in the change of the prototype for the domain externalise function. 4) You can now send SCM_TIMESTAMP messages. 5) Always use CMSG_DATA(cm) to determine the start where the data in unp_{ex,in}ternalize. It was using ((struct cmsghdr *)cm + 1) in some places, which gives the wrong alignment on the alpha. (NetBSD made this fix some time ago). This results in an ABI change for discriptor passing and creds passing on the alpha. (Probably on the IA64 and Spare ports too). 6) Fix userland programs to use CMSG_* macros too. 7) Be more careful about freeing mbufs containing (file *)s. This is made possible by the prototype change of externalise. PR: 29499 MFC after: 6 weeks
2001-10-04 13:11:48 +00:00
*/
fdp = data;
Replace custom file descriptor array sleep lock constructed using a mutex and flags with an sxlock. This leads to a significant and measurable performance improvement as a result of access to shared locking for frequent lookup operations, reduced general overhead, and reduced overhead in the event of contention. All of these are imported for threaded applications where simultaneous access to a shared file descriptor array occurs frequently. Kris has reported 2x-4x transaction rate improvements on 8-core MySQL benchmarks; smaller improvements can be expected for many workloads as a result of reduced overhead. - Generally eliminate the distinction between "fast" and regular acquisisition of the filedesc lock; the plan is that they will now all be fast. Change all locking instances to either shared or exclusive locks. - Correct a bug (pointed out by kib) in fdfree() where previously msleep() was called without the mutex held; sx_sleep() is now always called with the sxlock held exclusively. - Universally hold the struct file lock over changes to struct file, rather than the filedesc lock or no lock. Always update the f_ops field last. A further memory barrier is required here in the future (discussed with jhb). - Improve locking and reference management in linux_at(), which fails to properly acquire vnode references before using vnode pointers. Annotate improper use of vn_fullpath(), which will be replaced at a future date. In fcntl(), we conservatively acquire an exclusive lock, even though in some cases a shared lock may be sufficient, which should be revisited. The dropping of the filedesc lock in fdgrowtable() is no longer required as the sxlock can be held over the sleep operation; we should consider removing that (pointed out by attilio). Tested by: kris Discussed with: jhb, kris, attilio, jeff
2007-04-04 09:11:34 +00:00
FILEDESC_SLOCK(fdescp);
Hopefully improve control message passing over Unix domain sockets. 1) Allow the sending of more than one control message at a time over a unix domain socket. This should cover the PR 29499. 2) This requires that unp_{ex,in}ternalize and unp_scan understand mbufs with more than one control message at a time. 3) Internalize and externalize used to work on the mbuf in-place. This made life quite complicated and the code for sizeof(int) < sizeof(file *) could end up doing the wrong thing. The patch always create a new mbuf/cluster now. This resulted in the change of the prototype for the domain externalise function. 4) You can now send SCM_TIMESTAMP messages. 5) Always use CMSG_DATA(cm) to determine the start where the data in unp_{ex,in}ternalize. It was using ((struct cmsghdr *)cm + 1) in some places, which gives the wrong alignment on the alpha. (NetBSD made this fix some time ago). This results in an ABI change for discriptor passing and creds passing on the alpha. (Probably on the IA64 and Spare ports too). 6) Fix userland programs to use CMSG_* macros too. 7) Be more careful about freeing mbufs containing (file *)s. This is made possible by the prototype change of externalise. PR: 29499 MFC after: 6 weeks
2001-10-04 13:11:48 +00:00
for (i = 0; i < oldfds; i++) {
fd = *fdp++;
if ((unsigned)fd >= fdescp->fd_nfiles ||
fdescp->fd_ofiles[fd] == NULL) {
Replace custom file descriptor array sleep lock constructed using a mutex and flags with an sxlock. This leads to a significant and measurable performance improvement as a result of access to shared locking for frequent lookup operations, reduced general overhead, and reduced overhead in the event of contention. All of these are imported for threaded applications where simultaneous access to a shared file descriptor array occurs frequently. Kris has reported 2x-4x transaction rate improvements on 8-core MySQL benchmarks; smaller improvements can be expected for many workloads as a result of reduced overhead. - Generally eliminate the distinction between "fast" and regular acquisisition of the filedesc lock; the plan is that they will now all be fast. Change all locking instances to either shared or exclusive locks. - Correct a bug (pointed out by kib) in fdfree() where previously msleep() was called without the mutex held; sx_sleep() is now always called with the sxlock held exclusively. - Universally hold the struct file lock over changes to struct file, rather than the filedesc lock or no lock. Always update the f_ops field last. A further memory barrier is required here in the future (discussed with jhb). - Improve locking and reference management in linux_at(), which fails to properly acquire vnode references before using vnode pointers. Annotate improper use of vn_fullpath(), which will be replaced at a future date. In fcntl(), we conservatively acquire an exclusive lock, even though in some cases a shared lock may be sufficient, which should be revisited. The dropping of the filedesc lock in fdgrowtable() is no longer required as the sxlock can be held over the sleep operation; we should consider removing that (pointed out by attilio). Tested by: kris Discussed with: jhb, kris, attilio, jeff
2007-04-04 09:11:34 +00:00
FILEDESC_SUNLOCK(fdescp);
Hopefully improve control message passing over Unix domain sockets. 1) Allow the sending of more than one control message at a time over a unix domain socket. This should cover the PR 29499. 2) This requires that unp_{ex,in}ternalize and unp_scan understand mbufs with more than one control message at a time. 3) Internalize and externalize used to work on the mbuf in-place. This made life quite complicated and the code for sizeof(int) < sizeof(file *) could end up doing the wrong thing. The patch always create a new mbuf/cluster now. This resulted in the change of the prototype for the domain externalise function. 4) You can now send SCM_TIMESTAMP messages. 5) Always use CMSG_DATA(cm) to determine the start where the data in unp_{ex,in}ternalize. It was using ((struct cmsghdr *)cm + 1) in some places, which gives the wrong alignment on the alpha. (NetBSD made this fix some time ago). This results in an ABI change for discriptor passing and creds passing on the alpha. (Probably on the IA64 and Spare ports too). 6) Fix userland programs to use CMSG_* macros too. 7) Be more careful about freeing mbufs containing (file *)s. This is made possible by the prototype change of externalise. PR: 29499 MFC after: 6 weeks
2001-10-04 13:11:48 +00:00
error = EBADF;
goto out;
}
Do not allow kqueues to be passed via unix domain sockets.
2003-02-15 06:04:55 +00:00
fp = fdescp->fd_ofiles[fd];
if (!(fp->f_ops->fo_flags & DFLAG_PASSABLE)) {
Replace custom file descriptor array sleep lock constructed using a mutex and flags with an sxlock. This leads to a significant and measurable performance improvement as a result of access to shared locking for frequent lookup operations, reduced general overhead, and reduced overhead in the event of contention. All of these are imported for threaded applications where simultaneous access to a shared file descriptor array occurs frequently. Kris has reported 2x-4x transaction rate improvements on 8-core MySQL benchmarks; smaller improvements can be expected for many workloads as a result of reduced overhead. - Generally eliminate the distinction between "fast" and regular acquisisition of the filedesc lock; the plan is that they will now all be fast. Change all locking instances to either shared or exclusive locks. - Correct a bug (pointed out by kib) in fdfree() where previously msleep() was called without the mutex held; sx_sleep() is now always called with the sxlock held exclusively. - Universally hold the struct file lock over changes to struct file, rather than the filedesc lock or no lock. Always update the f_ops field last. A further memory barrier is required here in the future (discussed with jhb). - Improve locking and reference management in linux_at(), which fails to properly acquire vnode references before using vnode pointers. Annotate improper use of vn_fullpath(), which will be replaced at a future date. In fcntl(), we conservatively acquire an exclusive lock, even though in some cases a shared lock may be sufficient, which should be revisited. The dropping of the filedesc lock in fdgrowtable() is no longer required as the sxlock can be held over the sleep operation; we should consider removing that (pointed out by attilio). Tested by: kris Discussed with: jhb, kris, attilio, jeff
2007-04-04 09:11:34 +00:00
FILEDESC_SUNLOCK(fdescp);
Do not allow kqueues to be passed via unix domain sockets.
2003-02-15 06:04:55 +00:00
error = EOPNOTSUPP;
goto out;
}
Hopefully improve control message passing over Unix domain sockets. 1) Allow the sending of more than one control message at a time over a unix domain socket. This should cover the PR 29499. 2) This requires that unp_{ex,in}ternalize and unp_scan understand mbufs with more than one control message at a time. 3) Internalize and externalize used to work on the mbuf in-place. This made life quite complicated and the code for sizeof(int) < sizeof(file *) could end up doing the wrong thing. The patch always create a new mbuf/cluster now. This resulted in the change of the prototype for the domain externalise function. 4) You can now send SCM_TIMESTAMP messages. 5) Always use CMSG_DATA(cm) to determine the start where the data in unp_{ex,in}ternalize. It was using ((struct cmsghdr *)cm + 1) in some places, which gives the wrong alignment on the alpha. (NetBSD made this fix some time ago). This results in an ABI change for discriptor passing and creds passing on the alpha. (Probably on the IA64 and Spare ports too). 6) Fix userland programs to use CMSG_* macros too. 7) Be more careful about freeing mbufs containing (file *)s. This is made possible by the prototype change of externalise. PR: 29499 MFC after: 6 weeks
2001-10-04 13:11:48 +00:00
}
Replace custom file descriptor array sleep lock constructed using a mutex and flags with an sxlock. This leads to a significant and measurable performance improvement as a result of access to shared locking for frequent lookup operations, reduced general overhead, and reduced overhead in the event of contention. All of these are imported for threaded applications where simultaneous access to a shared file descriptor array occurs frequently. Kris has reported 2x-4x transaction rate improvements on 8-core MySQL benchmarks; smaller improvements can be expected for many workloads as a result of reduced overhead. - Generally eliminate the distinction between "fast" and regular acquisisition of the filedesc lock; the plan is that they will now all be fast. Change all locking instances to either shared or exclusive locks. - Correct a bug (pointed out by kib) in fdfree() where previously msleep() was called without the mutex held; sx_sleep() is now always called with the sxlock held exclusively. - Universally hold the struct file lock over changes to struct file, rather than the filedesc lock or no lock. Always update the f_ops field last. A further memory barrier is required here in the future (discussed with jhb). - Improve locking and reference management in linux_at(), which fails to properly acquire vnode references before using vnode pointers. Annotate improper use of vn_fullpath(), which will be replaced at a future date. In fcntl(), we conservatively acquire an exclusive lock, even though in some cases a shared lock may be sufficient, which should be revisited. The dropping of the filedesc lock in fdgrowtable() is no longer required as the sxlock can be held over the sleep operation; we should consider removing that (pointed out by attilio). Tested by: kris Discussed with: jhb, kris, attilio, jeff
2007-04-04 09:11:34 +00:00
Hopefully improve control message passing over Unix domain sockets. 1) Allow the sending of more than one control message at a time over a unix domain socket. This should cover the PR 29499. 2) This requires that unp_{ex,in}ternalize and unp_scan understand mbufs with more than one control message at a time. 3) Internalize and externalize used to work on the mbuf in-place. This made life quite complicated and the code for sizeof(int) < sizeof(file *) could end up doing the wrong thing. The patch always create a new mbuf/cluster now. This resulted in the change of the prototype for the domain externalise function. 4) You can now send SCM_TIMESTAMP messages. 5) Always use CMSG_DATA(cm) to determine the start where the data in unp_{ex,in}ternalize. It was using ((struct cmsghdr *)cm + 1) in some places, which gives the wrong alignment on the alpha. (NetBSD made this fix some time ago). This results in an ABI change for discriptor passing and creds passing on the alpha. (Probably on the IA64 and Spare ports too). 6) Fix userland programs to use CMSG_* macros too. 7) Be more careful about freeing mbufs containing (file *)s. This is made possible by the prototype change of externalise. PR: 29499 MFC after: 6 weeks
2001-10-04 13:11:48 +00:00
/*
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
* Now replace the integer FDs with pointers to
* the associated global file table entry..
Hopefully improve control message passing over Unix domain sockets. 1) Allow the sending of more than one control message at a time over a unix domain socket. This should cover the PR 29499. 2) This requires that unp_{ex,in}ternalize and unp_scan understand mbufs with more than one control message at a time. 3) Internalize and externalize used to work on the mbuf in-place. This made life quite complicated and the code for sizeof(int) < sizeof(file *) could end up doing the wrong thing. The patch always create a new mbuf/cluster now. This resulted in the change of the prototype for the domain externalise function. 4) You can now send SCM_TIMESTAMP messages. 5) Always use CMSG_DATA(cm) to determine the start where the data in unp_{ex,in}ternalize. It was using ((struct cmsghdr *)cm + 1) in some places, which gives the wrong alignment on the alpha. (NetBSD made this fix some time ago). This results in an ABI change for discriptor passing and creds passing on the alpha. (Probably on the IA64 and Spare ports too). 6) Fix userland programs to use CMSG_* macros too. 7) Be more careful about freeing mbufs containing (file *)s. This is made possible by the prototype change of externalise. PR: 29499 MFC after: 6 weeks
2001-10-04 13:11:48 +00:00
*/
newlen = oldfds * sizeof(struct file *);
*controlp = sbcreatecontrol(NULL, newlen,
SCM_RIGHTS, SOL_SOCKET);
if (*controlp == NULL) {
Replace custom file descriptor array sleep lock constructed using a mutex and flags with an sxlock. This leads to a significant and measurable performance improvement as a result of access to shared locking for frequent lookup operations, reduced general overhead, and reduced overhead in the event of contention. All of these are imported for threaded applications where simultaneous access to a shared file descriptor array occurs frequently. Kris has reported 2x-4x transaction rate improvements on 8-core MySQL benchmarks; smaller improvements can be expected for many workloads as a result of reduced overhead. - Generally eliminate the distinction between "fast" and regular acquisisition of the filedesc lock; the plan is that they will now all be fast. Change all locking instances to either shared or exclusive locks. - Correct a bug (pointed out by kib) in fdfree() where previously msleep() was called without the mutex held; sx_sleep() is now always called with the sxlock held exclusively. - Universally hold the struct file lock over changes to struct file, rather than the filedesc lock or no lock. Always update the f_ops field last. A further memory barrier is required here in the future (discussed with jhb). - Improve locking and reference management in linux_at(), which fails to properly acquire vnode references before using vnode pointers. Annotate improper use of vn_fullpath(), which will be replaced at a future date. In fcntl(), we conservatively acquire an exclusive lock, even though in some cases a shared lock may be sufficient, which should be revisited. The dropping of the filedesc lock in fdgrowtable() is no longer required as the sxlock can be held over the sleep operation; we should consider removing that (pointed out by attilio). Tested by: kris Discussed with: jhb, kris, attilio, jeff
2007-04-04 09:11:34 +00:00
FILEDESC_SUNLOCK(fdescp);
Hopefully improve control message passing over Unix domain sockets. 1) Allow the sending of more than one control message at a time over a unix domain socket. This should cover the PR 29499. 2) This requires that unp_{ex,in}ternalize and unp_scan understand mbufs with more than one control message at a time. 3) Internalize and externalize used to work on the mbuf in-place. This made life quite complicated and the code for sizeof(int) < sizeof(file *) could end up doing the wrong thing. The patch always create a new mbuf/cluster now. This resulted in the change of the prototype for the domain externalise function. 4) You can now send SCM_TIMESTAMP messages. 5) Always use CMSG_DATA(cm) to determine the start where the data in unp_{ex,in}ternalize. It was using ((struct cmsghdr *)cm + 1) in some places, which gives the wrong alignment on the alpha. (NetBSD made this fix some time ago). This results in an ABI change for discriptor passing and creds passing on the alpha. (Probably on the IA64 and Spare ports too). 6) Fix userland programs to use CMSG_* macros too. 7) Be more careful about freeing mbufs containing (file *)s. This is made possible by the prototype change of externalise. PR: 29499 MFC after: 6 weeks
2001-10-04 13:11:48 +00:00
error = E2BIG;
goto out;
}
Enable SCM_RIGHTS on alpha. Allocate necessary buffer as conversion between int and struct file *. Approved by: jkh Submitted by: brian Reviewed by: bde, brian, peter
2000-03-09 15:15:27 +00:00
Hopefully improve control message passing over Unix domain sockets. 1) Allow the sending of more than one control message at a time over a unix domain socket. This should cover the PR 29499. 2) This requires that unp_{ex,in}ternalize and unp_scan understand mbufs with more than one control message at a time. 3) Internalize and externalize used to work on the mbuf in-place. This made life quite complicated and the code for sizeof(int) < sizeof(file *) could end up doing the wrong thing. The patch always create a new mbuf/cluster now. This resulted in the change of the prototype for the domain externalise function. 4) You can now send SCM_TIMESTAMP messages. 5) Always use CMSG_DATA(cm) to determine the start where the data in unp_{ex,in}ternalize. It was using ((struct cmsghdr *)cm + 1) in some places, which gives the wrong alignment on the alpha. (NetBSD made this fix some time ago). This results in an ABI change for discriptor passing and creds passing on the alpha. (Probably on the IA64 and Spare ports too). 6) Fix userland programs to use CMSG_* macros too. 7) Be more careful about freeing mbufs containing (file *)s. This is made possible by the prototype change of externalise. PR: 29499 MFC after: 6 weeks
2001-10-04 13:11:48 +00:00
fdp = data;
rp = (struct file **)
CMSG_DATA(mtod(*controlp, struct cmsghdr *));
for (i = 0; i < oldfds; i++) {
fp = fdescp->fd_ofiles[*fdp++];
*rp++ = fp;
SMP Lock struct file, filedesc and the global file list. Seigo Tanimura (tanimura) posted the initial delta. I've polished it quite a bit reducing the need for locking and adapting it for KSE. Locks: 1 mutex in each filedesc protects all the fields. protects "struct file" initialization, while a struct file is being changed from &badfileops -> &pipeops or something the filedesc should be locked. 1 mutex in each struct file protects the refcount fields. doesn't protect anything else. the flags used for garbage collection have been moved to f_gcflag which was the FILLER short, this doesn't need locking because the garbage collection is a single threaded container. could likely be made to use a pool mutex. 1 sx lock for the global filelist. struct file * fhold(struct file *fp); /* increments reference count on a file */ struct file * fhold_locked(struct file *fp); /* like fhold but expects file to locked */ struct file * ffind_hold(struct thread *, int fd); /* finds the struct file in thread, adds one reference and returns it unlocked */ struct file * ffind_lock(struct thread *, int fd); /* ffind_hold, but returns file locked */ I still have to smp-safe the fget cruft, I'll get to that asap.
2002-01-13 11:58:06 +00:00
FILE_LOCK(fp);
Hopefully improve control message passing over Unix domain sockets. 1) Allow the sending of more than one control message at a time over a unix domain socket. This should cover the PR 29499. 2) This requires that unp_{ex,in}ternalize and unp_scan understand mbufs with more than one control message at a time. 3) Internalize and externalize used to work on the mbuf in-place. This made life quite complicated and the code for sizeof(int) < sizeof(file *) could end up doing the wrong thing. The patch always create a new mbuf/cluster now. This resulted in the change of the prototype for the domain externalise function. 4) You can now send SCM_TIMESTAMP messages. 5) Always use CMSG_DATA(cm) to determine the start where the data in unp_{ex,in}ternalize. It was using ((struct cmsghdr *)cm + 1) in some places, which gives the wrong alignment on the alpha. (NetBSD made this fix some time ago). This results in an ABI change for discriptor passing and creds passing on the alpha. (Probably on the IA64 and Spare ports too). 6) Fix userland programs to use CMSG_* macros too. 7) Be more careful about freeing mbufs containing (file *)s. This is made possible by the prototype change of externalise. PR: 29499 MFC after: 6 weeks
2001-10-04 13:11:48 +00:00
fp->f_count++;
fp->f_msgcount++;
SMP Lock struct file, filedesc and the global file list. Seigo Tanimura (tanimura) posted the initial delta. I've polished it quite a bit reducing the need for locking and adapting it for KSE. Locks: 1 mutex in each filedesc protects all the fields. protects "struct file" initialization, while a struct file is being changed from &badfileops -> &pipeops or something the filedesc should be locked. 1 mutex in each struct file protects the refcount fields. doesn't protect anything else. the flags used for garbage collection have been moved to f_gcflag which was the FILLER short, this doesn't need locking because the garbage collection is a single threaded container. could likely be made to use a pool mutex. 1 sx lock for the global filelist. struct file * fhold(struct file *fp); /* increments reference count on a file */ struct file * fhold_locked(struct file *fp); /* like fhold but expects file to locked */ struct file * ffind_hold(struct thread *, int fd); /* finds the struct file in thread, adds one reference and returns it unlocked */ struct file * ffind_lock(struct thread *, int fd); /* ffind_hold, but returns file locked */ I still have to smp-safe the fget cruft, I'll get to that asap.
2002-01-13 11:58:06 +00:00
FILE_UNLOCK(fp);
Hopefully improve control message passing over Unix domain sockets. 1) Allow the sending of more than one control message at a time over a unix domain socket. This should cover the PR 29499. 2) This requires that unp_{ex,in}ternalize and unp_scan understand mbufs with more than one control message at a time. 3) Internalize and externalize used to work on the mbuf in-place. This made life quite complicated and the code for sizeof(int) < sizeof(file *) could end up doing the wrong thing. The patch always create a new mbuf/cluster now. This resulted in the change of the prototype for the domain externalise function. 4) You can now send SCM_TIMESTAMP messages. 5) Always use CMSG_DATA(cm) to determine the start where the data in unp_{ex,in}ternalize. It was using ((struct cmsghdr *)cm + 1) in some places, which gives the wrong alignment on the alpha. (NetBSD made this fix some time ago). This results in an ABI change for discriptor passing and creds passing on the alpha. (Probably on the IA64 and Spare ports too). 6) Fix userland programs to use CMSG_* macros too. 7) Be more careful about freeing mbufs containing (file *)s. This is made possible by the prototype change of externalise. PR: 29499 MFC after: 6 weeks
2001-10-04 13:11:48 +00:00
unp_rights++;
}
Replace custom file descriptor array sleep lock constructed using a mutex and flags with an sxlock. This leads to a significant and measurable performance improvement as a result of access to shared locking for frequent lookup operations, reduced general overhead, and reduced overhead in the event of contention. All of these are imported for threaded applications where simultaneous access to a shared file descriptor array occurs frequently. Kris has reported 2x-4x transaction rate improvements on 8-core MySQL benchmarks; smaller improvements can be expected for many workloads as a result of reduced overhead. - Generally eliminate the distinction between "fast" and regular acquisisition of the filedesc lock; the plan is that they will now all be fast. Change all locking instances to either shared or exclusive locks. - Correct a bug (pointed out by kib) in fdfree() where previously msleep() was called without the mutex held; sx_sleep() is now always called with the sxlock held exclusively. - Universally hold the struct file lock over changes to struct file, rather than the filedesc lock or no lock. Always update the f_ops field last. A further memory barrier is required here in the future (discussed with jhb). - Improve locking and reference management in linux_at(), which fails to properly acquire vnode references before using vnode pointers. Annotate improper use of vn_fullpath(), which will be replaced at a future date. In fcntl(), we conservatively acquire an exclusive lock, even though in some cases a shared lock may be sufficient, which should be revisited. The dropping of the filedesc lock in fdgrowtable() is no longer required as the sxlock can be held over the sleep operation; we should consider removing that (pointed out by attilio). Tested by: kris Discussed with: jhb, kris, attilio, jeff
2007-04-04 09:11:34 +00:00
FILEDESC_SUNLOCK(fdescp);
Hopefully improve control message passing over Unix domain sockets. 1) Allow the sending of more than one control message at a time over a unix domain socket. This should cover the PR 29499. 2) This requires that unp_{ex,in}ternalize and unp_scan understand mbufs with more than one control message at a time. 3) Internalize and externalize used to work on the mbuf in-place. This made life quite complicated and the code for sizeof(int) < sizeof(file *) could end up doing the wrong thing. The patch always create a new mbuf/cluster now. This resulted in the change of the prototype for the domain externalise function. 4) You can now send SCM_TIMESTAMP messages. 5) Always use CMSG_DATA(cm) to determine the start where the data in unp_{ex,in}ternalize. It was using ((struct cmsghdr *)cm + 1) in some places, which gives the wrong alignment on the alpha. (NetBSD made this fix some time ago). This results in an ABI change for discriptor passing and creds passing on the alpha. (Probably on the IA64 and Spare ports too). 6) Fix userland programs to use CMSG_* macros too. 7) Be more careful about freeing mbufs containing (file *)s. This is made possible by the prototype change of externalise. PR: 29499 MFC after: 6 weeks
2001-10-04 13:11:48 +00:00
break;
Enable SCM_RIGHTS on alpha. Allocate necessary buffer as conversion between int and struct file *. Approved by: jkh Submitted by: brian Reviewed by: bde, brian, peter
2000-03-09 15:15:27 +00:00
Hopefully improve control message passing over Unix domain sockets. 1) Allow the sending of more than one control message at a time over a unix domain socket. This should cover the PR 29499. 2) This requires that unp_{ex,in}ternalize and unp_scan understand mbufs with more than one control message at a time. 3) Internalize and externalize used to work on the mbuf in-place. This made life quite complicated and the code for sizeof(int) < sizeof(file *) could end up doing the wrong thing. The patch always create a new mbuf/cluster now. This resulted in the change of the prototype for the domain externalise function. 4) You can now send SCM_TIMESTAMP messages. 5) Always use CMSG_DATA(cm) to determine the start where the data in unp_{ex,in}ternalize. It was using ((struct cmsghdr *)cm + 1) in some places, which gives the wrong alignment on the alpha. (NetBSD made this fix some time ago). This results in an ABI change for discriptor passing and creds passing on the alpha. (Probably on the IA64 and Spare ports too). 6) Fix userland programs to use CMSG_* macros too. 7) Be more careful about freeing mbufs containing (file *)s. This is made possible by the prototype change of externalise. PR: 29499 MFC after: 6 weeks
2001-10-04 13:11:48 +00:00
case SCM_TIMESTAMP:
*controlp = sbcreatecontrol(NULL, sizeof(*tv),
SCM_TIMESTAMP, SOL_SOCKET);
if (*controlp == NULL) {
error = ENOBUFS;
goto out;
}
tv = (struct timeval *)
CMSG_DATA(mtod(*controlp, struct cmsghdr *));
microtime(tv);
break;
default:
error = EINVAL;
goto out;
Enable SCM_RIGHTS on alpha. Allocate necessary buffer as conversion between int and struct file *. Approved by: jkh Submitted by: brian Reviewed by: bde, brian, peter
2000-03-09 15:15:27 +00:00
}
Hopefully improve control message passing over Unix domain sockets. 1) Allow the sending of more than one control message at a time over a unix domain socket. This should cover the PR 29499. 2) This requires that unp_{ex,in}ternalize and unp_scan understand mbufs with more than one control message at a time. 3) Internalize and externalize used to work on the mbuf in-place. This made life quite complicated and the code for sizeof(int) < sizeof(file *) could end up doing the wrong thing. The patch always create a new mbuf/cluster now. This resulted in the change of the prototype for the domain externalise function. 4) You can now send SCM_TIMESTAMP messages. 5) Always use CMSG_DATA(cm) to determine the start where the data in unp_{ex,in}ternalize. It was using ((struct cmsghdr *)cm + 1) in some places, which gives the wrong alignment on the alpha. (NetBSD made this fix some time ago). This results in an ABI change for discriptor passing and creds passing on the alpha. (Probably on the IA64 and Spare ports too). 6) Fix userland programs to use CMSG_* macros too. 7) Be more careful about freeing mbufs containing (file *)s. This is made possible by the prototype change of externalise. PR: 29499 MFC after: 6 weeks
2001-10-04 13:11:48 +00:00
controlp = &(*controlp)->m_next;
if (CMSG_SPACE(datalen) < clen) {
clen -= CMSG_SPACE(datalen);
cm = (struct cmsghdr *)
((caddr_t)cm + CMSG_SPACE(datalen));
} else {
clen = 0;
cm = NULL;
Enable SCM_RIGHTS on alpha. Allocate necessary buffer as conversion between int and struct file *. Approved by: jkh Submitted by: brian Reviewed by: bde, brian, peter
2000-03-09 15:15:27 +00:00
}
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
}
Hopefully improve control message passing over Unix domain sockets. 1) Allow the sending of more than one control message at a time over a unix domain socket. This should cover the PR 29499. 2) This requires that unp_{ex,in}ternalize and unp_scan understand mbufs with more than one control message at a time. 3) Internalize and externalize used to work on the mbuf in-place. This made life quite complicated and the code for sizeof(int) < sizeof(file *) could end up doing the wrong thing. The patch always create a new mbuf/cluster now. This resulted in the change of the prototype for the domain externalise function. 4) You can now send SCM_TIMESTAMP messages. 5) Always use CMSG_DATA(cm) to determine the start where the data in unp_{ex,in}ternalize. It was using ((struct cmsghdr *)cm + 1) in some places, which gives the wrong alignment on the alpha. (NetBSD made this fix some time ago). This results in an ABI change for discriptor passing and creds passing on the alpha. (Probably on the IA64 and Spare ports too). 6) Fix userland programs to use CMSG_* macros too. 7) Be more careful about freeing mbufs containing (file *)s. This is made possible by the prototype change of externalise. PR: 29499 MFC after: 6 weeks
2001-10-04 13:11:48 +00:00
out:
m_freem(control);
return (error);
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
}
Break introductory comment into two paragraphs to separate material on the garbage collection complications from general discussion of UNIX domain sockets. Staticize unp_addsockcred(). Remove XXX comment regarding Giant and v_socket -- v_socket is protected by the global UNIX domain socket lock.
2007-02-20 10:50:02 +00:00
static struct mbuf *
Implement unix(4) socket options LOCAL_CREDS and LOCAL_CONNWAIT. - Add unp_addsockcred() (for LOCAL_CREDS). - Add an argument to unp_connect2() to differentiate between PRU_CONNECT and PRU_CONNECT2. (for LOCAL_CONNWAIT) Obtained from: NetBSD (with some changes)
2005-04-13 00:01:46 +00:00
unp_addsockcred(struct thread *td, struct mbuf *control)
{
o There are two methods to get a process credentials over the unix sockets: 1) A sender sends SCM_CREDS message to a reciever, struct cmsgcred; 2) A reciever sets LOCAL_CREDS socket option and gets sender credentials in control message, struct sockcred. Both methods use the same control message type SCM_CREDS with the same control message level SOL_SOCKET, so they are indistinguishable for the receiver. A difference in struct cmsgcred and struct sockcred layouts may lead to unwanted effects. Now for sockets with LOCAL_CREDS option remove all previous linked SCM_CREDS control messages and then add a control message with struct sockcred so the process specifically asked for the peer credentials by LOCAL_CREDS option always gets struct sockcred. PR: kern/90800 Submitted by: Andrey Simonenko Regres. tests: tools/regression/sockets/unix_cmsg/ MFC after: 1 month
2006-06-13 14:33:35 +00:00
struct mbuf *m, *n, *n_prev;
Implement unix(4) socket options LOCAL_CREDS and LOCAL_CONNWAIT. - Add unp_addsockcred() (for LOCAL_CREDS). - Add an argument to unp_connect2() to differentiate between PRU_CONNECT and PRU_CONNECT2. (for LOCAL_CONNWAIT) Obtained from: NetBSD (with some changes)
2005-04-13 00:01:46 +00:00
struct sockcred *sc;
o There are two methods to get a process credentials over the unix sockets: 1) A sender sends SCM_CREDS message to a reciever, struct cmsgcred; 2) A reciever sets LOCAL_CREDS socket option and gets sender credentials in control message, struct sockcred. Both methods use the same control message type SCM_CREDS with the same control message level SOL_SOCKET, so they are indistinguishable for the receiver. A difference in struct cmsgcred and struct sockcred layouts may lead to unwanted effects. Now for sockets with LOCAL_CREDS option remove all previous linked SCM_CREDS control messages and then add a control message with struct sockcred so the process specifically asked for the peer credentials by LOCAL_CREDS option always gets struct sockcred. PR: kern/90800 Submitted by: Andrey Simonenko Regres. tests: tools/regression/sockets/unix_cmsg/ MFC after: 1 month
2006-06-13 14:33:35 +00:00
const struct cmsghdr *cm;
Implement unix(4) socket options LOCAL_CREDS and LOCAL_CONNWAIT. - Add unp_addsockcred() (for LOCAL_CREDS). - Add an argument to unp_connect2() to differentiate between PRU_CONNECT and PRU_CONNECT2. (for LOCAL_CONNWAIT) Obtained from: NetBSD (with some changes)
2005-04-13 00:01:46 +00:00
int ngroups;
int i;
ngroups = MIN(td->td_ucred->cr_ngroups, CMGROUP_MAX);
m = sbcreatecontrol(NULL, SOCKCREDSIZE(ngroups), SCM_CREDS, SOL_SOCKET);
if (m == NULL)
return (control);
sc = (struct sockcred *) CMSG_DATA(mtod(m, struct cmsghdr *));
sc->sc_uid = td->td_ucred->cr_ruid;
sc->sc_euid = td->td_ucred->cr_uid;
sc->sc_gid = td->td_ucred->cr_rgid;
sc->sc_egid = td->td_ucred->cr_gid;
sc->sc_ngroups = ngroups;
for (i = 0; i < sc->sc_ngroups; i++)
sc->sc_groups[i] = td->td_ucred->cr_groups[i];
/*
Re-wrap and other minor formatting and punctuation fixes for UNIX domain socket comments.
2006-07-22 17:24:55 +00:00
* Unlink SCM_CREDS control messages (struct cmsgcred), since just
* created SCM_CREDS control message (struct sockcred) has another
* format.
Implement unix(4) socket options LOCAL_CREDS and LOCAL_CONNWAIT. - Add unp_addsockcred() (for LOCAL_CREDS). - Add an argument to unp_connect2() to differentiate between PRU_CONNECT and PRU_CONNECT2. (for LOCAL_CONNWAIT) Obtained from: NetBSD (with some changes)
2005-04-13 00:01:46 +00:00
*/
o There are two methods to get a process credentials over the unix sockets: 1) A sender sends SCM_CREDS message to a reciever, struct cmsgcred; 2) A reciever sets LOCAL_CREDS socket option and gets sender credentials in control message, struct sockcred. Both methods use the same control message type SCM_CREDS with the same control message level SOL_SOCKET, so they are indistinguishable for the receiver. A difference in struct cmsgcred and struct sockcred layouts may lead to unwanted effects. Now for sockets with LOCAL_CREDS option remove all previous linked SCM_CREDS control messages and then add a control message with struct sockcred so the process specifically asked for the peer credentials by LOCAL_CREDS option always gets struct sockcred. PR: kern/90800 Submitted by: Andrey Simonenko Regres. tests: tools/regression/sockets/unix_cmsg/ MFC after: 1 month
2006-06-13 14:33:35 +00:00
if (control != NULL)
for (n = control, n_prev = NULL; n != NULL;) {
cm = mtod(n, struct cmsghdr *);
if (cm->cmsg_level == SOL_SOCKET &&
cm->cmsg_type == SCM_CREDS) {
if (n_prev == NULL)
control = n->m_next;
else
n_prev->m_next = n->m_next;
n = m_free(n);
} else {
n_prev = n;
n = n->m_next;
}
}
/* Prepend it to the head. */
m->m_next = control;
return (m);
Implement unix(4) socket options LOCAL_CREDS and LOCAL_CONNWAIT. - Add unp_addsockcred() (for LOCAL_CREDS). - Add an argument to unp_connect2() to differentiate between PRU_CONNECT and PRU_CONNECT2. (for LOCAL_CONNWAIT) Obtained from: NetBSD (with some changes)
2005-04-13 00:01:46 +00:00
}
Don't hold the UNIX domain socket subsystem lock over the body of the UNIX domain socket garbage collection implementation, as that risks holding the mutex over potentially sleeping operations (as well as introducing some nasty lock order issues, etc). unp_gc() will hold the lock long enough to do necessary deferal checks and set that it's running, but then release it until it needs to reset the gc state. RELENG_5 candidate. Discussed with: alfred
2004-08-25 21:24:36 +00:00
/*
Correct a number of serious and closely related bugs in the UNIX domain socket file descriptor garbage collection code, which is intended to detect and clear cycles of orphaned file descriptors that are "in-flight" in a socket when that socket is closed before they are received. The algorithm present was both run at poor times (resulting in recursion and reentrance), and also buggy in the presence of parallelism. In order to fix these problems, make the following changes: - When there are in-flight sockets and a UNIX domain socket is destroyed, asynchronously schedule the garbage collector, rather than running it synchronously in the current context. This avoids lock order issues when the garbage collection code reenters the UNIX domain socket code, avoiding lock order reversals, deadlocks, etc. Run the code asynchronously in a task queue. - In the garbage collector, when skipping file descriptors that have entered a closing state (i.e., have f_count == 0), re-test the FDEFER flag, and decrement unp_defer. As file descriptors can now transition to a closed state, while the garbage collector is running, it is no longer the case that unp_defer will remain an accurate count of deferred sockets in the mark portion of the GC algorithm. Otherwise, the garbage collector will loop waiting waiting for unp_defer to reach zero, which it will never do as it is skipping file descriptors that were marked in an earlier pass, but now closed. - Acquire the UNIX domain socket subsystem lock in unp_discard() when modifying the unp_rights counter, or a read/write race is risked with other threads also manipulating the counter. While here: - Remove #if 0'd code regarding acquiring the socket buffer sleep lock in the garbage collector, this is not required as we are able to use the socket buffer receive lock to protect scanning the receive buffer for in-flight file descriptors on the socket buffer. - Annotate that the description of the garbage collector implementation is increasingly inaccurate and needs to be updated. - Add counters of the number of deferred garbage collections and recycled file descriptors. This will be removed and is here temporarily for debugging purposes. With these changes in place, the unp_passfd regression test now appears to be passed consistently on UP and SMP systems for extended runs, whereas before it hung quickly or panicked, depending on which bug was triggered. Reported by: Philip Kizer <pckizer at nostrum dot com> MFC after: 2 weeks
2005-11-10 16:06:04 +00:00
* unp_defer indicates whether additional work has been defered for a future
* pass through unp_gc(). It is thread local and does not require explicit
* synchronization.
Don't hold the UNIX domain socket subsystem lock over the body of the UNIX domain socket garbage collection implementation, as that risks holding the mutex over potentially sleeping operations (as well as introducing some nasty lock order issues, etc). unp_gc() will hold the lock long enough to do necessary deferal checks and set that it's running, but then release it until it needs to reset the gc state. RELENG_5 candidate. Discussed with: alfred
2004-08-25 21:24:36 +00:00
*/
Correct a number of serious and closely related bugs in the UNIX domain socket file descriptor garbage collection code, which is intended to detect and clear cycles of orphaned file descriptors that are "in-flight" in a socket when that socket is closed before they are received. The algorithm present was both run at poor times (resulting in recursion and reentrance), and also buggy in the presence of parallelism. In order to fix these problems, make the following changes: - When there are in-flight sockets and a UNIX domain socket is destroyed, asynchronously schedule the garbage collector, rather than running it synchronously in the current context. This avoids lock order issues when the garbage collection code reenters the UNIX domain socket code, avoiding lock order reversals, deadlocks, etc. Run the code asynchronously in a task queue. - In the garbage collector, when skipping file descriptors that have entered a closing state (i.e., have f_count == 0), re-test the FDEFER flag, and decrement unp_defer. As file descriptors can now transition to a closed state, while the garbage collector is running, it is no longer the case that unp_defer will remain an accurate count of deferred sockets in the mark portion of the GC algorithm. Otherwise, the garbage collector will loop waiting waiting for unp_defer to reach zero, which it will never do as it is skipping file descriptors that were marked in an earlier pass, but now closed. - Acquire the UNIX domain socket subsystem lock in unp_discard() when modifying the unp_rights counter, or a read/write race is risked with other threads also manipulating the counter. While here: - Remove #if 0'd code regarding acquiring the socket buffer sleep lock in the garbage collector, this is not required as we are able to use the socket buffer receive lock to protect scanning the receive buffer for in-flight file descriptors on the socket buffer. - Annotate that the description of the garbage collector implementation is increasingly inaccurate and needs to be updated. - Add counters of the number of deferred garbage collections and recycled file descriptors. This will be removed and is here temporarily for debugging purposes. With these changes in place, the unp_passfd regression test now appears to be passed consistently on UP and SMP systems for extended runs, whereas before it hung quickly or panicked, depending on which bug was triggered. Reported by: Philip Kizer <pckizer at nostrum dot com> MFC after: 2 weeks
2005-11-10 16:06:04 +00:00
static int unp_defer;
static int unp_taskcount;
SYSCTL_INT(_net_local, OID_AUTO, taskcount, CTLFLAG_RD, &unp_taskcount, 0, "");
static int unp_recycled;
SYSCTL_INT(_net_local, OID_AUTO, recycled, CTLFLAG_RD, &unp_recycled, 0, "");
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
Another mega commit to staticize things.
1995-12-14 09:55:16 +00:00
static void
Correct a number of serious and closely related bugs in the UNIX domain socket file descriptor garbage collection code, which is intended to detect and clear cycles of orphaned file descriptors that are "in-flight" in a socket when that socket is closed before they are received. The algorithm present was both run at poor times (resulting in recursion and reentrance), and also buggy in the presence of parallelism. In order to fix these problems, make the following changes: - When there are in-flight sockets and a UNIX domain socket is destroyed, asynchronously schedule the garbage collector, rather than running it synchronously in the current context. This avoids lock order issues when the garbage collection code reenters the UNIX domain socket code, avoiding lock order reversals, deadlocks, etc. Run the code asynchronously in a task queue. - In the garbage collector, when skipping file descriptors that have entered a closing state (i.e., have f_count == 0), re-test the FDEFER flag, and decrement unp_defer. As file descriptors can now transition to a closed state, while the garbage collector is running, it is no longer the case that unp_defer will remain an accurate count of deferred sockets in the mark portion of the GC algorithm. Otherwise, the garbage collector will loop waiting waiting for unp_defer to reach zero, which it will never do as it is skipping file descriptors that were marked in an earlier pass, but now closed. - Acquire the UNIX domain socket subsystem lock in unp_discard() when modifying the unp_rights counter, or a read/write race is risked with other threads also manipulating the counter. While here: - Remove #if 0'd code regarding acquiring the socket buffer sleep lock in the garbage collector, this is not required as we are able to use the socket buffer receive lock to protect scanning the receive buffer for in-flight file descriptors on the socket buffer. - Annotate that the description of the garbage collector implementation is increasingly inaccurate and needs to be updated. - Add counters of the number of deferred garbage collections and recycled file descriptors. This will be removed and is here temporarily for debugging purposes. With these changes in place, the unp_passfd regression test now appears to be passed consistently on UP and SMP systems for extended runs, whereas before it hung quickly or panicked, depending on which bug was triggered. Reported by: Philip Kizer <pckizer at nostrum dot com> MFC after: 2 weeks
2005-11-10 16:06:04 +00:00
unp_gc(__unused void *arg, int pending)
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
{
style(9)-ize function headers, remove use of 'register'. MFC after: 3 days
2005-02-20 23:22:13 +00:00
struct file *fp, *nextfp;
struct socket *so;
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
struct file **extra_ref, **fpp;
int nunref, i;
We allocate an array of pointers to the global file table while not holding the filelist_lock. This means the filelist can change size while allocating. Detect this race and retry the allocation.
2004-07-02 07:40:10 +00:00
int nfiles_snap;
int nfiles_slack = 20;
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
Correct a number of serious and closely related bugs in the UNIX domain socket file descriptor garbage collection code, which is intended to detect and clear cycles of orphaned file descriptors that are "in-flight" in a socket when that socket is closed before they are received. The algorithm present was both run at poor times (resulting in recursion and reentrance), and also buggy in the presence of parallelism. In order to fix these problems, make the following changes: - When there are in-flight sockets and a UNIX domain socket is destroyed, asynchronously schedule the garbage collector, rather than running it synchronously in the current context. This avoids lock order issues when the garbage collection code reenters the UNIX domain socket code, avoiding lock order reversals, deadlocks, etc. Run the code asynchronously in a task queue. - In the garbage collector, when skipping file descriptors that have entered a closing state (i.e., have f_count == 0), re-test the FDEFER flag, and decrement unp_defer. As file descriptors can now transition to a closed state, while the garbage collector is running, it is no longer the case that unp_defer will remain an accurate count of deferred sockets in the mark portion of the GC algorithm. Otherwise, the garbage collector will loop waiting waiting for unp_defer to reach zero, which it will never do as it is skipping file descriptors that were marked in an earlier pass, but now closed. - Acquire the UNIX domain socket subsystem lock in unp_discard() when modifying the unp_rights counter, or a read/write race is risked with other threads also manipulating the counter. While here: - Remove #if 0'd code regarding acquiring the socket buffer sleep lock in the garbage collector, this is not required as we are able to use the socket buffer receive lock to protect scanning the receive buffer for in-flight file descriptors on the socket buffer. - Annotate that the description of the garbage collector implementation is increasingly inaccurate and needs to be updated. - Add counters of the number of deferred garbage collections and recycled file descriptors. This will be removed and is here temporarily for debugging purposes. With these changes in place, the unp_passfd regression test now appears to be passed consistently on UP and SMP systems for extended runs, whereas before it hung quickly or panicked, depending on which bug was triggered. Reported by: Philip Kizer <pckizer at nostrum dot com> MFC after: 2 weeks
2005-11-10 16:06:04 +00:00
unp_taskcount++;
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
unp_defer = 0;
Mechanical whitespace cleanup; parenthesize return values; other minor style nits.
2004-01-11 19:48:19 +00:00
/*
- Close a race between enumerating UNIX domain socket pcb structures via sysctl and socket teardown by adding a reference count to the UNIX domain pcb object and fixing the sysctl that enumerates unpcbs to grab a reference on each unpcb while it builds the list to copy out to userland. - Close a race between UNIX domain pcb garbage collection (unp_gc()) and file descriptor teardown (fdrop()) by adding a new garbage collection flag FWAIT. unp_gc() sets FWAIT while it walks the message buffers in a UNIX domain socket looking for nested file descriptor references and clears the flag when it is finished. fdrop() checks to see if the flag is set on a file descriptor whose refcount just dropped to 0 and waits for unp_gc() to clear the flag before completely destroying the file descriptor. MFC after: 1 week Reviewed by: rwatson Submitted by: ups Hopefully makes the panics go away: mx1
2007-01-05 19:59:46 +00:00
* Before going through all this, set all FDs to be NOT deferred and
Re-wrap and other minor formatting and punctuation fixes for UNIX domain socket comments.
2006-07-22 17:24:55 +00:00
* NOT externally accessible.
Add comments to hard-to-follow File descriptor handling code
1996-12-05 22:41:13 +00:00
*/
SMP Lock struct file, filedesc and the global file list. Seigo Tanimura (tanimura) posted the initial delta. I've polished it quite a bit reducing the need for locking and adapting it for KSE. Locks: 1 mutex in each filedesc protects all the fields. protects "struct file" initialization, while a struct file is being changed from &badfileops -> &pipeops or something the filedesc should be locked. 1 mutex in each struct file protects the refcount fields. doesn't protect anything else. the flags used for garbage collection have been moved to f_gcflag which was the FILLER short, this doesn't need locking because the garbage collection is a single threaded container. could likely be made to use a pool mutex. 1 sx lock for the global filelist. struct file * fhold(struct file *fp); /* increments reference count on a file */ struct file * fhold_locked(struct file *fp); /* like fhold but expects file to locked */ struct file * ffind_hold(struct thread *, int fd); /* finds the struct file in thread, adds one reference and returns it unlocked */ struct file * ffind_lock(struct thread *, int fd); /* ffind_hold, but returns file locked */ I still have to smp-safe the fget cruft, I'll get to that asap.
2002-01-13 11:58:06 +00:00
sx_slock(&filelist_lock);
This is a partial commit of the patch from PR 14914: Alot of the code in sys/kern directly accesses the *Q_HEAD and *Q_ENTRY structures for list operations. This patch makes all list operations in sys/kern use the queue(3) macros, rather than directly accessing the *Q_{HEAD,ENTRY} structures. This batch of changes compile to the same object files. Reviewed by: phk Submitted by: Jake Burkholder <jake@checker.org> PR: 14914
1999-11-16 10:56:05 +00:00
LIST_FOREACH(fp, &filehead, f_list)
SMP Lock struct file, filedesc and the global file list. Seigo Tanimura (tanimura) posted the initial delta. I've polished it quite a bit reducing the need for locking and adapting it for KSE. Locks: 1 mutex in each filedesc protects all the fields. protects "struct file" initialization, while a struct file is being changed from &badfileops -> &pipeops or something the filedesc should be locked. 1 mutex in each struct file protects the refcount fields. doesn't protect anything else. the flags used for garbage collection have been moved to f_gcflag which was the FILLER short, this doesn't need locking because the garbage collection is a single threaded container. could likely be made to use a pool mutex. 1 sx lock for the global filelist. struct file * fhold(struct file *fp); /* increments reference count on a file */ struct file * fhold_locked(struct file *fp); /* like fhold but expects file to locked */ struct file * ffind_hold(struct thread *, int fd); /* finds the struct file in thread, adds one reference and returns it unlocked */ struct file * ffind_lock(struct thread *, int fd); /* ffind_hold, but returns file locked */ I still have to smp-safe the fget cruft, I'll get to that asap.
2002-01-13 11:58:06 +00:00
fp->f_gcflag &= ~(FMARK|FDEFER);
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
do {
Normalize a significant number of kernel malloc type names: - Prefer '_' to ' ', as it results in more easily parsed results in memory monitoring tools such as vmstat. - Remove punctuation that is incompatible with using memory type names as file names, such as '/' characters. - Disambiguate some collisions by adding subsystem prefixes to some memory types. - Generally prefer lower case to upper case. - If the same type is defined in multiple architecture directories, attempt to use the same name in additional cases. Not all instances were caught in this change, so more work is required to finish this conversion. Similar changes are required for UMA zone names.
2005-10-31 15:41:29 +00:00
KASSERT(unp_defer >= 0, ("unp_gc: unp_defer %d", unp_defer));
This is a partial commit of the patch from PR 14914: Alot of the code in sys/kern directly accesses the *Q_HEAD and *Q_ENTRY structures for list operations. This patch makes all list operations in sys/kern use the queue(3) macros, rather than directly accessing the *Q_{HEAD,ENTRY} structures. This batch of changes compile to the same object files. Reviewed by: phk Submitted by: Jake Burkholder <jake@checker.org> PR: 14914
1999-11-16 10:56:05 +00:00
LIST_FOREACH(fp, &filehead, f_list) {
SMP Lock struct file, filedesc and the global file list. Seigo Tanimura (tanimura) posted the initial delta. I've polished it quite a bit reducing the need for locking and adapting it for KSE. Locks: 1 mutex in each filedesc protects all the fields. protects "struct file" initialization, while a struct file is being changed from &badfileops -> &pipeops or something the filedesc should be locked. 1 mutex in each struct file protects the refcount fields. doesn't protect anything else. the flags used for garbage collection have been moved to f_gcflag which was the FILLER short, this doesn't need locking because the garbage collection is a single threaded container. could likely be made to use a pool mutex. 1 sx lock for the global filelist. struct file * fhold(struct file *fp); /* increments reference count on a file */ struct file * fhold_locked(struct file *fp); /* like fhold but expects file to locked */ struct file * ffind_hold(struct thread *, int fd); /* finds the struct file in thread, adds one reference and returns it unlocked */ struct file * ffind_lock(struct thread *, int fd); /* ffind_hold, but returns file locked */ I still have to smp-safe the fget cruft, I'll get to that asap.
2002-01-13 11:58:06 +00:00
FILE_LOCK(fp);
Add comments to hard-to-follow File descriptor handling code
1996-12-05 22:41:13 +00:00
/*
Correct a number of serious and closely related bugs in the UNIX domain socket file descriptor garbage collection code, which is intended to detect and clear cycles of orphaned file descriptors that are "in-flight" in a socket when that socket is closed before they are received. The algorithm present was both run at poor times (resulting in recursion and reentrance), and also buggy in the presence of parallelism. In order to fix these problems, make the following changes: - When there are in-flight sockets and a UNIX domain socket is destroyed, asynchronously schedule the garbage collector, rather than running it synchronously in the current context. This avoids lock order issues when the garbage collection code reenters the UNIX domain socket code, avoiding lock order reversals, deadlocks, etc. Run the code asynchronously in a task queue. - In the garbage collector, when skipping file descriptors that have entered a closing state (i.e., have f_count == 0), re-test the FDEFER flag, and decrement unp_defer. As file descriptors can now transition to a closed state, while the garbage collector is running, it is no longer the case that unp_defer will remain an accurate count of deferred sockets in the mark portion of the GC algorithm. Otherwise, the garbage collector will loop waiting waiting for unp_defer to reach zero, which it will never do as it is skipping file descriptors that were marked in an earlier pass, but now closed. - Acquire the UNIX domain socket subsystem lock in unp_discard() when modifying the unp_rights counter, or a read/write race is risked with other threads also manipulating the counter. While here: - Remove #if 0'd code regarding acquiring the socket buffer sleep lock in the garbage collector, this is not required as we are able to use the socket buffer receive lock to protect scanning the receive buffer for in-flight file descriptors on the socket buffer. - Annotate that the description of the garbage collector implementation is increasingly inaccurate and needs to be updated. - Add counters of the number of deferred garbage collections and recycled file descriptors. This will be removed and is here temporarily for debugging purposes. With these changes in place, the unp_passfd regression test now appears to be passed consistently on UP and SMP systems for extended runs, whereas before it hung quickly or panicked, depending on which bug was triggered. Reported by: Philip Kizer <pckizer at nostrum dot com> MFC after: 2 weeks
2005-11-10 16:06:04 +00:00
* If the file is not open, skip it -- could be a
* file in the process of being opened, or in the
* process of being closed. If the file is
* "closing", it may have been marked for deferred
* consideration. Clear the flag now if so.
Add comments to hard-to-follow File descriptor handling code
1996-12-05 22:41:13 +00:00
*/
SMP Lock struct file, filedesc and the global file list. Seigo Tanimura (tanimura) posted the initial delta. I've polished it quite a bit reducing the need for locking and adapting it for KSE. Locks: 1 mutex in each filedesc protects all the fields. protects "struct file" initialization, while a struct file is being changed from &badfileops -> &pipeops or something the filedesc should be locked. 1 mutex in each struct file protects the refcount fields. doesn't protect anything else. the flags used for garbage collection have been moved to f_gcflag which was the FILLER short, this doesn't need locking because the garbage collection is a single threaded container. could likely be made to use a pool mutex. 1 sx lock for the global filelist. struct file * fhold(struct file *fp); /* increments reference count on a file */ struct file * fhold_locked(struct file *fp); /* like fhold but expects file to locked */ struct file * ffind_hold(struct thread *, int fd); /* finds the struct file in thread, adds one reference and returns it unlocked */ struct file * ffind_lock(struct thread *, int fd); /* ffind_hold, but returns file locked */ I still have to smp-safe the fget cruft, I'll get to that asap.
2002-01-13 11:58:06 +00:00
if (fp->f_count == 0) {
Correct a number of serious and closely related bugs in the UNIX domain socket file descriptor garbage collection code, which is intended to detect and clear cycles of orphaned file descriptors that are "in-flight" in a socket when that socket is closed before they are received. The algorithm present was both run at poor times (resulting in recursion and reentrance), and also buggy in the presence of parallelism. In order to fix these problems, make the following changes: - When there are in-flight sockets and a UNIX domain socket is destroyed, asynchronously schedule the garbage collector, rather than running it synchronously in the current context. This avoids lock order issues when the garbage collection code reenters the UNIX domain socket code, avoiding lock order reversals, deadlocks, etc. Run the code asynchronously in a task queue. - In the garbage collector, when skipping file descriptors that have entered a closing state (i.e., have f_count == 0), re-test the FDEFER flag, and decrement unp_defer. As file descriptors can now transition to a closed state, while the garbage collector is running, it is no longer the case that unp_defer will remain an accurate count of deferred sockets in the mark portion of the GC algorithm. Otherwise, the garbage collector will loop waiting waiting for unp_defer to reach zero, which it will never do as it is skipping file descriptors that were marked in an earlier pass, but now closed. - Acquire the UNIX domain socket subsystem lock in unp_discard() when modifying the unp_rights counter, or a read/write race is risked with other threads also manipulating the counter. While here: - Remove #if 0'd code regarding acquiring the socket buffer sleep lock in the garbage collector, this is not required as we are able to use the socket buffer receive lock to protect scanning the receive buffer for in-flight file descriptors on the socket buffer. - Annotate that the description of the garbage collector implementation is increasingly inaccurate and needs to be updated. - Add counters of the number of deferred garbage collections and recycled file descriptors. This will be removed and is here temporarily for debugging purposes. With these changes in place, the unp_passfd regression test now appears to be passed consistently on UP and SMP systems for extended runs, whereas before it hung quickly or panicked, depending on which bug was triggered. Reported by: Philip Kizer <pckizer at nostrum dot com> MFC after: 2 weeks
2005-11-10 16:06:04 +00:00
if (fp->f_gcflag & FDEFER)
unp_defer--;
fp->f_gcflag &= ~(FMARK|FDEFER);
SMP Lock struct file, filedesc and the global file list. Seigo Tanimura (tanimura) posted the initial delta. I've polished it quite a bit reducing the need for locking and adapting it for KSE. Locks: 1 mutex in each filedesc protects all the fields. protects "struct file" initialization, while a struct file is being changed from &badfileops -> &pipeops or something the filedesc should be locked. 1 mutex in each struct file protects the refcount fields. doesn't protect anything else. the flags used for garbage collection have been moved to f_gcflag which was the FILLER short, this doesn't need locking because the garbage collection is a single threaded container. could likely be made to use a pool mutex. 1 sx lock for the global filelist. struct file * fhold(struct file *fp); /* increments reference count on a file */ struct file * fhold_locked(struct file *fp); /* like fhold but expects file to locked */ struct file * ffind_hold(struct thread *, int fd); /* finds the struct file in thread, adds one reference and returns it unlocked */ struct file * ffind_lock(struct thread *, int fd); /* ffind_hold, but returns file locked */ I still have to smp-safe the fget cruft, I'll get to that asap.
2002-01-13 11:58:06 +00:00
FILE_UNLOCK(fp);
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
continue;
SMP Lock struct file, filedesc and the global file list. Seigo Tanimura (tanimura) posted the initial delta. I've polished it quite a bit reducing the need for locking and adapting it for KSE. Locks: 1 mutex in each filedesc protects all the fields. protects "struct file" initialization, while a struct file is being changed from &badfileops -> &pipeops or something the filedesc should be locked. 1 mutex in each struct file protects the refcount fields. doesn't protect anything else. the flags used for garbage collection have been moved to f_gcflag which was the FILLER short, this doesn't need locking because the garbage collection is a single threaded container. could likely be made to use a pool mutex. 1 sx lock for the global filelist. struct file * fhold(struct file *fp); /* increments reference count on a file */ struct file * fhold_locked(struct file *fp); /* like fhold but expects file to locked */ struct file * ffind_hold(struct thread *, int fd); /* finds the struct file in thread, adds one reference and returns it unlocked */ struct file * ffind_lock(struct thread *, int fd); /* ffind_hold, but returns file locked */ I still have to smp-safe the fget cruft, I'll get to that asap.
2002-01-13 11:58:06 +00:00
}
Add comments to hard-to-follow File descriptor handling code
1996-12-05 22:41:13 +00:00
/*
- Close a race between enumerating UNIX domain socket pcb structures via sysctl and socket teardown by adding a reference count to the UNIX domain pcb object and fixing the sysctl that enumerates unpcbs to grab a reference on each unpcb while it builds the list to copy out to userland. - Close a race between UNIX domain pcb garbage collection (unp_gc()) and file descriptor teardown (fdrop()) by adding a new garbage collection flag FWAIT. unp_gc() sets FWAIT while it walks the message buffers in a UNIX domain socket looking for nested file descriptor references and clears the flag when it is finished. fdrop() checks to see if the flag is set on a file descriptor whose refcount just dropped to 0 and waits for unp_gc() to clear the flag before completely destroying the file descriptor. MFC after: 1 week Reviewed by: rwatson Submitted by: ups Hopefully makes the panics go away: mx1
2007-01-05 19:59:46 +00:00
* If we already marked it as 'defer' in a
* previous pass, then try to process it this
* time and un-mark it.
Add comments to hard-to-follow File descriptor handling code
1996-12-05 22:41:13 +00:00
*/
SMP Lock struct file, filedesc and the global file list. Seigo Tanimura (tanimura) posted the initial delta. I've polished it quite a bit reducing the need for locking and adapting it for KSE. Locks: 1 mutex in each filedesc protects all the fields. protects "struct file" initialization, while a struct file is being changed from &badfileops -> &pipeops or something the filedesc should be locked. 1 mutex in each struct file protects the refcount fields. doesn't protect anything else. the flags used for garbage collection have been moved to f_gcflag which was the FILLER short, this doesn't need locking because the garbage collection is a single threaded container. could likely be made to use a pool mutex. 1 sx lock for the global filelist. struct file * fhold(struct file *fp); /* increments reference count on a file */ struct file * fhold_locked(struct file *fp); /* like fhold but expects file to locked */ struct file * ffind_hold(struct thread *, int fd); /* finds the struct file in thread, adds one reference and returns it unlocked */ struct file * ffind_lock(struct thread *, int fd); /* ffind_hold, but returns file locked */ I still have to smp-safe the fget cruft, I'll get to that asap.
2002-01-13 11:58:06 +00:00
if (fp->f_gcflag & FDEFER) {
fp->f_gcflag &= ~FDEFER;
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
unp_defer--;
} else {
Add comments to hard-to-follow File descriptor handling code
1996-12-05 22:41:13 +00:00
/*
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
* If it's not deferred, then check if it's
Add comments to hard-to-follow File descriptor handling code
1996-12-05 22:41:13 +00:00
* already marked.. if so skip it
*/
SMP Lock struct file, filedesc and the global file list. Seigo Tanimura (tanimura) posted the initial delta. I've polished it quite a bit reducing the need for locking and adapting it for KSE. Locks: 1 mutex in each filedesc protects all the fields. protects "struct file" initialization, while a struct file is being changed from &badfileops -> &pipeops or something the filedesc should be locked. 1 mutex in each struct file protects the refcount fields. doesn't protect anything else. the flags used for garbage collection have been moved to f_gcflag which was the FILLER short, this doesn't need locking because the garbage collection is a single threaded container. could likely be made to use a pool mutex. 1 sx lock for the global filelist. struct file * fhold(struct file *fp); /* increments reference count on a file */ struct file * fhold_locked(struct file *fp); /* like fhold but expects file to locked */ struct file * ffind_hold(struct thread *, int fd); /* finds the struct file in thread, adds one reference and returns it unlocked */ struct file * ffind_lock(struct thread *, int fd); /* ffind_hold, but returns file locked */ I still have to smp-safe the fget cruft, I'll get to that asap.
2002-01-13 11:58:06 +00:00
if (fp->f_gcflag & FMARK) {
FILE_UNLOCK(fp);
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
continue;
SMP Lock struct file, filedesc and the global file list. Seigo Tanimura (tanimura) posted the initial delta. I've polished it quite a bit reducing the need for locking and adapting it for KSE. Locks: 1 mutex in each filedesc protects all the fields. protects "struct file" initialization, while a struct file is being changed from &badfileops -> &pipeops or something the filedesc should be locked. 1 mutex in each struct file protects the refcount fields. doesn't protect anything else. the flags used for garbage collection have been moved to f_gcflag which was the FILLER short, this doesn't need locking because the garbage collection is a single threaded container. could likely be made to use a pool mutex. 1 sx lock for the global filelist. struct file * fhold(struct file *fp); /* increments reference count on a file */ struct file * fhold_locked(struct file *fp); /* like fhold but expects file to locked */ struct file * ffind_hold(struct thread *, int fd); /* finds the struct file in thread, adds one reference and returns it unlocked */ struct file * ffind_lock(struct thread *, int fd); /* ffind_hold, but returns file locked */ I still have to smp-safe the fget cruft, I'll get to that asap.
2002-01-13 11:58:06 +00:00
}
Mechanical whitespace cleanup; parenthesize return values; other minor style nits.
2004-01-11 19:48:19 +00:00
/*
Re-wrap and other minor formatting and punctuation fixes for UNIX domain socket comments.
2006-07-22 17:24:55 +00:00
* If all references are from messages in
* transit, then skip it. it's not externally
* accessible.
Mechanical whitespace cleanup; parenthesize return values; other minor style nits.
2004-01-11 19:48:19 +00:00
*/
SMP Lock struct file, filedesc and the global file list. Seigo Tanimura (tanimura) posted the initial delta. I've polished it quite a bit reducing the need for locking and adapting it for KSE. Locks: 1 mutex in each filedesc protects all the fields. protects "struct file" initialization, while a struct file is being changed from &badfileops -> &pipeops or something the filedesc should be locked. 1 mutex in each struct file protects the refcount fields. doesn't protect anything else. the flags used for garbage collection have been moved to f_gcflag which was the FILLER short, this doesn't need locking because the garbage collection is a single threaded container. could likely be made to use a pool mutex. 1 sx lock for the global filelist. struct file * fhold(struct file *fp); /* increments reference count on a file */ struct file * fhold_locked(struct file *fp); /* like fhold but expects file to locked */ struct file * ffind_hold(struct thread *, int fd); /* finds the struct file in thread, adds one reference and returns it unlocked */ struct file * ffind_lock(struct thread *, int fd); /* ffind_hold, but returns file locked */ I still have to smp-safe the fget cruft, I'll get to that asap.
2002-01-13 11:58:06 +00:00
if (fp->f_count == fp->f_msgcount) {
FILE_UNLOCK(fp);
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
continue;
SMP Lock struct file, filedesc and the global file list. Seigo Tanimura (tanimura) posted the initial delta. I've polished it quite a bit reducing the need for locking and adapting it for KSE. Locks: 1 mutex in each filedesc protects all the fields. protects "struct file" initialization, while a struct file is being changed from &badfileops -> &pipeops or something the filedesc should be locked. 1 mutex in each struct file protects the refcount fields. doesn't protect anything else. the flags used for garbage collection have been moved to f_gcflag which was the FILLER short, this doesn't need locking because the garbage collection is a single threaded container. could likely be made to use a pool mutex. 1 sx lock for the global filelist. struct file * fhold(struct file *fp); /* increments reference count on a file */ struct file * fhold_locked(struct file *fp); /* like fhold but expects file to locked */ struct file * ffind_hold(struct thread *, int fd); /* finds the struct file in thread, adds one reference and returns it unlocked */ struct file * ffind_lock(struct thread *, int fd); /* ffind_hold, but returns file locked */ I still have to smp-safe the fget cruft, I'll get to that asap.
2002-01-13 11:58:06 +00:00
}
Mechanical whitespace cleanup; parenthesize return values; other minor style nits.
2004-01-11 19:48:19 +00:00
/*
Add comments to hard-to-follow File descriptor handling code
1996-12-05 22:41:13 +00:00
* If it got this far then it must be
* externally accessible.
*/
SMP Lock struct file, filedesc and the global file list. Seigo Tanimura (tanimura) posted the initial delta. I've polished it quite a bit reducing the need for locking and adapting it for KSE. Locks: 1 mutex in each filedesc protects all the fields. protects "struct file" initialization, while a struct file is being changed from &badfileops -> &pipeops or something the filedesc should be locked. 1 mutex in each struct file protects the refcount fields. doesn't protect anything else. the flags used for garbage collection have been moved to f_gcflag which was the FILLER short, this doesn't need locking because the garbage collection is a single threaded container. could likely be made to use a pool mutex. 1 sx lock for the global filelist. struct file * fhold(struct file *fp); /* increments reference count on a file */ struct file * fhold_locked(struct file *fp); /* like fhold but expects file to locked */ struct file * ffind_hold(struct thread *, int fd); /* finds the struct file in thread, adds one reference and returns it unlocked */ struct file * ffind_lock(struct thread *, int fd); /* ffind_hold, but returns file locked */ I still have to smp-safe the fget cruft, I'll get to that asap.
2002-01-13 11:58:06 +00:00
fp->f_gcflag |= FMARK;
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
}
Add comments to hard-to-follow File descriptor handling code
1996-12-05 22:41:13 +00:00
/*
- Close a race between enumerating UNIX domain socket pcb structures via sysctl and socket teardown by adding a reference count to the UNIX domain pcb object and fixing the sysctl that enumerates unpcbs to grab a reference on each unpcb while it builds the list to copy out to userland. - Close a race between UNIX domain pcb garbage collection (unp_gc()) and file descriptor teardown (fdrop()) by adding a new garbage collection flag FWAIT. unp_gc() sets FWAIT while it walks the message buffers in a UNIX domain socket looking for nested file descriptor references and clears the flag when it is finished. fdrop() checks to see if the flag is set on a file descriptor whose refcount just dropped to 0 and waits for unp_gc() to clear the flag before completely destroying the file descriptor. MFC after: 1 week Reviewed by: rwatson Submitted by: ups Hopefully makes the panics go away: mx1
2007-01-05 19:59:46 +00:00
* Either it was deferred, or it is externally
Re-wrap and other minor formatting and punctuation fixes for UNIX domain socket comments.
2006-07-22 17:24:55 +00:00
* accessible and not already marked so. Now check
* if it is possibly one of OUR sockets.
Mechanical whitespace cleanup; parenthesize return values; other minor style nits.
2004-01-11 19:48:19 +00:00
*/
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
if (fp->f_type != DTYPE_SOCKET ||
Bow to the whining masses and change a union back into void *. Retain removal of unnecessary casts and throw in some minor cleanups to see if anyone complains, just for the hell of it.
2003-01-13 00:33:17 +00:00
(so = fp->f_data) == NULL) {
SMP Lock struct file, filedesc and the global file list. Seigo Tanimura (tanimura) posted the initial delta. I've polished it quite a bit reducing the need for locking and adapting it for KSE. Locks: 1 mutex in each filedesc protects all the fields. protects "struct file" initialization, while a struct file is being changed from &badfileops -> &pipeops or something the filedesc should be locked. 1 mutex in each struct file protects the refcount fields. doesn't protect anything else. the flags used for garbage collection have been moved to f_gcflag which was the FILLER short, this doesn't need locking because the garbage collection is a single threaded container. could likely be made to use a pool mutex. 1 sx lock for the global filelist. struct file * fhold(struct file *fp); /* increments reference count on a file */ struct file * fhold_locked(struct file *fp); /* like fhold but expects file to locked */ struct file * ffind_hold(struct thread *, int fd); /* finds the struct file in thread, adds one reference and returns it unlocked */ struct file * ffind_lock(struct thread *, int fd); /* ffind_hold, but returns file locked */ I still have to smp-safe the fget cruft, I'll get to that asap.
2002-01-13 11:58:06 +00:00
FILE_UNLOCK(fp);
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
continue;
SMP Lock struct file, filedesc and the global file list. Seigo Tanimura (tanimura) posted the initial delta. I've polished it quite a bit reducing the need for locking and adapting it for KSE. Locks: 1 mutex in each filedesc protects all the fields. protects "struct file" initialization, while a struct file is being changed from &badfileops -> &pipeops or something the filedesc should be locked. 1 mutex in each struct file protects the refcount fields. doesn't protect anything else. the flags used for garbage collection have been moved to f_gcflag which was the FILLER short, this doesn't need locking because the garbage collection is a single threaded container. could likely be made to use a pool mutex. 1 sx lock for the global filelist. struct file * fhold(struct file *fp); /* increments reference count on a file */ struct file * fhold_locked(struct file *fp); /* like fhold but expects file to locked */ struct file * ffind_hold(struct thread *, int fd); /* finds the struct file in thread, adds one reference and returns it unlocked */ struct file * ffind_lock(struct thread *, int fd); /* ffind_hold, but returns file locked */ I still have to smp-safe the fget cruft, I'll get to that asap.
2002-01-13 11:58:06 +00:00
}
Make networking domains drop-ins, through the magic of GNU ld. (Some day, there may even be LKMs.) Also, change the internal name of `unixdomain' to `localdomain' since AF_LOCAL is now the preferred name of this family. Declare netisr correctly and in the right place.
1995-05-11 00:13:26 +00:00
if (so->so_proto->pr_domain != &localdomain ||
- Close a race between enumerating UNIX domain socket pcb structures via sysctl and socket teardown by adding a reference count to the UNIX domain pcb object and fixing the sysctl that enumerates unpcbs to grab a reference on each unpcb while it builds the list to copy out to userland. - Close a race between UNIX domain pcb garbage collection (unp_gc()) and file descriptor teardown (fdrop()) by adding a new garbage collection flag FWAIT. unp_gc() sets FWAIT while it walks the message buffers in a UNIX domain socket looking for nested file descriptor references and clears the flag when it is finished. fdrop() checks to see if the flag is set on a file descriptor whose refcount just dropped to 0 and waits for unp_gc() to clear the flag before completely destroying the file descriptor. MFC after: 1 week Reviewed by: rwatson Submitted by: ups Hopefully makes the panics go away: mx1
2007-01-05 19:59:46 +00:00
(so->so_proto->pr_flags & PR_RIGHTS) == 0) {
FILE_UNLOCK(fp);
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
continue;
- Close a race between enumerating UNIX domain socket pcb structures via sysctl and socket teardown by adding a reference count to the UNIX domain pcb object and fixing the sysctl that enumerates unpcbs to grab a reference on each unpcb while it builds the list to copy out to userland. - Close a race between UNIX domain pcb garbage collection (unp_gc()) and file descriptor teardown (fdrop()) by adding a new garbage collection flag FWAIT. unp_gc() sets FWAIT while it walks the message buffers in a UNIX domain socket looking for nested file descriptor references and clears the flag when it is finished. fdrop() checks to see if the flag is set on a file descriptor whose refcount just dropped to 0 and waits for unp_gc() to clear the flag before completely destroying the file descriptor. MFC after: 1 week Reviewed by: rwatson Submitted by: ups Hopefully makes the panics go away: mx1
2007-01-05 19:59:46 +00:00
}
/*
* Tell any other threads that do a subsequent
* fdrop() that we are scanning the message
* buffers.
*/
fp->f_gcflag |= FWAIT;
FILE_UNLOCK(fp);
Add comments to hard-to-follow File descriptor handling code
1996-12-05 22:41:13 +00:00
/*
Re-wrap and other minor formatting and punctuation fixes for UNIX domain socket comments.
2006-07-22 17:24:55 +00:00
* So, Ok, it's one of our sockets and it IS
- Close a race between enumerating UNIX domain socket pcb structures via sysctl and socket teardown by adding a reference count to the UNIX domain pcb object and fixing the sysctl that enumerates unpcbs to grab a reference on each unpcb while it builds the list to copy out to userland. - Close a race between UNIX domain pcb garbage collection (unp_gc()) and file descriptor teardown (fdrop()) by adding a new garbage collection flag FWAIT. unp_gc() sets FWAIT while it walks the message buffers in a UNIX domain socket looking for nested file descriptor references and clears the flag when it is finished. fdrop() checks to see if the flag is set on a file descriptor whose refcount just dropped to 0 and waits for unp_gc() to clear the flag before completely destroying the file descriptor. MFC after: 1 week Reviewed by: rwatson Submitted by: ups Hopefully makes the panics go away: mx1
2007-01-05 19:59:46 +00:00
* externally accessible (or was deferred). Now we
Re-wrap and other minor formatting and punctuation fixes for UNIX domain socket comments.
2006-07-22 17:24:55 +00:00
* look to see if we hold any file descriptors in its
Mechanical whitespace cleanup; parenthesize return values; other minor style nits.
2004-01-11 19:48:19 +00:00
* message buffers. Follow those links and mark them
Add comments to hard-to-follow File descriptor handling code
1996-12-05 22:41:13 +00:00
* as accessible too.
*/
Acquire the socket buffer lock when calling unp_scan() on so->so_rcv.sb_mb to prevent the mbuf chain from changing during the scan.
2004-06-27 03:29:25 +00:00
SOCKBUF_LOCK(&so->so_rcv);
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
unp_scan(so->so_rcv.sb_mb, unp_mark);
Acquire the socket buffer lock when calling unp_scan() on so->so_rcv.sb_mb to prevent the mbuf chain from changing during the scan.
2004-06-27 03:29:25 +00:00
SOCKBUF_UNLOCK(&so->so_rcv);
- Close a race between enumerating UNIX domain socket pcb structures via sysctl and socket teardown by adding a reference count to the UNIX domain pcb object and fixing the sysctl that enumerates unpcbs to grab a reference on each unpcb while it builds the list to copy out to userland. - Close a race between UNIX domain pcb garbage collection (unp_gc()) and file descriptor teardown (fdrop()) by adding a new garbage collection flag FWAIT. unp_gc() sets FWAIT while it walks the message buffers in a UNIX domain socket looking for nested file descriptor references and clears the flag when it is finished. fdrop() checks to see if the flag is set on a file descriptor whose refcount just dropped to 0 and waits for unp_gc() to clear the flag before completely destroying the file descriptor. MFC after: 1 week Reviewed by: rwatson Submitted by: ups Hopefully makes the panics go away: mx1
2007-01-05 19:59:46 +00:00
/*
* Wake up any threads waiting in fdrop().
*/
FILE_LOCK(fp);
fp->f_gcflag &= ~FWAIT;
wakeup(&fp->f_gcflag);
FILE_UNLOCK(fp);
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
}
} while (unp_defer);
SMP Lock struct file, filedesc and the global file list. Seigo Tanimura (tanimura) posted the initial delta. I've polished it quite a bit reducing the need for locking and adapting it for KSE. Locks: 1 mutex in each filedesc protects all the fields. protects "struct file" initialization, while a struct file is being changed from &badfileops -> &pipeops or something the filedesc should be locked. 1 mutex in each struct file protects the refcount fields. doesn't protect anything else. the flags used for garbage collection have been moved to f_gcflag which was the FILLER short, this doesn't need locking because the garbage collection is a single threaded container. could likely be made to use a pool mutex. 1 sx lock for the global filelist. struct file * fhold(struct file *fp); /* increments reference count on a file */ struct file * fhold_locked(struct file *fp); /* like fhold but expects file to locked */ struct file * ffind_hold(struct thread *, int fd); /* finds the struct file in thread, adds one reference and returns it unlocked */ struct file * ffind_lock(struct thread *, int fd); /* ffind_hold, but returns file locked */ I still have to smp-safe the fget cruft, I'll get to that asap.
2002-01-13 11:58:06 +00:00
sx_sunlock(&filelist_lock);
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
/*
Correct a number of serious and closely related bugs in the UNIX domain socket file descriptor garbage collection code, which is intended to detect and clear cycles of orphaned file descriptors that are "in-flight" in a socket when that socket is closed before they are received. The algorithm present was both run at poor times (resulting in recursion and reentrance), and also buggy in the presence of parallelism. In order to fix these problems, make the following changes: - When there are in-flight sockets and a UNIX domain socket is destroyed, asynchronously schedule the garbage collector, rather than running it synchronously in the current context. This avoids lock order issues when the garbage collection code reenters the UNIX domain socket code, avoiding lock order reversals, deadlocks, etc. Run the code asynchronously in a task queue. - In the garbage collector, when skipping file descriptors that have entered a closing state (i.e., have f_count == 0), re-test the FDEFER flag, and decrement unp_defer. As file descriptors can now transition to a closed state, while the garbage collector is running, it is no longer the case that unp_defer will remain an accurate count of deferred sockets in the mark portion of the GC algorithm. Otherwise, the garbage collector will loop waiting waiting for unp_defer to reach zero, which it will never do as it is skipping file descriptors that were marked in an earlier pass, but now closed. - Acquire the UNIX domain socket subsystem lock in unp_discard() when modifying the unp_rights counter, or a read/write race is risked with other threads also manipulating the counter. While here: - Remove #if 0'd code regarding acquiring the socket buffer sleep lock in the garbage collector, this is not required as we are able to use the socket buffer receive lock to protect scanning the receive buffer for in-flight file descriptors on the socket buffer. - Annotate that the description of the garbage collector implementation is increasingly inaccurate and needs to be updated. - Add counters of the number of deferred garbage collections and recycled file descriptors. This will be removed and is here temporarily for debugging purposes. With these changes in place, the unp_passfd regression test now appears to be passed consistently on UP and SMP systems for extended runs, whereas before it hung quickly or panicked, depending on which bug was triggered. Reported by: Philip Kizer <pckizer at nostrum dot com> MFC after: 2 weeks
2005-11-10 16:06:04 +00:00
* XXXRW: The following comments need updating for a post-SMPng and
* deferred unp_gc() world, but are still generally accurate.
*
Re-wrap and other minor formatting and punctuation fixes for UNIX domain socket comments.
2006-07-22 17:24:55 +00:00
* We grab an extra reference to each of the file table entries that
* are not otherwise accessible and then free the rights that are
* stored in messages on them.
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
*
* The bug in the orginal code is a little tricky, so I'll describe
* what's wrong with it here.
*
* It is incorrect to simply unp_discard each entry for f_msgcount
* times -- consider the case of sockets A and B that contain
* references to each other. On a last close of some other socket,
* we trigger a gc since the number of outstanding rights (unp_rights)
Correct a number of serious and closely related bugs in the UNIX domain socket file descriptor garbage collection code, which is intended to detect and clear cycles of orphaned file descriptors that are "in-flight" in a socket when that socket is closed before they are received. The algorithm present was both run at poor times (resulting in recursion and reentrance), and also buggy in the presence of parallelism. In order to fix these problems, make the following changes: - When there are in-flight sockets and a UNIX domain socket is destroyed, asynchronously schedule the garbage collector, rather than running it synchronously in the current context. This avoids lock order issues when the garbage collection code reenters the UNIX domain socket code, avoiding lock order reversals, deadlocks, etc. Run the code asynchronously in a task queue. - In the garbage collector, when skipping file descriptors that have entered a closing state (i.e., have f_count == 0), re-test the FDEFER flag, and decrement unp_defer. As file descriptors can now transition to a closed state, while the garbage collector is running, it is no longer the case that unp_defer will remain an accurate count of deferred sockets in the mark portion of the GC algorithm. Otherwise, the garbage collector will loop waiting waiting for unp_defer to reach zero, which it will never do as it is skipping file descriptors that were marked in an earlier pass, but now closed. - Acquire the UNIX domain socket subsystem lock in unp_discard() when modifying the unp_rights counter, or a read/write race is risked with other threads also manipulating the counter. While here: - Remove #if 0'd code regarding acquiring the socket buffer sleep lock in the garbage collector, this is not required as we are able to use the socket buffer receive lock to protect scanning the receive buffer for in-flight file descriptors on the socket buffer. - Annotate that the description of the garbage collector implementation is increasingly inaccurate and needs to be updated. - Add counters of the number of deferred garbage collections and recycled file descriptors. This will be removed and is here temporarily for debugging purposes. With these changes in place, the unp_passfd regression test now appears to be passed consistently on UP and SMP systems for extended runs, whereas before it hung quickly or panicked, depending on which bug was triggered. Reported by: Philip Kizer <pckizer at nostrum dot com> MFC after: 2 weeks
2005-11-10 16:06:04 +00:00
* is non-zero. If during the sweep phase the gc code unp_discards,
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
* we end up doing a (full) closef on the descriptor. A closef on A
* results in the following chain. Closef calls soo_close, which
* calls soclose. Soclose calls first (through the switch
* uipc_usrreq) unp_detach, which re-invokes unp_gc. Unp_gc simply
Re-wrap and other minor formatting and punctuation fixes for UNIX domain socket comments.
2006-07-22 17:24:55 +00:00
* returns because the previous instance had set unp_gcing, and we
* return all the way back to soclose, which marks the socket with
* SS_NOFDREF, and then calls sofree. Sofree calls sorflush to free
* up the rights that are queued in messages on the socket A, i.e.,
* the reference on B. The sorflush calls via the dom_dispose switch
* unp_dispose, which unp_scans with unp_discard. This second
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
* instance of unp_discard just calls closef on B.
*
* Well, a similar chain occurs on B, resulting in a sorflush on B,
* which results in another closef on A. Unfortunately, A is already
* being closed, and the descriptor has already been marked with
* SS_NOFDREF, and soclose panics at this point.
*
* Here, we first take an extra reference to each inaccessible
Re-wrap and other minor formatting and punctuation fixes for UNIX domain socket comments.
2006-07-22 17:24:55 +00:00
* descriptor. Then, we call sorflush ourself, since we know it is a
* Unix domain socket anyhow. After we destroy all the rights
* carried in messages, we do a last closef to get rid of our extra
* reference. This is the last close, and the unp_detach etc will
* shut down the socket.
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
*
* 91/09/19, bsy@cs.cmu.edu
*/
We allocate an array of pointers to the global file table while not holding the filelist_lock. This means the filelist can change size while allocating. Detect this race and retry the allocation.
2004-07-02 07:40:10 +00:00
again:
"nfiles" is a bad name for a global variable. Call it "openfiles" instead as this is more correct and matches the sysctl variable.
2004-12-01 09:22:26 +00:00
nfiles_snap = openfiles + nfiles_slack; /* some slack */
We allocate an array of pointers to the global file table while not holding the filelist_lock. This means the filelist can change size while allocating. Detect this race and retry the allocation.
2004-07-02 07:40:10 +00:00
extra_ref = malloc(nfiles_snap * sizeof(struct file *), M_TEMP,
M_WAITOK);
SMP Lock struct file, filedesc and the global file list. Seigo Tanimura (tanimura) posted the initial delta. I've polished it quite a bit reducing the need for locking and adapting it for KSE. Locks: 1 mutex in each filedesc protects all the fields. protects "struct file" initialization, while a struct file is being changed from &badfileops -> &pipeops or something the filedesc should be locked. 1 mutex in each struct file protects the refcount fields. doesn't protect anything else. the flags used for garbage collection have been moved to f_gcflag which was the FILLER short, this doesn't need locking because the garbage collection is a single threaded container. could likely be made to use a pool mutex. 1 sx lock for the global filelist. struct file * fhold(struct file *fp); /* increments reference count on a file */ struct file * fhold_locked(struct file *fp); /* like fhold but expects file to locked */ struct file * ffind_hold(struct thread *, int fd); /* finds the struct file in thread, adds one reference and returns it unlocked */ struct file * ffind_lock(struct thread *, int fd); /* ffind_hold, but returns file locked */ I still have to smp-safe the fget cruft, I'll get to that asap.
2002-01-13 11:58:06 +00:00
sx_slock(&filelist_lock);
"nfiles" is a bad name for a global variable. Call it "openfiles" instead as this is more correct and matches the sysctl variable.
2004-12-01 09:22:26 +00:00
if (nfiles_snap < openfiles) {
We allocate an array of pointers to the global file table while not holding the filelist_lock. This means the filelist can change size while allocating. Detect this race and retry the allocation.
2004-07-02 07:40:10 +00:00
sx_sunlock(&filelist_lock);
free(extra_ref, M_TEMP);
nfiles_slack += 20;
goto again;
}
Prefer NULL to 0 when testing and assigning pointer values.
2004-03-30 02:16:25 +00:00
for (nunref = 0, fp = LIST_FIRST(&filehead), fpp = extra_ref;
fp != NULL; fp = nextfp) {
This is a partial commit of the patch from PR 14914: Alot of the code in sys/kern directly accesses the *Q_HEAD and *Q_ENTRY structures for list operations. This patch makes all list operations in sys/kern use the queue(3) macros, rather than directly accessing the *Q_{HEAD,ENTRY} structures. This batch of changes compile to the same object files. Reviewed by: phk Submitted by: Jake Burkholder <jake@checker.org> PR: 14914
1999-11-16 10:56:05 +00:00
nextfp = LIST_NEXT(fp, f_list);
SMP Lock struct file, filedesc and the global file list. Seigo Tanimura (tanimura) posted the initial delta. I've polished it quite a bit reducing the need for locking and adapting it for KSE. Locks: 1 mutex in each filedesc protects all the fields. protects "struct file" initialization, while a struct file is being changed from &badfileops -> &pipeops or something the filedesc should be locked. 1 mutex in each struct file protects the refcount fields. doesn't protect anything else. the flags used for garbage collection have been moved to f_gcflag which was the FILLER short, this doesn't need locking because the garbage collection is a single threaded container. could likely be made to use a pool mutex. 1 sx lock for the global filelist. struct file * fhold(struct file *fp); /* increments reference count on a file */ struct file * fhold_locked(struct file *fp); /* like fhold but expects file to locked */ struct file * ffind_hold(struct thread *, int fd); /* finds the struct file in thread, adds one reference and returns it unlocked */ struct file * ffind_lock(struct thread *, int fd); /* ffind_hold, but returns file locked */ I still have to smp-safe the fget cruft, I'll get to that asap.
2002-01-13 11:58:06 +00:00
FILE_LOCK(fp);
Mechanical whitespace cleanup; parenthesize return values; other minor style nits.
2004-01-11 19:48:19 +00:00
/*
Add comments to hard-to-follow File descriptor handling code
1996-12-05 22:41:13 +00:00
* If it's not open, skip it
*/
SMP Lock struct file, filedesc and the global file list. Seigo Tanimura (tanimura) posted the initial delta. I've polished it quite a bit reducing the need for locking and adapting it for KSE. Locks: 1 mutex in each filedesc protects all the fields. protects "struct file" initialization, while a struct file is being changed from &badfileops -> &pipeops or something the filedesc should be locked. 1 mutex in each struct file protects the refcount fields. doesn't protect anything else. the flags used for garbage collection have been moved to f_gcflag which was the FILLER short, this doesn't need locking because the garbage collection is a single threaded container. could likely be made to use a pool mutex. 1 sx lock for the global filelist. struct file * fhold(struct file *fp); /* increments reference count on a file */ struct file * fhold_locked(struct file *fp); /* like fhold but expects file to locked */ struct file * ffind_hold(struct thread *, int fd); /* finds the struct file in thread, adds one reference and returns it unlocked */ struct file * ffind_lock(struct thread *, int fd); /* ffind_hold, but returns file locked */ I still have to smp-safe the fget cruft, I'll get to that asap.
2002-01-13 11:58:06 +00:00
if (fp->f_count == 0) {
FILE_UNLOCK(fp);
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
continue;
SMP Lock struct file, filedesc and the global file list. Seigo Tanimura (tanimura) posted the initial delta. I've polished it quite a bit reducing the need for locking and adapting it for KSE. Locks: 1 mutex in each filedesc protects all the fields. protects "struct file" initialization, while a struct file is being changed from &badfileops -> &pipeops or something the filedesc should be locked. 1 mutex in each struct file protects the refcount fields. doesn't protect anything else. the flags used for garbage collection have been moved to f_gcflag which was the FILLER short, this doesn't need locking because the garbage collection is a single threaded container. could likely be made to use a pool mutex. 1 sx lock for the global filelist. struct file * fhold(struct file *fp); /* increments reference count on a file */ struct file * fhold_locked(struct file *fp); /* like fhold but expects file to locked */ struct file * ffind_hold(struct thread *, int fd); /* finds the struct file in thread, adds one reference and returns it unlocked */ struct file * ffind_lock(struct thread *, int fd); /* ffind_hold, but returns file locked */ I still have to smp-safe the fget cruft, I'll get to that asap.
2002-01-13 11:58:06 +00:00
}
Mechanical whitespace cleanup; parenthesize return values; other minor style nits.
2004-01-11 19:48:19 +00:00
/*
Add comments to hard-to-follow File descriptor handling code
1996-12-05 22:41:13 +00:00
* If all refs are from msgs, and it's not marked accessible
Re-wrap and other minor formatting and punctuation fixes for UNIX domain socket comments.
2006-07-22 17:24:55 +00:00
* then it must be referenced from some unreachable cycle of
* (shut-down) FDs, so include it in our list of FDs to
* remove.
Add comments to hard-to-follow File descriptor handling code
1996-12-05 22:41:13 +00:00
*/
SMP Lock struct file, filedesc and the global file list. Seigo Tanimura (tanimura) posted the initial delta. I've polished it quite a bit reducing the need for locking and adapting it for KSE. Locks: 1 mutex in each filedesc protects all the fields. protects "struct file" initialization, while a struct file is being changed from &badfileops -> &pipeops or something the filedesc should be locked. 1 mutex in each struct file protects the refcount fields. doesn't protect anything else. the flags used for garbage collection have been moved to f_gcflag which was the FILLER short, this doesn't need locking because the garbage collection is a single threaded container. could likely be made to use a pool mutex. 1 sx lock for the global filelist. struct file * fhold(struct file *fp); /* increments reference count on a file */ struct file * fhold_locked(struct file *fp); /* like fhold but expects file to locked */ struct file * ffind_hold(struct thread *, int fd); /* finds the struct file in thread, adds one reference and returns it unlocked */ struct file * ffind_lock(struct thread *, int fd); /* ffind_hold, but returns file locked */ I still have to smp-safe the fget cruft, I'll get to that asap.
2002-01-13 11:58:06 +00:00
if (fp->f_count == fp->f_msgcount && !(fp->f_gcflag & FMARK)) {
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
*fpp++ = fp;
nunref++;
fp->f_count++;
}
SMP Lock struct file, filedesc and the global file list. Seigo Tanimura (tanimura) posted the initial delta. I've polished it quite a bit reducing the need for locking and adapting it for KSE. Locks: 1 mutex in each filedesc protects all the fields. protects "struct file" initialization, while a struct file is being changed from &badfileops -> &pipeops or something the filedesc should be locked. 1 mutex in each struct file protects the refcount fields. doesn't protect anything else. the flags used for garbage collection have been moved to f_gcflag which was the FILLER short, this doesn't need locking because the garbage collection is a single threaded container. could likely be made to use a pool mutex. 1 sx lock for the global filelist. struct file * fhold(struct file *fp); /* increments reference count on a file */ struct file * fhold_locked(struct file *fp); /* like fhold but expects file to locked */ struct file * ffind_hold(struct thread *, int fd); /* finds the struct file in thread, adds one reference and returns it unlocked */ struct file * ffind_lock(struct thread *, int fd); /* ffind_hold, but returns file locked */ I still have to smp-safe the fget cruft, I'll get to that asap.
2002-01-13 11:58:06 +00:00
FILE_UNLOCK(fp);
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
}
SMP Lock struct file, filedesc and the global file list. Seigo Tanimura (tanimura) posted the initial delta. I've polished it quite a bit reducing the need for locking and adapting it for KSE. Locks: 1 mutex in each filedesc protects all the fields. protects "struct file" initialization, while a struct file is being changed from &badfileops -> &pipeops or something the filedesc should be locked. 1 mutex in each struct file protects the refcount fields. doesn't protect anything else. the flags used for garbage collection have been moved to f_gcflag which was the FILLER short, this doesn't need locking because the garbage collection is a single threaded container. could likely be made to use a pool mutex. 1 sx lock for the global filelist. struct file * fhold(struct file *fp); /* increments reference count on a file */ struct file * fhold_locked(struct file *fp); /* like fhold but expects file to locked */ struct file * ffind_hold(struct thread *, int fd); /* finds the struct file in thread, adds one reference and returns it unlocked */ struct file * ffind_lock(struct thread *, int fd); /* ffind_hold, but returns file locked */ I still have to smp-safe the fget cruft, I'll get to that asap.
2002-01-13 11:58:06 +00:00
sx_sunlock(&filelist_lock);
Mechanical whitespace cleanup; parenthesize return values; other minor style nits.
2004-01-11 19:48:19 +00:00
/*
Re-wrap and other minor formatting and punctuation fixes for UNIX domain socket comments.
2006-07-22 17:24:55 +00:00
* For each FD on our hit list, do the following two things:
Add comments to hard-to-follow File descriptor handling code
1996-12-05 22:41:13 +00:00
*/
This is a rather large commit that encompasses the new swapper, changes to the VM system to support the new swapper, VM bug fixes, several VM optimizations, and some additional revamping of the VM code. The specific bug fixes will be documented with additional forced commits. This commit is somewhat rough in regards to code cleanup issues. Reviewed by: "John S. Dyson" <root@dyson.iquest.net>, "David Greenman" <dg@root.com>
1999-01-21 08:29:12 +00:00
for (i = nunref, fpp = extra_ref; --i >= 0; ++fpp) {
struct file *tfp = *fpp;
SMP Lock struct file, filedesc and the global file list. Seigo Tanimura (tanimura) posted the initial delta. I've polished it quite a bit reducing the need for locking and adapting it for KSE. Locks: 1 mutex in each filedesc protects all the fields. protects "struct file" initialization, while a struct file is being changed from &badfileops -> &pipeops or something the filedesc should be locked. 1 mutex in each struct file protects the refcount fields. doesn't protect anything else. the flags used for garbage collection have been moved to f_gcflag which was the FILLER short, this doesn't need locking because the garbage collection is a single threaded container. could likely be made to use a pool mutex. 1 sx lock for the global filelist. struct file * fhold(struct file *fp); /* increments reference count on a file */ struct file * fhold_locked(struct file *fp); /* like fhold but expects file to locked */ struct file * ffind_hold(struct thread *, int fd); /* finds the struct file in thread, adds one reference and returns it unlocked */ struct file * ffind_lock(struct thread *, int fd); /* ffind_hold, but returns file locked */ I still have to smp-safe the fget cruft, I'll get to that asap.
2002-01-13 11:58:06 +00:00
FILE_LOCK(tfp);
Change struct file f_data to un_data, a union of the correct struct pointer types, and remove a huge number of casts from code using it. Change struct xfile xf_data to xun_data (ABI is still compatible). If we need to add a #define for f_data and xf_data we can, but I don't think it will be necessary. There are no operational changes in this commit.
2003-01-12 01:37:13 +00:00
if (tfp->f_type == DTYPE_SOCKET &&
Bow to the whining masses and change a union back into void *. Retain removal of unnecessary casts and throw in some minor cleanups to see if anyone complains, just for the hell of it.
2003-01-13 00:33:17 +00:00
tfp->f_data != NULL) {
SMP Lock struct file, filedesc and the global file list. Seigo Tanimura (tanimura) posted the initial delta. I've polished it quite a bit reducing the need for locking and adapting it for KSE. Locks: 1 mutex in each filedesc protects all the fields. protects "struct file" initialization, while a struct file is being changed from &badfileops -> &pipeops or something the filedesc should be locked. 1 mutex in each struct file protects the refcount fields. doesn't protect anything else. the flags used for garbage collection have been moved to f_gcflag which was the FILLER short, this doesn't need locking because the garbage collection is a single threaded container. could likely be made to use a pool mutex. 1 sx lock for the global filelist. struct file * fhold(struct file *fp); /* increments reference count on a file */ struct file * fhold_locked(struct file *fp); /* like fhold but expects file to locked */ struct file * ffind_hold(struct thread *, int fd); /* finds the struct file in thread, adds one reference and returns it unlocked */ struct file * ffind_lock(struct thread *, int fd); /* ffind_hold, but returns file locked */ I still have to smp-safe the fget cruft, I'll get to that asap.
2002-01-13 11:58:06 +00:00
FILE_UNLOCK(tfp);
Bow to the whining masses and change a union back into void *. Retain removal of unnecessary casts and throw in some minor cleanups to see if anyone complains, just for the hell of it.
2003-01-13 00:33:17 +00:00
sorflush(tfp->f_data);
Mechanical whitespace cleanup; parenthesize return values; other minor style nits.
2004-01-11 19:48:19 +00:00
} else {
SMP Lock struct file, filedesc and the global file list. Seigo Tanimura (tanimura) posted the initial delta. I've polished it quite a bit reducing the need for locking and adapting it for KSE. Locks: 1 mutex in each filedesc protects all the fields. protects "struct file" initialization, while a struct file is being changed from &badfileops -> &pipeops or something the filedesc should be locked. 1 mutex in each struct file protects the refcount fields. doesn't protect anything else. the flags used for garbage collection have been moved to f_gcflag which was the FILLER short, this doesn't need locking because the garbage collection is a single threaded container. could likely be made to use a pool mutex. 1 sx lock for the global filelist. struct file * fhold(struct file *fp); /* increments reference count on a file */ struct file * fhold_locked(struct file *fp); /* like fhold but expects file to locked */ struct file * ffind_hold(struct thread *, int fd); /* finds the struct file in thread, adds one reference and returns it unlocked */ struct file * ffind_lock(struct thread *, int fd); /* ffind_hold, but returns file locked */ I still have to smp-safe the fget cruft, I'll get to that asap.
2002-01-13 11:58:06 +00:00
FILE_UNLOCK(tfp);
Mechanical whitespace cleanup; parenthesize return values; other minor style nits.
2004-01-11 19:48:19 +00:00
}
This is a rather large commit that encompasses the new swapper, changes to the VM system to support the new swapper, VM bug fixes, several VM optimizations, and some additional revamping of the VM code. The specific bug fixes will be documented with additional forced commits. This commit is somewhat rough in regards to code cleanup issues. Reviewed by: "John S. Dyson" <root@dyson.iquest.net>, "David Greenman" <dg@root.com>
1999-01-21 08:29:12 +00:00
}
Correct a number of serious and closely related bugs in the UNIX domain socket file descriptor garbage collection code, which is intended to detect and clear cycles of orphaned file descriptors that are "in-flight" in a socket when that socket is closed before they are received. The algorithm present was both run at poor times (resulting in recursion and reentrance), and also buggy in the presence of parallelism. In order to fix these problems, make the following changes: - When there are in-flight sockets and a UNIX domain socket is destroyed, asynchronously schedule the garbage collector, rather than running it synchronously in the current context. This avoids lock order issues when the garbage collection code reenters the UNIX domain socket code, avoiding lock order reversals, deadlocks, etc. Run the code asynchronously in a task queue. - In the garbage collector, when skipping file descriptors that have entered a closing state (i.e., have f_count == 0), re-test the FDEFER flag, and decrement unp_defer. As file descriptors can now transition to a closed state, while the garbage collector is running, it is no longer the case that unp_defer will remain an accurate count of deferred sockets in the mark portion of the GC algorithm. Otherwise, the garbage collector will loop waiting waiting for unp_defer to reach zero, which it will never do as it is skipping file descriptors that were marked in an earlier pass, but now closed. - Acquire the UNIX domain socket subsystem lock in unp_discard() when modifying the unp_rights counter, or a read/write race is risked with other threads also manipulating the counter. While here: - Remove #if 0'd code regarding acquiring the socket buffer sleep lock in the garbage collector, this is not required as we are able to use the socket buffer receive lock to protect scanning the receive buffer for in-flight file descriptors on the socket buffer. - Annotate that the description of the garbage collector implementation is increasingly inaccurate and needs to be updated. - Add counters of the number of deferred garbage collections and recycled file descriptors. This will be removed and is here temporarily for debugging purposes. With these changes in place, the unp_passfd regression test now appears to be passed consistently on UP and SMP systems for extended runs, whereas before it hung quickly or panicked, depending on which bug was triggered. Reported by: Philip Kizer <pckizer at nostrum dot com> MFC after: 2 weeks
2005-11-10 16:06:04 +00:00
for (i = nunref, fpp = extra_ref; --i >= 0; ++fpp) {
KSE Milestone 2 Note ALL MODULES MUST BE RECOMPILED make the kernel aware that there are smaller units of scheduling than the process. (but only allow one thread per process at this time). This is functionally equivalent to teh previousl -current except that there is a thread associated with each process. Sorry john! (your next MFC will be a doosie!) Reviewed by: peter@freebsd.org, dillon@freebsd.org X-MFC after: ha ha ha ha
2001-09-12 08:38:13 +00:00
closef(*fpp, (struct thread *) NULL);
Correct a number of serious and closely related bugs in the UNIX domain socket file descriptor garbage collection code, which is intended to detect and clear cycles of orphaned file descriptors that are "in-flight" in a socket when that socket is closed before they are received. The algorithm present was both run at poor times (resulting in recursion and reentrance), and also buggy in the presence of parallelism. In order to fix these problems, make the following changes: - When there are in-flight sockets and a UNIX domain socket is destroyed, asynchronously schedule the garbage collector, rather than running it synchronously in the current context. This avoids lock order issues when the garbage collection code reenters the UNIX domain socket code, avoiding lock order reversals, deadlocks, etc. Run the code asynchronously in a task queue. - In the garbage collector, when skipping file descriptors that have entered a closing state (i.e., have f_count == 0), re-test the FDEFER flag, and decrement unp_defer. As file descriptors can now transition to a closed state, while the garbage collector is running, it is no longer the case that unp_defer will remain an accurate count of deferred sockets in the mark portion of the GC algorithm. Otherwise, the garbage collector will loop waiting waiting for unp_defer to reach zero, which it will never do as it is skipping file descriptors that were marked in an earlier pass, but now closed. - Acquire the UNIX domain socket subsystem lock in unp_discard() when modifying the unp_rights counter, or a read/write race is risked with other threads also manipulating the counter. While here: - Remove #if 0'd code regarding acquiring the socket buffer sleep lock in the garbage collector, this is not required as we are able to use the socket buffer receive lock to protect scanning the receive buffer for in-flight file descriptors on the socket buffer. - Annotate that the description of the garbage collector implementation is increasingly inaccurate and needs to be updated. - Add counters of the number of deferred garbage collections and recycled file descriptors. This will be removed and is here temporarily for debugging purposes. With these changes in place, the unp_passfd regression test now appears to be passed consistently on UP and SMP systems for extended runs, whereas before it hung quickly or panicked, depending on which bug was triggered. Reported by: Philip Kizer <pckizer at nostrum dot com> MFC after: 2 weeks
2005-11-10 16:06:04 +00:00
unp_recycled++;
}
nuke caddr_t.
2002-06-28 23:17:36 +00:00
free(extra_ref, M_TEMP);
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
}
The big 4.4BSD Lite to FreeBSD 2.0.0 (Development) patch. Reviewed by: Rodney W. Grimes Submitted by: John Dyson and David Greenman
1994-05-25 09:21:21 +00:00
void
style(9)-ize function headers, remove use of 'register'. MFC after: 3 days
2005-02-20 23:22:13 +00:00
unp_dispose(struct mbuf *m)
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
{
This is the kernel Lite/2 commit. There are some requisite userland changes, so don't expect to be able to run the kernel as-is (very well) without the appropriate Lite/2 userland changes. The system boots and can mount UFS filesystems. Untested: ext2fs, msdosfs, NFS Known problems: Incorrect Berkeley ID strings in some files. Mount_std mounts will not work until the getfsent library routine is changed. Reviewed by: various people Submitted by: Jeffery Hsu <hsu@freebsd.org>
1997-02-10 02:22:35 +00:00
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
if (m)
unp_scan(m, unp_discard);
}
Another mega commit to staticize things.
1995-12-14 09:55:16 +00:00
static void
style(9)-ize function headers, remove use of 'register'. MFC after: 3 days
2005-02-20 23:22:13 +00:00
unp_scan(struct mbuf *m0, void (*op)(struct file *))
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
{
Hopefully improve control message passing over Unix domain sockets. 1) Allow the sending of more than one control message at a time over a unix domain socket. This should cover the PR 29499. 2) This requires that unp_{ex,in}ternalize and unp_scan understand mbufs with more than one control message at a time. 3) Internalize and externalize used to work on the mbuf in-place. This made life quite complicated and the code for sizeof(int) < sizeof(file *) could end up doing the wrong thing. The patch always create a new mbuf/cluster now. This resulted in the change of the prototype for the domain externalise function. 4) You can now send SCM_TIMESTAMP messages. 5) Always use CMSG_DATA(cm) to determine the start where the data in unp_{ex,in}ternalize. It was using ((struct cmsghdr *)cm + 1) in some places, which gives the wrong alignment on the alpha. (NetBSD made this fix some time ago). This results in an ABI change for discriptor passing and creds passing on the alpha. (Probably on the IA64 and Spare ports too). 6) Fix userland programs to use CMSG_* macros too. 7) Be more careful about freeing mbufs containing (file *)s. This is made possible by the prototype change of externalise. PR: 29499 MFC after: 6 weeks
2001-10-04 13:11:48 +00:00
struct mbuf *m;
struct file **rp;
struct cmsghdr *cm;
void *data;
int i;
socklen_t clen, datalen;
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
int qfds;
Prefer NULL to 0 when testing and assigning pointer values.
2004-03-30 02:16:25 +00:00
while (m0 != NULL) {
Hopefully improve control message passing over Unix domain sockets. 1) Allow the sending of more than one control message at a time over a unix domain socket. This should cover the PR 29499. 2) This requires that unp_{ex,in}ternalize and unp_scan understand mbufs with more than one control message at a time. 3) Internalize and externalize used to work on the mbuf in-place. This made life quite complicated and the code for sizeof(int) < sizeof(file *) could end up doing the wrong thing. The patch always create a new mbuf/cluster now. This resulted in the change of the prototype for the domain externalise function. 4) You can now send SCM_TIMESTAMP messages. 5) Always use CMSG_DATA(cm) to determine the start where the data in unp_{ex,in}ternalize. It was using ((struct cmsghdr *)cm + 1) in some places, which gives the wrong alignment on the alpha. (NetBSD made this fix some time ago). This results in an ABI change for discriptor passing and creds passing on the alpha. (Probably on the IA64 and Spare ports too). 6) Fix userland programs to use CMSG_* macros too. 7) Be more careful about freeing mbufs containing (file *)s. This is made possible by the prototype change of externalise. PR: 29499 MFC after: 6 weeks
2001-10-04 13:11:48 +00:00
for (m = m0; m; m = m->m_next) {
When scanning for control messages, don't process the data mbufs. This could cause hangs if a unix domain socket was closed with data still to be read from it. Tested by: Andrea Campi <andrea@webcom.it>
2001-10-29 20:04:03 +00:00
if (m->m_type != MT_CONTROL)
Hopefully improve control message passing over Unix domain sockets. 1) Allow the sending of more than one control message at a time over a unix domain socket. This should cover the PR 29499. 2) This requires that unp_{ex,in}ternalize and unp_scan understand mbufs with more than one control message at a time. 3) Internalize and externalize used to work on the mbuf in-place. This made life quite complicated and the code for sizeof(int) < sizeof(file *) could end up doing the wrong thing. The patch always create a new mbuf/cluster now. This resulted in the change of the prototype for the domain externalise function. 4) You can now send SCM_TIMESTAMP messages. 5) Always use CMSG_DATA(cm) to determine the start where the data in unp_{ex,in}ternalize. It was using ((struct cmsghdr *)cm + 1) in some places, which gives the wrong alignment on the alpha. (NetBSD made this fix some time ago). This results in an ABI change for discriptor passing and creds passing on the alpha. (Probably on the IA64 and Spare ports too). 6) Fix userland programs to use CMSG_* macros too. 7) Be more careful about freeing mbufs containing (file *)s. This is made possible by the prototype change of externalise. PR: 29499 MFC after: 6 weeks
2001-10-04 13:11:48 +00:00
continue;
cm = mtod(m, struct cmsghdr *);
clen = m->m_len;
while (cm != NULL) {
if (sizeof(*cm) > clen || cm->cmsg_len > clen)
break;
data = CMSG_DATA(cm);
datalen = (caddr_t)cm + cm->cmsg_len
- (caddr_t)data;
if (cm->cmsg_level == SOL_SOCKET &&
cm->cmsg_type == SCM_RIGHTS) {
qfds = datalen / sizeof (struct file *);
rp = data;
for (i = 0; i < qfds; i++)
(*op)(*rp++);
}
if (CMSG_SPACE(datalen) < clen) {
clen -= CMSG_SPACE(datalen);
cm = (struct cmsghdr *)
((caddr_t)cm + CMSG_SPACE(datalen));
} else {
clen = 0;
cm = NULL;
}
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
}
Hopefully improve control message passing over Unix domain sockets. 1) Allow the sending of more than one control message at a time over a unix domain socket. This should cover the PR 29499. 2) This requires that unp_{ex,in}ternalize and unp_scan understand mbufs with more than one control message at a time. 3) Internalize and externalize used to work on the mbuf in-place. This made life quite complicated and the code for sizeof(int) < sizeof(file *) could end up doing the wrong thing. The patch always create a new mbuf/cluster now. This resulted in the change of the prototype for the domain externalise function. 4) You can now send SCM_TIMESTAMP messages. 5) Always use CMSG_DATA(cm) to determine the start where the data in unp_{ex,in}ternalize. It was using ((struct cmsghdr *)cm + 1) in some places, which gives the wrong alignment on the alpha. (NetBSD made this fix some time ago). This results in an ABI change for discriptor passing and creds passing on the alpha. (Probably on the IA64 and Spare ports too). 6) Fix userland programs to use CMSG_* macros too. 7) Be more careful about freeing mbufs containing (file *)s. This is made possible by the prototype change of externalise. PR: 29499 MFC after: 6 weeks
2001-10-04 13:11:48 +00:00
}
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
m0 = m0->m_act;
}
}
Another mega commit to staticize things.
1995-12-14 09:55:16 +00:00
static void
style(9)-ize function headers, remove use of 'register'. MFC after: 3 days
2005-02-20 23:22:13 +00:00
unp_mark(struct file *fp)
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
{
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
/* XXXRW: Should probably assert file list lock here. */
SMP Lock struct file, filedesc and the global file list. Seigo Tanimura (tanimura) posted the initial delta. I've polished it quite a bit reducing the need for locking and adapting it for KSE. Locks: 1 mutex in each filedesc protects all the fields. protects "struct file" initialization, while a struct file is being changed from &badfileops -> &pipeops or something the filedesc should be locked. 1 mutex in each struct file protects the refcount fields. doesn't protect anything else. the flags used for garbage collection have been moved to f_gcflag which was the FILLER short, this doesn't need locking because the garbage collection is a single threaded container. could likely be made to use a pool mutex. 1 sx lock for the global filelist. struct file * fhold(struct file *fp); /* increments reference count on a file */ struct file * fhold_locked(struct file *fp); /* like fhold but expects file to locked */ struct file * ffind_hold(struct thread *, int fd); /* finds the struct file in thread, adds one reference and returns it unlocked */ struct file * ffind_lock(struct thread *, int fd); /* ffind_hold, but returns file locked */ I still have to smp-safe the fget cruft, I'll get to that asap.
2002-01-13 11:58:06 +00:00
if (fp->f_gcflag & FMARK)
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
return;
unp_defer++;
SMP Lock struct file, filedesc and the global file list. Seigo Tanimura (tanimura) posted the initial delta. I've polished it quite a bit reducing the need for locking and adapting it for KSE. Locks: 1 mutex in each filedesc protects all the fields. protects "struct file" initialization, while a struct file is being changed from &badfileops -> &pipeops or something the filedesc should be locked. 1 mutex in each struct file protects the refcount fields. doesn't protect anything else. the flags used for garbage collection have been moved to f_gcflag which was the FILLER short, this doesn't need locking because the garbage collection is a single threaded container. could likely be made to use a pool mutex. 1 sx lock for the global filelist. struct file * fhold(struct file *fp); /* increments reference count on a file */ struct file * fhold_locked(struct file *fp); /* like fhold but expects file to locked */ struct file * ffind_hold(struct thread *, int fd); /* finds the struct file in thread, adds one reference and returns it unlocked */ struct file * ffind_lock(struct thread *, int fd); /* ffind_hold, but returns file locked */ I still have to smp-safe the fget cruft, I'll get to that asap.
2002-01-13 11:58:06 +00:00
fp->f_gcflag |= (FMARK|FDEFER);
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
}
Another mega commit to staticize things.
1995-12-14 09:55:16 +00:00
static void
style(9)-ize function headers, remove use of 'register'. MFC after: 3 days
2005-02-20 23:22:13 +00:00
unp_discard(struct file *fp)
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
{
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
UNP_GLOBAL_WLOCK();
SMP Lock struct file, filedesc and the global file list. Seigo Tanimura (tanimura) posted the initial delta. I've polished it quite a bit reducing the need for locking and adapting it for KSE. Locks: 1 mutex in each filedesc protects all the fields. protects "struct file" initialization, while a struct file is being changed from &badfileops -> &pipeops or something the filedesc should be locked. 1 mutex in each struct file protects the refcount fields. doesn't protect anything else. the flags used for garbage collection have been moved to f_gcflag which was the FILLER short, this doesn't need locking because the garbage collection is a single threaded container. could likely be made to use a pool mutex. 1 sx lock for the global filelist. struct file * fhold(struct file *fp); /* increments reference count on a file */ struct file * fhold_locked(struct file *fp); /* like fhold but expects file to locked */ struct file * ffind_hold(struct thread *, int fd); /* finds the struct file in thread, adds one reference and returns it unlocked */ struct file * ffind_lock(struct thread *, int fd); /* ffind_hold, but returns file locked */ I still have to smp-safe the fget cruft, I'll get to that asap.
2002-01-13 11:58:06 +00:00
FILE_LOCK(fp);
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
fp->f_msgcount--;
unp_rights--;
SMP Lock struct file, filedesc and the global file list. Seigo Tanimura (tanimura) posted the initial delta. I've polished it quite a bit reducing the need for locking and adapting it for KSE. Locks: 1 mutex in each filedesc protects all the fields. protects "struct file" initialization, while a struct file is being changed from &badfileops -> &pipeops or something the filedesc should be locked. 1 mutex in each struct file protects the refcount fields. doesn't protect anything else. the flags used for garbage collection have been moved to f_gcflag which was the FILLER short, this doesn't need locking because the garbage collection is a single threaded container. could likely be made to use a pool mutex. 1 sx lock for the global filelist. struct file * fhold(struct file *fp); /* increments reference count on a file */ struct file * fhold_locked(struct file *fp); /* like fhold but expects file to locked */ struct file * ffind_hold(struct thread *, int fd); /* finds the struct file in thread, adds one reference and returns it unlocked */ struct file * ffind_lock(struct thread *, int fd); /* ffind_hold, but returns file locked */ I still have to smp-safe the fget cruft, I'll get to that asap.
2002-01-13 11:58:06 +00:00
FILE_UNLOCK(fp);
Revise locking strategy used for UNIX domain sockets in order to improve concurrency: - Add per-unpcb mutexes protecting unpcb connection state, fields, etc. - Replace global UNP mutex with a global UNP rwlock, which will protect the UNIX domain socket connection topology, v_socket, and be acquired exclusively before acquiring more than per-unpcb at a time in order to avoid lock order issues. In performance measurements involving MySQL, this change has little or no overhead on UP (+/- 1%), but leads to a significant (5%-30%) improvement in multi-processor measurements using the sysbench and supersmack benchmarks. Much testing by: kris Approved by: re (kensmith)
2007-02-26 20:47:52 +00:00
UNP_GLOBAL_WUNLOCK();
KSE Milestone 2 Note ALL MODULES MUST BE RECOMPILED make the kernel aware that there are smaller units of scheduling than the process. (but only allow one thread per process at this time). This is functionally equivalent to teh previousl -current except that there is a thread associated with each process. Sorry john! (your next MFC will be a doosie!) Reviewed by: peter@freebsd.org, dillon@freebsd.org X-MFC after: ha ha ha ha
2001-09-12 08:38:13 +00:00
(void) closef(fp, (struct thread *)NULL);
BSD 4.4 Lite Kernel Sources
1994-05-24 10:09:53 +00:00
}
Reference in New Issue Copy Permalink