2005-01-07 02:35:34 +00:00
|
|
|
/*-
|
Reimplement the netisr framework in order to support parallel netisr
threads:
- Support up to one netisr thread per CPU, each processings its own
workstream, or set of per-protocol queues. Threads may be bound
to specific CPUs, or allowed to migrate, based on a global policy.
In the future it would be desirable to support topology-centric
policies, such as "one netisr per package".
- Allow each protocol to advertise an ordering policy, which can
currently be one of:
NETISR_POLICY_SOURCE: packets must maintain ordering with respect to
an implicit or explicit source (such as an interface or socket).
NETISR_POLICY_FLOW: make use of mbuf flow identifiers to place work,
as well as allowing protocols to provide a flow generation function
for mbufs without flow identifers (m2flow). Falls back on
NETISR_POLICY_SOURCE if now flow ID is available.
NETISR_POLICY_CPU: allow protocols to inspect and assign a CPU for
each packet handled by netisr (m2cpuid).
- Provide utility functions for querying the number of workstreams
being used, as well as a mapping function from workstream to CPU ID,
which protocols may use in work placement decisions.
- Add explicit interfaces to get and set per-protocol queue limits, and
get and clear drop counters, which query data or apply changes across
all workstreams.
- Add a more extensible netisr registration interface, in which
protocols declare 'struct netisr_handler' structures for each
registered NETISR_ type. These include name, handler function,
optional mbuf to flow ID function, optional mbuf to CPU ID function,
queue limit, and ordering policy. Padding is present to allow these
to be expanded in the future. If no queue limit is declared, then
a default is used.
- Queue limits are now per-workstream, and raised from the previous
IFQ_MAXLEN default of 50 to 256.
- All protocols are updated to use the new registration interface, and
with the exception of netnatm, default queue limits. Most protocols
register as NETISR_POLICY_SOURCE, except IPv4 and IPv6, which use
NETISR_POLICY_FLOW, and will therefore take advantage of driver-
generated flow IDs if present.
- Formalize a non-packet based interface between interface polling and
the netisr, rather than having polling pretend to be two protocols.
Provide two explicit hooks in the netisr worker for start and end
events for runs: netisr_poll() and netisr_pollmore(), as well as a
function, netisr_sched_poll(), to allow the polling code to schedule
netisr execution. DEVICE_POLLING still embeds single-netisr
assumptions in its implementation, so for now if it is compiled into
the kernel, a single and un-bound netisr thread is enforced
regardless of tunable configuration.
In the default configuration, the new netisr implementation maintains
the same basic assumptions as the previous implementation: a single,
un-bound worker thread processes all deferred work, and direct dispatch
is enabled by default wherever possible.
Performance measurement shows a marginal performance improvement over
the old implementation due to the use of batched dequeue.
An rmlock is used to synchronize use and registration/unregistration
using the framework; currently, synchronized use is disabled
(replicating current netisr policy) due to a measurable 3%-6% hit in
ping-pong micro-benchmarking. It will be enabled once further rmlock
optimization has taken place. However, in practice, netisrs are
rarely registered or unregistered at runtime.
A new man page for netisr will follow, but since one doesn't currently
exist, it hasn't been updated.
This change is not appropriate for MFC, although the polling shutdown
handler should be merged to 7-STABLE.
Bump __FreeBSD_version.
Reviewed by: bz
2009-06-01 10:41:38 +00:00
|
|
|
* Copyright (c) 2004-2009 Robert N. M. Watson
|
2007-01-08 17:58:27 +00:00
|
|
|
* All rights reserved.
|
|
|
|
*
|
|
|
|
* Redistribution and use in source and binary forms, with or without
|
|
|
|
* modification, are permitted provided that the following conditions
|
|
|
|
* are met:
|
|
|
|
* 1. Redistributions of source code must retain the above copyright
|
|
|
|
* notice, this list of conditions and the following disclaimer.
|
|
|
|
* 2. Redistributions in binary form must reproduce the above copyright
|
|
|
|
* notice, this list of conditions and the following disclaimer in the
|
|
|
|
* documentation and/or other materials provided with the distribution.
|
|
|
|
*
|
|
|
|
* THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
|
|
|
|
* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
|
|
|
|
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
|
|
|
|
* ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
|
|
|
|
* FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
|
|
|
|
* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
|
|
|
|
* OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
|
|
|
|
* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
|
|
|
|
* LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
|
|
|
|
* OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
|
|
|
|
* SUCH DAMAGE.
|
|
|
|
*
|
2007-01-12 15:07:51 +00:00
|
|
|
* Copyright (c) 1990, 1994 Regents of The University of Michigan.
|
2005-01-07 02:35:34 +00:00
|
|
|
* All Rights Reserved.
|
|
|
|
*
|
|
|
|
* Permission to use, copy, modify, and distribute this software and
|
|
|
|
* its documentation for any purpose and without fee is hereby granted,
|
|
|
|
* provided that the above copyright notice appears in all copies and
|
|
|
|
* that both that copyright notice and this permission notice appear
|
|
|
|
* in supporting documentation, and that the name of The University
|
|
|
|
* of Michigan not be used in advertising or publicity pertaining to
|
|
|
|
* distribution of the software without specific, written prior
|
|
|
|
* permission. This software is supplied as is without expressed or
|
|
|
|
* implied warranties of any kind.
|
|
|
|
*
|
|
|
|
* This product includes software developed by the University of
|
|
|
|
* California, Berkeley and its contributors.
|
|
|
|
*
|
|
|
|
* Research Systems Unix Group
|
|
|
|
* The University of Michigan
|
|
|
|
* c/o Wesley Craig
|
|
|
|
* 535 W. William Street
|
|
|
|
* Ann Arbor, Michigan
|
|
|
|
* +1-313-764-2278
|
|
|
|
* netatalk@umich.edu
|
2000-10-29 16:06:56 +00:00
|
|
|
*
|
|
|
|
* $FreeBSD$
|
1996-05-24 01:35:45 +00:00
|
|
|
*/
|
|
|
|
|
|
|
|
#include <sys/param.h>
|
|
|
|
#include <sys/systm.h>
|
1997-09-02 01:19:47 +00:00
|
|
|
#include <sys/malloc.h>
|
1996-05-24 01:35:45 +00:00
|
|
|
#include <sys/mbuf.h>
|
|
|
|
#include <sys/socket.h>
|
|
|
|
#include <sys/socketvar.h>
|
|
|
|
#include <sys/protosw.h>
|
|
|
|
#include <net/if.h>
|
2013-10-26 17:58:36 +00:00
|
|
|
#include <net/if_var.h>
|
1996-05-24 01:35:45 +00:00
|
|
|
#include <net/route.h>
|
2003-03-04 23:19:55 +00:00
|
|
|
#include <net/netisr.h>
|
1996-05-24 01:35:45 +00:00
|
|
|
|
1996-09-10 08:32:01 +00:00
|
|
|
#include <netatalk/at.h>
|
|
|
|
#include <netatalk/at_var.h>
|
|
|
|
#include <netatalk/ddp_var.h>
|
2004-03-19 07:21:22 +00:00
|
|
|
#include <netatalk/ddp_pcb.h>
|
1996-05-24 01:35:45 +00:00
|
|
|
#include <netatalk/at_extern.h>
|
|
|
|
|
1998-02-09 06:11:36 +00:00
|
|
|
static u_long ddp_sendspace = DDP_MAXSZ; /* Max ddp size + 1 (ddp_type) */
|
2004-03-22 03:24:10 +00:00
|
|
|
static u_long ddp_recvspace = 10 * (587 + sizeof(struct sockaddr_at));
|
1996-05-24 01:35:45 +00:00
|
|
|
|
Reimplement the netisr framework in order to support parallel netisr
threads:
- Support up to one netisr thread per CPU, each processings its own
workstream, or set of per-protocol queues. Threads may be bound
to specific CPUs, or allowed to migrate, based on a global policy.
In the future it would be desirable to support topology-centric
policies, such as "one netisr per package".
- Allow each protocol to advertise an ordering policy, which can
currently be one of:
NETISR_POLICY_SOURCE: packets must maintain ordering with respect to
an implicit or explicit source (such as an interface or socket).
NETISR_POLICY_FLOW: make use of mbuf flow identifiers to place work,
as well as allowing protocols to provide a flow generation function
for mbufs without flow identifers (m2flow). Falls back on
NETISR_POLICY_SOURCE if now flow ID is available.
NETISR_POLICY_CPU: allow protocols to inspect and assign a CPU for
each packet handled by netisr (m2cpuid).
- Provide utility functions for querying the number of workstreams
being used, as well as a mapping function from workstream to CPU ID,
which protocols may use in work placement decisions.
- Add explicit interfaces to get and set per-protocol queue limits, and
get and clear drop counters, which query data or apply changes across
all workstreams.
- Add a more extensible netisr registration interface, in which
protocols declare 'struct netisr_handler' structures for each
registered NETISR_ type. These include name, handler function,
optional mbuf to flow ID function, optional mbuf to CPU ID function,
queue limit, and ordering policy. Padding is present to allow these
to be expanded in the future. If no queue limit is declared, then
a default is used.
- Queue limits are now per-workstream, and raised from the previous
IFQ_MAXLEN default of 50 to 256.
- All protocols are updated to use the new registration interface, and
with the exception of netnatm, default queue limits. Most protocols
register as NETISR_POLICY_SOURCE, except IPv4 and IPv6, which use
NETISR_POLICY_FLOW, and will therefore take advantage of driver-
generated flow IDs if present.
- Formalize a non-packet based interface between interface polling and
the netisr, rather than having polling pretend to be two protocols.
Provide two explicit hooks in the netisr worker for start and end
events for runs: netisr_poll() and netisr_pollmore(), as well as a
function, netisr_sched_poll(), to allow the polling code to schedule
netisr execution. DEVICE_POLLING still embeds single-netisr
assumptions in its implementation, so for now if it is compiled into
the kernel, a single and un-bound netisr thread is enforced
regardless of tunable configuration.
In the default configuration, the new netisr implementation maintains
the same basic assumptions as the previous implementation: a single,
un-bound worker thread processes all deferred work, and direct dispatch
is enabled by default wherever possible.
Performance measurement shows a marginal performance improvement over
the old implementation due to the use of batched dequeue.
An rmlock is used to synchronize use and registration/unregistration
using the framework; currently, synchronized use is disabled
(replicating current netisr policy) due to a measurable 3%-6% hit in
ping-pong micro-benchmarking. It will be enabled once further rmlock
optimization has taken place. However, in practice, netisrs are
rarely registered or unregistered at runtime.
A new man page for netisr will follow, but since one doesn't currently
exist, it hasn't been updated.
This change is not appropriate for MFC, although the polling shutdown
handler should be merged to 7-STABLE.
Bump __FreeBSD_version.
Reviewed by: bz
2009-06-01 10:41:38 +00:00
|
|
|
static const struct netisr_handler atalk1_nh = {
|
|
|
|
.nh_name = "atalk1",
|
|
|
|
.nh_handler = at1intr,
|
|
|
|
.nh_proto = NETISR_ATALK1,
|
|
|
|
.nh_policy = NETISR_POLICY_SOURCE,
|
|
|
|
};
|
|
|
|
|
|
|
|
static const struct netisr_handler atalk2_nh = {
|
|
|
|
.nh_name = "atalk2",
|
|
|
|
.nh_handler = at2intr,
|
|
|
|
.nh_proto = NETISR_ATALK2,
|
|
|
|
.nh_policy = NETISR_POLICY_SOURCE,
|
|
|
|
};
|
|
|
|
|
|
|
|
static const struct netisr_handler aarp_nh = {
|
|
|
|
.nh_name = "aarp",
|
|
|
|
.nh_handler = aarpintr,
|
|
|
|
.nh_proto = NETISR_AARP,
|
|
|
|
.nh_policy = NETISR_POLICY_SOURCE,
|
|
|
|
};
|
1996-05-24 01:35:45 +00:00
|
|
|
|
1997-05-13 21:01:45 +00:00
|
|
|
static int
|
2001-09-12 08:38:13 +00:00
|
|
|
ddp_attach(struct socket *so, int proto, struct thread *td)
|
1997-05-13 21:01:45 +00:00
|
|
|
{
|
2007-01-12 15:07:51 +00:00
|
|
|
int error = 0;
|
1997-05-13 21:01:45 +00:00
|
|
|
|
2009-02-04 20:04:32 +00:00
|
|
|
KASSERT(sotoddpcb(so) == NULL, ("ddp_attach: ddp != NULL"));
|
1997-05-13 21:01:45 +00:00
|
|
|
|
2004-07-12 18:39:59 +00:00
|
|
|
/*
|
|
|
|
* Allocate socket buffer space first so that it's present
|
|
|
|
* before first use.
|
|
|
|
*/
|
|
|
|
error = soreserve(so, ddp_sendspace, ddp_recvspace);
|
|
|
|
if (error)
|
|
|
|
return (error);
|
|
|
|
|
|
|
|
DDP_LIST_XLOCK();
|
2004-03-22 03:24:10 +00:00
|
|
|
error = at_pcballoc(so);
|
2004-07-12 18:39:59 +00:00
|
|
|
DDP_LIST_XUNLOCK();
|
|
|
|
return (error);
|
1997-05-13 21:01:45 +00:00
|
|
|
}
|
1996-05-24 01:35:45 +00:00
|
|
|
|
Chance protocol switch method pru_detach() so that it returns void
rather than an error. Detaches do not "fail", they other occur or
the protocol flags SS_PROTOREF to take ownership of the socket.
soclose() no longer looks at so_pcb to see if it's NULL, relying
entirely on the protocol to decide whether it's time to free the
socket or not using SS_PROTOREF. so_pcb is now entirely owned and
managed by the protocol code. Likewise, no longer test so_pcb in
other socket functions, such as soreceive(), which have no business
digging into protocol internals.
Protocol detach routines no longer try to free the socket on detach,
this is performed in the socket code if the protocol permits it.
In rts_detach(), no longer test for rp != NULL in detach, and
likewise in other protocols that don't permit a NULL so_pcb, reduce
the incidence of testing for it during detach.
netinet and netinet6 are not fully updated to this change, which
will be in an upcoming commit. In their current state they may leak
memory or panic.
MFC after: 3 months
2006-04-01 15:42:02 +00:00
|
|
|
static void
|
1997-05-13 21:01:45 +00:00
|
|
|
ddp_detach(struct socket *so)
|
|
|
|
{
|
2007-01-12 15:07:51 +00:00
|
|
|
struct ddpcb *ddp;
|
1997-05-13 21:01:45 +00:00
|
|
|
|
2004-03-22 03:24:10 +00:00
|
|
|
ddp = sotoddpcb(so);
|
2006-03-17 20:40:17 +00:00
|
|
|
KASSERT(ddp != NULL, ("ddp_detach: ddp == NULL"));
|
2004-07-12 18:39:59 +00:00
|
|
|
|
|
|
|
DDP_LIST_XLOCK();
|
|
|
|
DDP_LOCK(ddp);
|
2004-03-22 03:24:10 +00:00
|
|
|
at_pcbdetach(so, ddp);
|
2004-07-12 18:39:59 +00:00
|
|
|
DDP_LIST_XUNLOCK();
|
1997-05-13 21:01:45 +00:00
|
|
|
}
|
1996-05-24 01:35:45 +00:00
|
|
|
|
1997-05-13 21:01:45 +00:00
|
|
|
static int
|
2001-09-12 08:38:13 +00:00
|
|
|
ddp_bind(struct socket *so, struct sockaddr *nam, struct thread *td)
|
1997-05-13 21:01:45 +00:00
|
|
|
{
|
2007-01-12 15:07:51 +00:00
|
|
|
struct ddpcb *ddp;
|
|
|
|
int error = 0;
|
1997-05-13 21:01:45 +00:00
|
|
|
|
2004-03-22 03:24:10 +00:00
|
|
|
ddp = sotoddpcb(so);
|
2006-03-17 20:40:17 +00:00
|
|
|
KASSERT(ddp != NULL, ("ddp_bind: ddp == NULL"));
|
2006-04-01 16:54:37 +00:00
|
|
|
|
2004-07-12 18:39:59 +00:00
|
|
|
DDP_LIST_XLOCK();
|
|
|
|
DDP_LOCK(ddp);
|
2001-09-12 08:38:13 +00:00
|
|
|
error = at_pcbsetaddr(ddp, nam, td);
|
2004-07-12 18:39:59 +00:00
|
|
|
DDP_UNLOCK(ddp);
|
|
|
|
DDP_LIST_XUNLOCK();
|
1997-05-13 21:01:45 +00:00
|
|
|
return (error);
|
|
|
|
}
|
1996-05-24 01:35:45 +00:00
|
|
|
|
1997-05-13 21:01:45 +00:00
|
|
|
static int
|
2001-09-12 08:38:13 +00:00
|
|
|
ddp_connect(struct socket *so, struct sockaddr *nam, struct thread *td)
|
1997-05-13 21:01:45 +00:00
|
|
|
{
|
2007-01-12 15:07:51 +00:00
|
|
|
struct ddpcb *ddp;
|
|
|
|
int error = 0;
|
1997-05-13 21:01:45 +00:00
|
|
|
|
2004-03-22 03:24:10 +00:00
|
|
|
ddp = sotoddpcb(so);
|
2006-03-17 20:40:17 +00:00
|
|
|
KASSERT(ddp != NULL, ("ddp_connect: ddp == NULL"));
|
2006-04-01 16:54:37 +00:00
|
|
|
|
2004-07-12 18:39:59 +00:00
|
|
|
DDP_LIST_XLOCK();
|
|
|
|
DDP_LOCK(ddp);
|
2004-03-22 03:24:10 +00:00
|
|
|
if (ddp->ddp_fsat.sat_port != ATADDR_ANYPORT) {
|
2007-01-12 15:07:51 +00:00
|
|
|
DDP_UNLOCK(ddp);
|
|
|
|
DDP_LIST_XUNLOCK();
|
|
|
|
return (EISCONN);
|
1996-05-24 01:35:45 +00:00
|
|
|
}
|
|
|
|
|
2004-07-12 18:39:59 +00:00
|
|
|
error = at_pcbconnect( ddp, nam, td );
|
|
|
|
DDP_UNLOCK(ddp);
|
|
|
|
DDP_LIST_XUNLOCK();
|
2004-03-22 03:24:10 +00:00
|
|
|
if (error == 0)
|
2007-01-12 15:07:51 +00:00
|
|
|
soisconnected(so);
|
2004-03-22 03:24:10 +00:00
|
|
|
return (error);
|
1997-05-13 21:01:45 +00:00
|
|
|
}
|
1996-05-24 01:35:45 +00:00
|
|
|
|
1997-05-13 21:01:45 +00:00
|
|
|
static int
|
|
|
|
ddp_disconnect(struct socket *so)
|
|
|
|
{
|
2007-01-12 15:07:51 +00:00
|
|
|
struct ddpcb *ddp;
|
1997-05-13 21:01:45 +00:00
|
|
|
|
2004-03-22 03:24:10 +00:00
|
|
|
ddp = sotoddpcb(so);
|
2006-03-17 20:40:17 +00:00
|
|
|
KASSERT(ddp != NULL, ("ddp_disconnect: ddp == NULL"));
|
2006-04-01 16:54:37 +00:00
|
|
|
|
2004-07-12 18:39:59 +00:00
|
|
|
DDP_LOCK(ddp);
|
2004-03-22 03:24:10 +00:00
|
|
|
if (ddp->ddp_fsat.sat_addr.s_node == ATADDR_ANYNODE) {
|
2007-01-12 15:07:51 +00:00
|
|
|
DDP_UNLOCK(ddp);
|
|
|
|
return (ENOTCONN);
|
1996-05-24 01:35:45 +00:00
|
|
|
}
|
1997-05-13 21:01:45 +00:00
|
|
|
|
2004-03-22 03:24:10 +00:00
|
|
|
at_pcbdisconnect(ddp);
|
1997-05-13 21:01:45 +00:00
|
|
|
ddp->ddp_fsat.sat_addr.s_node = ATADDR_ANYNODE;
|
2004-07-12 18:39:59 +00:00
|
|
|
DDP_UNLOCK(ddp);
|
2004-03-22 03:24:10 +00:00
|
|
|
soisdisconnected(so);
|
|
|
|
return (0);
|
1997-05-13 21:01:45 +00:00
|
|
|
}
|
1996-05-24 01:35:45 +00:00
|
|
|
|
1997-05-13 21:01:45 +00:00
|
|
|
static int
|
|
|
|
ddp_shutdown(struct socket *so)
|
|
|
|
{
|
1998-12-07 21:58:50 +00:00
|
|
|
|
2009-02-04 20:04:32 +00:00
|
|
|
KASSERT(sotoddpcb(so) != NULL, ("ddp_shutdown: ddp == NULL"));
|
2006-04-01 16:54:37 +00:00
|
|
|
|
2004-03-22 03:24:10 +00:00
|
|
|
socantsendmore(so);
|
|
|
|
return (0);
|
1997-05-13 21:01:45 +00:00
|
|
|
}
|
1996-05-24 01:35:45 +00:00
|
|
|
|
1997-05-13 21:01:45 +00:00
|
|
|
static int
|
1997-08-16 19:16:27 +00:00
|
|
|
ddp_send(struct socket *so, int flags, struct mbuf *m, struct sockaddr *addr,
|
2007-01-12 15:07:51 +00:00
|
|
|
struct mbuf *control, struct thread *td)
|
1997-05-13 21:01:45 +00:00
|
|
|
{
|
2007-01-12 15:07:51 +00:00
|
|
|
struct ddpcb *ddp;
|
|
|
|
int error = 0;
|
1997-05-13 21:01:45 +00:00
|
|
|
|
2004-03-22 03:24:10 +00:00
|
|
|
ddp = sotoddpcb(so);
|
2006-03-17 20:40:17 +00:00
|
|
|
KASSERT(ddp != NULL, ("ddp_send: ddp == NULL"));
|
1997-05-13 21:01:45 +00:00
|
|
|
|
2006-04-01 16:54:37 +00:00
|
|
|
if (control && control->m_len)
|
2004-03-22 03:24:10 +00:00
|
|
|
return (EINVAL);
|
1996-05-24 01:35:45 +00:00
|
|
|
|
2004-03-22 03:57:01 +00:00
|
|
|
if (addr != NULL) {
|
2004-07-12 18:39:59 +00:00
|
|
|
DDP_LIST_XLOCK();
|
|
|
|
DDP_LOCK(ddp);
|
2004-03-22 03:24:10 +00:00
|
|
|
if (ddp->ddp_fsat.sat_port != ATADDR_ANYPORT) {
|
2004-07-12 18:39:59 +00:00
|
|
|
error = EISCONN;
|
|
|
|
goto out;
|
1997-05-13 21:01:45 +00:00
|
|
|
}
|
1996-05-24 01:35:45 +00:00
|
|
|
|
2001-09-12 08:38:13 +00:00
|
|
|
error = at_pcbconnect(ddp, addr, td);
|
2004-07-12 18:39:59 +00:00
|
|
|
if (error == 0) {
|
|
|
|
error = ddp_output(m, so);
|
|
|
|
at_pcbdisconnect(ddp);
|
1997-05-13 21:01:45 +00:00
|
|
|
}
|
2004-07-12 18:39:59 +00:00
|
|
|
out:
|
|
|
|
DDP_UNLOCK(ddp);
|
|
|
|
DDP_LIST_XUNLOCK();
|
1996-05-24 01:35:45 +00:00
|
|
|
} else {
|
2004-07-12 18:39:59 +00:00
|
|
|
DDP_LOCK(ddp);
|
|
|
|
if (ddp->ddp_fsat.sat_port == ATADDR_ANYPORT)
|
|
|
|
error = ENOTCONN;
|
|
|
|
else
|
|
|
|
error = ddp_output(m, so);
|
|
|
|
DDP_UNLOCK(ddp);
|
1996-05-24 01:35:45 +00:00
|
|
|
}
|
2004-03-22 03:24:10 +00:00
|
|
|
return (error);
|
1997-05-13 21:01:45 +00:00
|
|
|
}
|
1996-05-24 01:35:45 +00:00
|
|
|
|
2006-07-21 17:11:15 +00:00
|
|
|
/*
|
|
|
|
* XXXRW: This is never called because we only invoke abort on stream
|
|
|
|
* protocols.
|
|
|
|
*/
|
2006-04-01 15:15:05 +00:00
|
|
|
static void
|
1997-05-13 21:01:45 +00:00
|
|
|
ddp_abort(struct socket *so)
|
|
|
|
{
|
|
|
|
struct ddpcb *ddp;
|
|
|
|
|
2004-03-22 03:24:10 +00:00
|
|
|
ddp = sotoddpcb(so);
|
2006-03-17 20:40:17 +00:00
|
|
|
KASSERT(ddp != NULL, ("ddp_abort: ddp == NULL"));
|
2006-04-01 16:54:37 +00:00
|
|
|
|
2004-07-12 18:39:59 +00:00
|
|
|
DDP_LOCK(ddp);
|
2006-07-21 17:11:15 +00:00
|
|
|
at_pcbdisconnect(ddp);
|
|
|
|
DDP_UNLOCK(ddp);
|
2006-08-05 14:14:34 +00:00
|
|
|
soisdisconnected(so);
|
2006-07-21 17:11:15 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
static void
|
|
|
|
ddp_close(struct socket *so)
|
|
|
|
{
|
|
|
|
struct ddpcb *ddp;
|
|
|
|
|
|
|
|
ddp = sotoddpcb(so);
|
|
|
|
KASSERT(ddp != NULL, ("ddp_close: ddp == NULL"));
|
|
|
|
|
|
|
|
DDP_LOCK(ddp);
|
|
|
|
at_pcbdisconnect(ddp);
|
|
|
|
DDP_UNLOCK(ddp);
|
2006-08-05 14:14:34 +00:00
|
|
|
soisdisconnected(so);
|
1996-05-24 01:35:45 +00:00
|
|
|
}
|
|
|
|
|
2004-03-19 07:21:22 +00:00
|
|
|
void
|
|
|
|
ddp_init(void)
|
1996-05-24 01:35:45 +00:00
|
|
|
{
|
2007-01-12 15:07:51 +00:00
|
|
|
|
2004-07-12 18:39:59 +00:00
|
|
|
DDP_LIST_LOCK_INIT();
|
2009-06-24 21:36:09 +00:00
|
|
|
TAILQ_INIT(&at_ifaddrhead);
|
Reimplement the netisr framework in order to support parallel netisr
threads:
- Support up to one netisr thread per CPU, each processings its own
workstream, or set of per-protocol queues. Threads may be bound
to specific CPUs, or allowed to migrate, based on a global policy.
In the future it would be desirable to support topology-centric
policies, such as "one netisr per package".
- Allow each protocol to advertise an ordering policy, which can
currently be one of:
NETISR_POLICY_SOURCE: packets must maintain ordering with respect to
an implicit or explicit source (such as an interface or socket).
NETISR_POLICY_FLOW: make use of mbuf flow identifiers to place work,
as well as allowing protocols to provide a flow generation function
for mbufs without flow identifers (m2flow). Falls back on
NETISR_POLICY_SOURCE if now flow ID is available.
NETISR_POLICY_CPU: allow protocols to inspect and assign a CPU for
each packet handled by netisr (m2cpuid).
- Provide utility functions for querying the number of workstreams
being used, as well as a mapping function from workstream to CPU ID,
which protocols may use in work placement decisions.
- Add explicit interfaces to get and set per-protocol queue limits, and
get and clear drop counters, which query data or apply changes across
all workstreams.
- Add a more extensible netisr registration interface, in which
protocols declare 'struct netisr_handler' structures for each
registered NETISR_ type. These include name, handler function,
optional mbuf to flow ID function, optional mbuf to CPU ID function,
queue limit, and ordering policy. Padding is present to allow these
to be expanded in the future. If no queue limit is declared, then
a default is used.
- Queue limits are now per-workstream, and raised from the previous
IFQ_MAXLEN default of 50 to 256.
- All protocols are updated to use the new registration interface, and
with the exception of netnatm, default queue limits. Most protocols
register as NETISR_POLICY_SOURCE, except IPv4 and IPv6, which use
NETISR_POLICY_FLOW, and will therefore take advantage of driver-
generated flow IDs if present.
- Formalize a non-packet based interface between interface polling and
the netisr, rather than having polling pretend to be two protocols.
Provide two explicit hooks in the netisr worker for start and end
events for runs: netisr_poll() and netisr_pollmore(), as well as a
function, netisr_sched_poll(), to allow the polling code to schedule
netisr execution. DEVICE_POLLING still embeds single-netisr
assumptions in its implementation, so for now if it is compiled into
the kernel, a single and un-bound netisr thread is enforced
regardless of tunable configuration.
In the default configuration, the new netisr implementation maintains
the same basic assumptions as the previous implementation: a single,
un-bound worker thread processes all deferred work, and direct dispatch
is enabled by default wherever possible.
Performance measurement shows a marginal performance improvement over
the old implementation due to the use of batched dequeue.
An rmlock is used to synchronize use and registration/unregistration
using the framework; currently, synchronized use is disabled
(replicating current netisr policy) due to a measurable 3%-6% hit in
ping-pong micro-benchmarking. It will be enabled once further rmlock
optimization has taken place. However, in practice, netisrs are
rarely registered or unregistered at runtime.
A new man page for netisr will follow, but since one doesn't currently
exist, it hasn't been updated.
This change is not appropriate for MFC, although the polling shutdown
handler should be merged to 7-STABLE.
Bump __FreeBSD_version.
Reviewed by: bz
2009-06-01 10:41:38 +00:00
|
|
|
netisr_register(&atalk1_nh);
|
|
|
|
netisr_register(&atalk2_nh);
|
|
|
|
netisr_register(&aarp_nh);
|
1996-05-24 01:35:45 +00:00
|
|
|
}
|
|
|
|
|
2004-03-19 07:21:22 +00:00
|
|
|
#if 0
|
1996-05-24 01:35:45 +00:00
|
|
|
static void
|
2004-03-22 03:24:10 +00:00
|
|
|
ddp_clean(void)
|
1996-05-24 01:35:45 +00:00
|
|
|
{
|
2007-01-12 15:07:51 +00:00
|
|
|
struct ddpcp *ddp;
|
1996-05-24 01:35:45 +00:00
|
|
|
|
2007-01-12 15:07:51 +00:00
|
|
|
for (ddp = ddpcb_list; ddp != NULL; ddp = ddp->ddp_next)
|
|
|
|
at_pcbdetach(ddp->ddp_socket, ddp);
|
|
|
|
DDP_LIST_LOCK_DESTROY();
|
1996-05-24 01:35:45 +00:00
|
|
|
}
|
2004-03-19 07:21:22 +00:00
|
|
|
#endif
|
|
|
|
|
1997-05-13 21:01:45 +00:00
|
|
|
static int
|
2007-05-11 10:20:51 +00:00
|
|
|
at_getpeeraddr(struct socket *so, struct sockaddr **nam)
|
1997-05-13 21:01:45 +00:00
|
|
|
{
|
2007-01-12 15:07:51 +00:00
|
|
|
|
2004-03-22 03:24:10 +00:00
|
|
|
return (EOPNOTSUPP);
|
1997-05-13 21:01:45 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
static int
|
2007-05-11 10:20:51 +00:00
|
|
|
at_getsockaddr(struct socket *so, struct sockaddr **nam)
|
1997-05-13 21:01:45 +00:00
|
|
|
{
|
|
|
|
struct ddpcb *ddp;
|
1998-12-07 21:58:50 +00:00
|
|
|
|
2004-03-22 03:24:10 +00:00
|
|
|
ddp = sotoddpcb(so);
|
2007-05-11 10:20:51 +00:00
|
|
|
KASSERT(ddp != NULL, ("at_getsockaddr: ddp == NULL"));
|
2006-03-25 18:54:17 +00:00
|
|
|
|
2004-07-12 18:39:59 +00:00
|
|
|
DDP_LOCK(ddp);
|
2004-03-22 03:24:10 +00:00
|
|
|
at_sockaddr(ddp, nam);
|
2004-07-12 18:39:59 +00:00
|
|
|
DDP_UNLOCK(ddp);
|
2004-03-22 03:24:10 +00:00
|
|
|
return (0);
|
1997-05-13 21:01:45 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
struct pr_usrreqs ddp_usrreqs = {
|
2004-11-08 14:44:54 +00:00
|
|
|
.pru_abort = ddp_abort,
|
|
|
|
.pru_attach = ddp_attach,
|
|
|
|
.pru_bind = ddp_bind,
|
|
|
|
.pru_connect = ddp_connect,
|
|
|
|
.pru_control = at_control,
|
|
|
|
.pru_detach = ddp_detach,
|
|
|
|
.pru_disconnect = ddp_disconnect,
|
2007-05-11 10:20:51 +00:00
|
|
|
.pru_peeraddr = at_getpeeraddr,
|
2004-11-08 14:44:54 +00:00
|
|
|
.pru_send = ddp_send,
|
|
|
|
.pru_shutdown = ddp_shutdown,
|
2007-05-11 10:20:51 +00:00
|
|
|
.pru_sockaddr = at_getsockaddr,
|
2006-07-21 17:11:15 +00:00
|
|
|
.pru_close = ddp_close,
|
1997-05-13 21:01:45 +00:00
|
|
|
};
|