freebsd-dev/sys/netipx
Robert Watson d4b5cae49b Reimplement the netisr framework in order to support parallel netisr
threads:

- Support up to one netisr thread per CPU, each processings its own
  workstream, or set of per-protocol queues.  Threads may be bound
  to specific CPUs, or allowed to migrate, based on a global policy.

  In the future it would be desirable to support topology-centric
  policies, such as "one netisr per package".

- Allow each protocol to advertise an ordering policy, which can
  currently be one of:

  NETISR_POLICY_SOURCE: packets must maintain ordering with respect to
    an implicit or explicit source (such as an interface or socket).

  NETISR_POLICY_FLOW: make use of mbuf flow identifiers to place work,
    as well as allowing protocols to provide a flow generation function
    for mbufs without flow identifers (m2flow).  Falls back on
    NETISR_POLICY_SOURCE if now flow ID is available.

  NETISR_POLICY_CPU: allow protocols to inspect and assign a CPU for
    each packet handled by netisr (m2cpuid).

- Provide utility functions for querying the number of workstreams
  being used, as well as a mapping function from workstream to CPU ID,
  which protocols may use in work placement decisions.

- Add explicit interfaces to get and set per-protocol queue limits, and
  get and clear drop counters, which query data or apply changes across
  all workstreams.

- Add a more extensible netisr registration interface, in which
  protocols declare 'struct netisr_handler' structures for each
  registered NETISR_ type.  These include name, handler function,
  optional mbuf to flow ID function, optional mbuf to CPU ID function,
  queue limit, and ordering policy.  Padding is present to allow these
  to be expanded in the future.  If no queue limit is declared, then
  a default is used.

- Queue limits are now per-workstream, and raised from the previous
  IFQ_MAXLEN default of 50 to 256.

- All protocols are updated to use the new registration interface, and
  with the exception of netnatm, default queue limits.  Most protocols
  register as NETISR_POLICY_SOURCE, except IPv4 and IPv6, which use
  NETISR_POLICY_FLOW, and will therefore take advantage of driver-
  generated flow IDs if present.

- Formalize a non-packet based interface between interface polling and
  the netisr, rather than having polling pretend to be two protocols.
  Provide two explicit hooks in the netisr worker for start and end
  events for runs: netisr_poll() and netisr_pollmore(), as well as a
  function, netisr_sched_poll(), to allow the polling code to schedule
  netisr execution.  DEVICE_POLLING still embeds single-netisr
  assumptions in its implementation, so for now if it is compiled into
  the kernel, a single and un-bound netisr thread is enforced
  regardless of tunable configuration.

In the default configuration, the new netisr implementation maintains
the same basic assumptions as the previous implementation: a single,
un-bound worker thread processes all deferred work, and direct dispatch
is enabled by default wherever possible.

Performance measurement shows a marginal performance improvement over
the old implementation due to the use of batched dequeue.

An rmlock is used to synchronize use and registration/unregistration
using the framework; currently, synchronized use is disabled
(replicating current netisr policy) due to a measurable 3%-6% hit in
ping-pong micro-benchmarking.  It will be enabled once further rmlock
optimization has taken place.  However, in practice, netisrs are
rarely registered or unregistered at runtime.

A new man page for netisr will follow, but since one doesn't currently
exist, it hasn't been updated.

This change is not appropriate for MFC, although the polling shutdown
handler should be merged to 7-STABLE.

Bump __FreeBSD_version.

Reviewed by:	bz
2009-06-01 10:41:38 +00:00
..
ipx_cksum.c Use ANSI C function declarations throughout netipx. 2007-05-11 10:38:34 +00:00
ipx_if.h Remove IPX over IP tunneling support, which allows IPX routing over IP 2007-06-13 14:01:43 +00:00
ipx_input.c Reimplement the netisr framework in order to support parallel netisr 2009-06-01 10:41:38 +00:00
ipx_outputfl.c Change if_output to take a struct route as its fourth argument in order 2009-04-16 20:30:28 +00:00
ipx_pcb.c Retire the MALLOC and FREE macros. They are an abomination unto style(9). 2008-10-23 15:53:51 +00:00
ipx_pcb.h Use ANSI C function declarations throughout netipx. 2007-05-11 10:38:34 +00:00
ipx_proto.c Add code to allow the system to handle multiple routing tables. 2008-05-09 23:03:00 +00:00
ipx_usrreq.c Add missing "goto set_head" for SO_IPX_CHECKSUM; otherwise we fall through 2008-12-11 10:29:35 +00:00
ipx_var.h Use ANSI C function declarations throughout netipx. 2007-05-11 10:38:34 +00:00
ipx.c Remove the suser(9) interface from the kernel. It has been replaced from 2008-09-17 15:49:44 +00:00
ipx.h Remove IPX over IP tunneling support, which allows IPX routing over IP 2007-06-13 14:01:43 +00:00
README Add a simple netipx TODO list to the end of README, since there are a 2006-03-27 09:10:09 +00:00
spx_debug.c Make tcpstates[] static, and make sure TCPSTATES is defined before 2007-07-30 11:06:42 +00:00
spx_debug.h Make tcpstates[] static, and make sure TCPSTATES is defined before 2007-07-30 11:06:42 +00:00
spx_reass.c Staticize spx_remque() now that it's only used from spx_reass.c. 2009-05-25 13:52:51 +00:00
spx_timer.h Factor out UCB and my copyrights from copyrights of Mike Mitchell; 2007-01-08 22:14:00 +00:00
spx_usrreq.c Add missing call to ipx_pcbdetach() during SPX socket tear-down: not 2009-05-25 13:34:03 +00:00
spx_var.h Staticize spx_remque() now that it's only used from spx_reass.c. 2009-05-25 13:52:51 +00:00
spx.h Rather than store a skeleton IPX header in an mbuf hung off the SPX 2009-05-25 11:50:58 +00:00

$FreeBSD$

This protocol implements IPX/SPX over Ethernet_II frame type 0x8137.
Please note: the SPX implementation may require further work and testing
to insure proper operation.

Mike Mitchell, Network Engineer
AMTECH Systems Corporation, Technology and Manufacturing
8600 Jefferson Street, Albuquerque, New Mexico 87113 (505) 856-8000
supervisor@alb.asctmd.com

John Hay
Some Company
Some Address
jhay@mikom.csir.co.za

Adapted for multi-processor, multi-threaded network stack by Robert N. M.
Watson, Computer Laboratory, University of Cambridge.

--- Copyright Information ---
/*-

Copyright (c) 1984, 1985, 1986, 1987, 1993
The Regents of the University of California.  All rights reserved.

Modifications Copyright (c) 1995, Mike Mitchell
Modifications Copyright (c) 1995, John Hay
Modifications Copyright (c) 2004-2006 Robert N. M. Watson

*/

--- TODO ---

(1) netipx default socket buffer sizes are very small by contemporary
    standards, and should be increased following testing and measurement.

(2) SPX will free the PCB and socket buffer memory on close(), which means
    close() in effects terminates the transfer of any outstanding buffered
    but unsent data.  As with TCP, it should instead grab its own
    reference to the socket so that it is not released, as hold onto it
    until the data transfer is complete.

(3) Raw socket capture of IPX output intercepts packets in the SPX output
    routine in order to feed them back into the raw socket.  This results
    in recursion into the socket code in the transmit path; instead,
    captured packets should be fed into a netisr that reinjects them into
    raw sockets from a new (asynchronous) context.

(4) IPX over IP encapsulation needs work to make it properly MPSAFE.