Commit Graph

252 Commits

Author SHA1 Message Date
np
85e970b274 There is no need to log anything for a ctrlq stall or restart. These are
normal events.

Approved by:	gnn (mentor)
MFC after:	1 month
2009-09-09 18:55:18 +00:00
jhb
81de4d9abd Fill the reverse RSS map with 0xff's so that the subsequent loop to
calculate the values will work properly.

Reviewed by:	np
MFC after:	1 month
2009-09-04 21:00:45 +00:00
rwatson
5c6699ad3d Many network stack subsystems use a single global data structure to hold
all pertinent statatistics for the subsystem.  These structures are
sometimes "borrowed" by kernel modules that require a place to store
statistics for similar events.

Add KPI accessor functions for statistics structures referenced by kernel
modules so that they no longer encode certain specifics of how the data
structures are named and stored.  This change is intended to make it
easier to move to per-CPU network stats following 8.0-RELEASE.

The following modules are affected by this change:

      if_bridge
      if_cxgb
      if_gif
      ip_mroute
      ipdivert
      pf

In practice, most of these statistics consumers should, in fact, maintain
their own statistics data structures rather than borrowing structures
from the base network stack.  However, that change is too agressive for
this point in the release cycle.

Reviewed by:	bz
Approved by:	re (kib)
2009-08-02 19:43:32 +00:00
rwatson
fb9ffed650 Merge the remainder of kern_vimage.c and vimage.h into vnet.c and
vnet.h, we now use jails (rather than vimages) as the abstraction
for virtualization management, and what remained was specific to
virtual network stacks.  Minor cleanups are done in the process,
and comments updated to reflect these changes.

Reviewed by:	bz
Approved by:	re (vimage blanket)
2009-08-01 19:26:27 +00:00
rwatson
57ca4583e7 Build on Jeff Roberson's linker-set based dynamic per-CPU allocator
(DPCPU), as suggested by Peter Wemm, and implement a new per-virtual
network stack memory allocator.  Modify vnet to use the allocator
instead of monolithic global container structures (vinet, ...).  This
change solves many binary compatibility problems associated with
VIMAGE, and restores ELF symbols for virtualized global variables.

Each virtualized global variable exists as a "reference copy", and also
once per virtual network stack.  Virtualized global variables are
tagged at compile-time, placing the in a special linker set, which is
loaded into a contiguous region of kernel memory.  Virtualized global
variables in the base kernel are linked as normal, but those in modules
are copied and relocated to a reserved portion of the kernel's vnet
region with the help of a the kernel linker.

Virtualized global variables exist in per-vnet memory set up when the
network stack instance is created, and are initialized statically from
the reference copy.  Run-time access occurs via an accessor macro, which
converts from the current vnet and requested symbol to a per-vnet
address.  When "options VIMAGE" is not compiled into the kernel, normal
global ELF symbols will be used instead and indirection is avoided.

This change restores static initialization for network stack global
variables, restores support for non-global symbols and types, eliminates
the need for many subsystem constructors, eliminates large per-subsystem
structures that caused many binary compatibility issues both for
monitoring applications (netstat) and kernel modules, removes the
per-function INIT_VNET_*() macros throughout the stack, eliminates the
need for vnet_symmap ksym(2) munging, and eliminates duplicate
definitions of virtualized globals under VIMAGE_GLOBALS.

Bump __FreeBSD_version and update UPDATING.

Portions submitted by:  bz
Reviewed by:            bz, zec
Discussed with:         gnn, jamie, jeff, jhb, julian, sam
Suggested by:           peter
Approved by:            re (kensmith)
2009-07-14 22:48:30 +00:00
lstewart
d561c3ad27 Fix a buglet that slipped into r195654. My buildworld/buildkernel sanity
check missed this because cxgb's TOM is currently commented out of the build
system.

Submitted by:	Navdeep Parhar <np at FreeBSD dot org>
Approved by:	re (kensmith), kensmith (mentor temporarily unavailable)
2009-07-14 11:53:21 +00:00
lstewart
31cb6fd0f6 Replace struct tcpopt with a proxy toeopt struct in the TOE driver interface to
the TCP syncache. This returns struct tcpopt to being private within the TCP
implementation, thus allowing it to be modified without ABI concerns.

The patch breaks the ABI. Bump __FreeBSD_version to 800103 accordingly. The cxgb
driver is the only TOE consumer affected by this change, and needs to be
recompiled along with the kernel.

Suggested by:	rwatson
Reviewed by:	rwatson, kmacy
Approved by:	re (kensmith), kensmith (mentor temporarily unavailable)
2009-07-13 11:51:02 +00:00
np
dcfe611845 Fix cxgb(4) panic with jumbo frames.
Reviewed by:	kmacy
Approved by:	re (kib), gnn (mentor)
2009-07-09 19:27:58 +00:00
rwatson
12c75db4f1 Use if_maddr_rlock() instead of IF_ADDR_LOCK() to protect access to
if_multiaddrs in if_cxgb.

Approved by:	re (kib)
MFC after:	6 weeks
2009-06-26 19:04:08 +00:00
np
a14b0d6132 mvec routines should have no knowledge of the SG engine.
Reviewed by:	kmacy
Approved by:	gnn (mentor)
2009-06-25 21:50:15 +00:00
np
a1a0aeecef Various ifmedia related fixes in cxgb(4), including:
- build ifmedia list based on phy->caps, not string comparisons.
- rebuild media list when a transceiver change is detected.
- return EOPNOTSUPP instead of ENXIO in cxgb_media_status.

Approved by:	gnn (mentor)
MFC after:	2 weeks.
2009-06-24 21:56:05 +00:00
bz
0808d0b1a6 After cleaning up rt_tables from vnet.h and cleaning up opt_route.h
a lot of files no longer need route.h either. Garbage collect them.
While here remove now unneeded vnet.h #includes as well.
2009-06-23 17:03:45 +00:00
np
a7cd46d35b Fix cxgb's ifmedia ioctl handling. Also fixed a comment.
Reviewed by:	kmacy
Approved by:	gnn (mentor)
2009-06-22 21:42:57 +00:00
rwatson
3c02410d55 Add a new function, ifa_ifwithaddr_check(), which rather than returning
a pointer to an ifaddr matching the passed socket address, returns a
boolean indicating whether one was present.  In the (near) future,
ifa_ifwithaddr() will return a referenced ifaddr rather than a raw
ifaddr pointer, and the new wrapper will allow callers that care only
about the boolean condition to avoid having to free that reference.

MFC after:	3 weeks
2009-06-22 10:59:34 +00:00
kmacy
a6e30dd398 fix !x86 cxgb compile 2009-06-21 01:17:38 +00:00
kmacy
d10b0f80a7 fix typo in conditional 2009-06-20 19:09:41 +00:00
kmacy
4f52e22a6e - fix dma map handling for !x86 case
- fix allocation failure handing in refill_fl
2009-06-20 18:57:14 +00:00
kmacy
473a60e6d2 Greatly simplify cxgb by removing almost all of the custom mbuf management logic
- remove mbuf iovec - useful, but adds too much complexity when isolated to
   the driver

- remove driver private caching - insufficient benefit over UMA to justify
  the added complexity and maintenance overhead

- remove separate logic for managing multiple transmit queues, with the
  new drbr routines the control flow can be made to much more closely resemble
  legacy drivers

- remove dedicated service threads, with per-cpu callouts one can get the same
  benefit much more simply by registering a callout 1 tick in the future if there
  are still buffered packets

- remove embedded mbuf usage - Jeffr's changes will (I hope) soon be integrated
  greatly reducing the overhead of using kernel APIs for reference counting
  clusters

- add hysteresis to descriptor coalescing logic

- add coalesce threshold sysctls to allow users to decide at run-time
  between optimizing for forwarding / UDP or optimizing for TCP

- add once per second watchdog to effectively close the very rare races
  occurring from coalescing

- incorporate Navdeep's changes to the initialization path required to
  convert port and adapter locks back to ordinary mutexes (silencing BPF
  LOR complaints)

- enable prefetches in get_packet and tx cleaning

Reviewed by:	navdeep@
MFC after:	2 weeks
2009-06-19 23:34:32 +00:00
sam
df30f1c9f2 r193336 moved ifq_detach to if_free which broke if_alloc followed
by if_free (w/o doing if_attach); move ifq_attach to if_alloc and
rename ifq_attach/detach to ifq_init/ifq_delete to better identify
their purpose

Reviewed by:	jhb, kmacy
2009-06-15 19:50:03 +00:00
gnn
807f4c1bf8 Re-add the send queue tunable for people who do not use buffering.
Reviewed by:	jhb
MFC after:	3 days
2009-06-11 21:32:26 +00:00
gnn
6753234074 Add a missing error statistic, the number of FCS errors on receive.
Reviewed by:	jhb
MFC after:	1 day
2009-06-10 14:34:56 +00:00
kmacy
3f394b4e78 - add drbr routines for accessing #qentries and conditionally dequeueing
- track bytes enqueued in buf_ring
2009-06-09 19:19:16 +00:00
bz
b7ff2bdc20 After r193232 rt_tables in vnet.h are no longer indirectly dependent on
the ROUTETABLES kernel option thus there is no need to include opt_route.h
anymore in all consumers of vnet.h and no longer depend on it for module
builds.

Remove the hidden include in flowtable.h as well and leave the two
explicit #includes in ip_input.c and ip_output.c.
2009-06-08 19:57:35 +00:00
jhb
a1af9ecca4 Rework socket upcalls to close some races with setup/teardown of upcalls.
- Each socket upcall is now invoked with the appropriate socket buffer
  locked.  It is not permissible to call soisconnected() with this lock
  held; however, so socket upcalls now return an integer value.  The two
  possible values are SU_OK and SU_ISCONNECTED.  If an upcall returns
  SU_ISCONNECTED, then the soisconnected() will be invoked on the
  socket after the socket buffer lock is dropped.
- A new API is provided for setting and clearing socket upcalls.  The
  API consists of soupcall_set() and soupcall_clear().
- To simplify locking, each socket buffer now has a separate upcall.
- When a socket upcall returns SU_ISCONNECTED, the upcall is cleared from
  the receive socket buffer automatically.  Note that a SO_SND upcall
  should never return SU_ISCONNECTED.
- All this means that accept filters should now return SU_ISCONNECTED
  instead of calling soisconnected() directly.  They also no longer need
  to explicitly clear the upcall on the new socket.
- The HTTP accept filter still uses soupcall_set() to manage its internal
  state machine, but other accept filters no longer have any explicit
  knowlege of socket upcall internals aside from their return value.
- The various RPC client upcalls currently drop the socket buffer lock
  while invoking soreceive() as a temporary band-aid.  The plan for
  the future is to add a new flag to allow soreceive() to be called with
  the socket buffer locked.
- The AIO callback for socket I/O is now also invoked with the socket
  buffer locked.  Previously sowakeup() would drop the socket buffer
  lock only to call aio_swake() which immediately re-acquired the socket
  buffer lock for the duration of the function call.

Discussed with:	rwatson, rmacklem
2009-06-01 21:17:03 +00:00
zec
9956d85f16 Update VNET base pointer setting macro to use a correct source of
vnet context.

Approved by:	julian (mentor)
2009-06-01 21:10:23 +00:00
gnn
7cc11d24d8 Rework interrupt bringup and teardown.
Calculate the exact number of vectors we'll use before calling
pci_alloc_msix.  Don't grab nine all the time.

Call cxgb_setup_interrupts once per T3, not once per port.  Ditto
for cxgb_teardown_interrupts.

Don't leak resources when interrupt setup fails in the middle.

Obtained from:	Navdeep Parhar
MFC after:	10 days
2009-05-27 20:13:36 +00:00
gnn
56fa0af673 Partial reversion of previous commit. The CXGB_SHUTDOWN flag does NOT
need to be inverted when doing an ifconfig down of an interface.

Pointed out by:	Navdeep Parhar
MFC after: 1 week
2009-05-22 18:26:47 +00:00
gnn
a0b8a56168 Fix a possible panic cxgb_controller_attach() routine that would occur
only if prepping the adapter failed.

Slight adjustment to comments.

Fix a bug whereby downing the interface didn't preven it from
processing packets.

Submitted by:	Navdeep Parhar
MFC after:	1 week
2009-05-22 15:06:03 +00:00
gnn
9857e6abfd Integrate three changes from Chelsio.
1) Add a sysctl that will say what type of PHYs exist on the card.
2) Fix a bug that occurs when an AEL 2005 PHY resets without a transciever
in the card.
3) Unify the PHY link detection code.

Obtained from:	Navdeep Parhar
MFC after:	10 days
2009-05-21 15:08:03 +00:00
gnn
baf3db66b3 Modified the attach and detach routines to handle bringing ports up
and down more cleanly.  This addresses a problem where if we have the
link flap during boot the driver would lock up the system.

Reviewed by:	jhb
MFC after:	1 week
2009-05-21 14:43:12 +00:00
imp
3ca3ea7190 We no longer need to use d_thread_t, migrate to struct thread *. 2009-05-20 17:29:21 +00:00
kmacy
ec4d3eb88b fix bug introduced by last change
Submitted by:	Navdeep Parhar
2009-05-12 03:30:25 +00:00
zec
d78a1b1a82 Change the curvnet variable from a global const struct vnet *,
previously always pointing to the default vnet context, to a
dynamically changing thread-local one.  The currvnet context
should be set on entry to networking code via CURVNET_SET() macros,
and reverted to previous state via CURVNET_RESTORE().  Recursions
on curvnet are permitted, though strongly discuouraged.

This change should have no functional impact on nooptions VIMAGE
kernel builds, where CURVNET_* macros expand to whitespace.

The curthread->td_vnet (aka curvnet) variable's purpose is to be an
indicator of the vnet context in which the current network-related
operation takes place, in case we cannot deduce the current vnet
context from any other source, such as by looking at mbuf's
m->m_pkthdr.rcvif->if_vnet, sockets's so->so_vnet etc.  Moreover, so
far curvnet has turned out to be an invaluable consistency checking
aid: it helps to catch cases when sockets, ifnets or any other
vnet-aware structures may have leaked from one vnet to another.

The exact placement of the CURVNET_SET() / CURVNET_RESTORE() macros
was a result of an empirical iterative process, whith an aim to
reduce recursions on CURVNET_SET() to a minimum, while still reducing
the scope of CURVNET_SET() to networking only operations - the
alternative would be calling CURVNET_SET() on each system call entry.
In general, curvnet has to be set in three typicall cases: when
processing socket-related requests from userspace or from within the
kernel; when processing inbound traffic flowing from device drivers
to upper layers of the networking stack, and when executing
timer-driven networking functions.

This change also introduces a DDB subcommand to show the list of all
vnet instances.

Approved by:	julian (mentor)
2009-05-05 10:56:12 +00:00
kmacy
cbcf0bd3a0 simplify by removing dead code 2009-04-27 22:54:30 +00:00
rwatson
b79ff9a30d Update stats in struct tcpstat using two new macros, TCPSTAT_ADD() and
TCPSTAT_INC(), rather than directly manipulating the fields across the
kernel.  This will make it easier to change the implementation of
these statistics, such as using per-CPU versions of the data structures.

MFC after:	3 days
2009-04-11 22:07:19 +00:00
kmacy
fbd3646842 Import "flowid" support for serializing flows across transmit queues
Reviewed by:	rwatson and jeli
2009-04-10 06:16:14 +00:00
gnn
463cb1ae28 Minor updates to the Chelsio driver, including removing an LOR.
Submitted by:	Navdeep Parhar at Chelsio
Reviewed by:	gnn
MFC after: 	3 weeks
2009-03-23 19:58:26 +00:00
gnn
739eecddb9 Fix a bug in the recent update to the Chelsio driver.
The tick routine was not being restarted in the init_locked routine
which could resulted in loss of carrier when updating the MTU.

Submitted by:	Navdeep Parhar at Chelsio
MFC after:	3 weeks
2009-03-21 17:09:00 +00:00
rwatson
9997d44640 Prefer ENETDOWN to ENXIO when returning queuing errors due to a link
down, interface down, etc, with if_cxgb's if_transmit routine.

MFC after:	3 days
Reviewed by:	kmacy
2009-03-10 22:35:45 +00:00
gnn
73c9516af4 Update the Chelsio driver to the latest bits from Chelsio
Firmware upgraded to 7.1.0 (from 5.0.0).
T3C EEPROM and SRAM added; Code to update eeprom/sram fixed.
fl_empty and rx_fifo_ovfl counters can be observed via sysctl.
Two new cxgbtool commands to get uP logic analyzer info and uP IOQs
Synced up with Chelsio's "common code" (as of 03/03/09)

Submitted by:	 Navdeep Parhar at Chelsio
Reviewed by:	gnn
MFC after:	2 weeks
2009-03-10 19:22:45 +00:00
bz
df2be82cec For all files including net/vnet.h directly include opt_route.h and
net/route.h.

Remove the hidden include of opt_route.h and net/route.h from net/vnet.h.

We need to make sure that both opt_route.h and net/route.h are included
before net/vnet.h because of the way MRT figures out the number of FIBs
from the kernel option. If we do not, we end up with the default number
of 1 when including net/vnet.h and array sizes are wrong.

This does not change the list of files which depend on opt_route.h
but we can identify them now more easily.
2009-02-27 14:12:05 +00:00
gnn
7e56c10808 Check in the actual module recognition code for the Chelsio
driver.

Obtained from:	Chelsio Inc.
2008-12-18 14:21:35 +00:00
bz
ea0d9d2e9a Use inc_flags instead of the inc_isipv6 alias which so far
had been the only flag with random usage patterns.
Switch inc_flags to be used as a real bit field by using
INC_ISIPV6 with bitops to check for the 'isipv6' condition.

While here fix a place or two where in case of v4 inc_flags
were not properly initialized before.[1]

Found by:	rwatson during review [1]
Discussed with:	rwatson
Reviewed by:	rwatson
MFC after:	4 weeks
2008-12-17 12:52:34 +00:00
qingli
ec826ad5c7 This main goals of this project are:
1. separating L2 tables (ARP, NDP) from the L3 routing tables
2. removing as much locking dependencies among these layers as
   possible to allow for some parallelism in the search operations
3. simplify the logic in the routing code,

The most notable end result is the obsolescent of the route
cloning (RTF_CLONING) concept, which translated into code reduction
in both IPv4 ARP and IPv6 NDP related modules, and size reduction in
struct rtentry{}. The change in design obsoletes the semantics of
RTF_CLONING, RTF_WASCLONE and RTF_LLINFO routing flags. The userland
applications such as "arp" and "ndp" have been modified to reflect
those changes. The output from "netstat -r" shows only the routing
entries.

Quite a few developers have contributed to this project in the
past: Glebius Smirnoff, Luigi Rizzo, Alessandro Cerri, and
Andre Oppermann. And most recently:

- Kip Macy revised the locking code completely, thus completing
  the last piece of the puzzle, Kip has also been conducting
  active functional testing
- Sam Leffler has helped me improving/refactoring the code, and
  provided valuable reviews
- Julian Elischer setup the perforce tree for me and has helped
  me maintaining that branch before the svn conversion
2008-12-15 06:10:57 +00:00
gnn
0b3880cbd3 Bug fix to support N310 version of Chelsio cards (board ID 1088).
Obtained from:	Chelsio Inc.
MFC after:	3 days
2008-12-06 02:10:53 +00:00
gnn
5832ddeb00 Re submit code to print the part and serial number for Chelsio cards.
The original code was accidentally removed in another commit.

MFC after: 1 day
2008-12-05 21:40:11 +00:00
gnn
886616b211 Fix a bug with the ael1006 PHY. The bug shows up as persistent but incomplete
packet loss, of between 10-30%. The fix is to put the PHY into
and take it out of local loopback mode when resetting the interface.

Obtained from:	Chelsio Inc.
MFC after:	3 days
2008-12-04 20:32:53 +00:00
bz
604d89458a Rather than using hidden includes (with cicular dependencies),
directly include only the header files needed. This reduces the
unneeded spamming of various headers into lots of files.

For now, this leaves us with very few modules including vnet.h
and thus needing to depend on opt_route.h.

Reviewed by:	brooks, gnn, des, zec, imp
Sponsored by:	The FreeBSD Foundation
2008-12-02 21:37:28 +00:00
gnn
cc0f9f6606 Bug fix from Chelsio which addresses the issue of the device resetting
when it sees only received packets.  In some cases where a device only
recieves data it mistakenly thinks that its transmitting side is broken
and resets the device.

Obtained from:	Chelsio Inc.
MFC after:	3 days
2008-12-02 15:42:47 +00:00
kmacy
15d87f5061 - fix bug where dnsperf would stop transmitting after a few seconds
- break complex conditionals in to multiple lines to avoid wrapping
- remove copious unused debug statements
- be more aggressive about cleaning in the calling thread
- eliminate usage of ENOSPC
- increase number of iterations that cxgbsp can do
- eliminate "initerr" usage to simplify ENOBUFS handling
- when coalescing pass all packets to BPF
- always set overrun if hardware queue is full
2008-12-02 07:01:18 +00:00