Commit Graph

23 Commits

Author SHA1 Message Date
Navdeep Parhar
c66c36a454 Overhaul the stid allocator so that it can be used for IPv6 servers
too.  The entry for an IPv6 server in the TCAM takes up the equivalent
of two ordinary stids and must be properly aligned too.

MFC after:	1 week
2013-01-11 00:07:01 +00:00
Navdeep Parhar
b174b65819 cxgbe(4): Add functions to help synchronize "slow" operations (those not
on the fast data path) and use them instead of frobbing the adapter lock
and busy flag directly.

Other changes made while reworking all slow operations:
- Wait for the reply to a filter request (add/delete).  This guarantees
  that the operation is complete by the time the ioctl returns.
- Tidy up the tid_info structure.
- Do not allow the tx queue size to be set to something that's not a
  power of 2.

MFC after:	1 week
2013-01-10 23:56:50 +00:00
Gleb Smirnoff
c6499eccad Mechanically substitute flags from historic mbuf allocator with
malloc(9) flags in sys/dev.
2012-12-04 09:32:43 +00:00
Navdeep Parhar
d588c1f9ba cxgbe/tom: Handle the case where the chip falls out of DDP mode by
itself.  The hole in the receive sequence space corresponds to the
number of bytes placed directly up to that point.

MFC after:	1 week
2012-11-29 19:39:27 +00:00
Navdeep Parhar
000da5202e cxgbe/tom: Add a flag to indicate that the L2 table entry for an
embryonic connection has been setup and never attempt to abort a tid
before this is done.  This fixes a bad race where a listening socket is
closed when the driver is in the middle of step (b) here.  The symptom
of this were "ARP miss" errors from the driver followed by tid leaks.

A hardware-offloaded passive open works this way:

a) A SYN "hits" the TCAM entry for a server tid and the chip delivers it
to the queue associated with the server tid (say, queue A).  It waits
for a response from the driver telling it what to do.

b) The driver decides it is ok to proceed.  It adds the new tid to the
list of embryonic connections associated with the server tid and then
hands off the SYN to the kernel's syncache to make sure that the kernel
okays it too.  If it does then the driver provides an L2 table entry,
queue id (say, queue B), etc. and instructs the chip to send the SYN/ACK
response.

c) The chip delivers a status to queue B depending on how the third step
of the 3-way handshake goes.  The driver removes the tid from its list
of embryonic connections and either expands the syncache entry or
destroys the tid.  In any case all subsequent messages for the new tid
will be delivered to queue B, not queue A.  Anything running in queue B
knows that the L2 entry has long been setup and the new flag is of no
interest from here on.  If the listener is closed it will deal with
so_comp as normal.

MFC after:	1 week
2012-11-29 19:10:04 +00:00
Navdeep Parhar
e1e0aa8d3e cxgbe/tom: Plug mbuf leak.
MFC after:	3 days
2012-11-16 00:21:54 +00:00
Navdeep Parhar
274b95d3ac Make sure the inp hasn't been dropped before trying to access its socket
and tcpcb.

MFC after:	3 days
2012-11-06 20:22:39 +00:00
Navdeep Parhar
c5dddc6694 Remove the tid from the software table (and bump down the in-use
counter) when the syncache doesn't want the driver to reply to an
incoming SYN.  This fixes a harmless bug where tids_in_use would
go out of sync with the hardware counter.

MFC after:	3 days
2012-11-06 18:58:57 +00:00
Ed Schouten
6b946662d9 Prefer __containerof() over __member2struct().
The former works better with qualifiers, but also properly type checks
the input pointer.
2012-10-19 13:26:40 +00:00
Navdeep Parhar
fb51680534 Always provide sndbuf and MSS values in a flowc command, even when the
driver is going to abort the connection right after the flowc.

MFC after:	3 days
2012-10-17 16:37:16 +00:00
Navdeep Parhar
726793aa8d Whitespace cleanup.
MFC after:	3 days
2012-10-17 05:08:35 +00:00
Kevin Lo
9823d52705 Revert previous commit...
Pointyhat to:	kevlo (myself)
2012-10-10 08:36:38 +00:00
Kevin Lo
a10cee30c9 Prefer NULL over 0 for pointers 2012-10-09 08:27:40 +00:00
Navdeep Parhar
57c60f98b8 Use native FreeBSD facilities everywhere except the shared code in common/
MFC after:	1 week
2012-09-13 09:10:10 +00:00
Navdeep Parhar
87a74dd6e3 Deal with the case where a syncache entry added by the TOE driver is
evicted from the syncache but a later syncache_expand succeeds because
of syncookies.  The TOE driver has to resort to more direct means to
install its hooks in the socket in this case.
2012-08-21 22:23:17 +00:00
Navdeep Parhar
f9796f4373 Avoid a NULL pointer dereference. 2012-08-21 19:45:19 +00:00
Navdeep Parhar
36fd646e38 Cannot hold a mutex around vm_fault_quick_hold_pages, so don't. Tweak
some comments while here.
2012-08-21 19:39:09 +00:00
Navdeep Parhar
c91bcaaab5 Minor cleanup: use bitwise ops instead of pointless wrappers around
setbit/clrbit.
2012-08-21 18:30:16 +00:00
Navdeep Parhar
06fd9875aa Correctly handle the case where an inp has already been dropped by the time
the TOE driver reports that an active open failed.  toe_connect_failed is
supposed to handle this but it should be provided the inpcb instead of the
tcpcb which may no longer be around.
2012-08-21 18:09:33 +00:00
Navdeep Parhar
e682d02e12 Support for TCP DDP (Direct Data Placement) in the T4 TOE module.
Basically, this is automatic rx zero copy when feasible.  TCP payload is
DMA'd directly into the userspace buffer described by the uio submitted
in soreceive by an application.

- Works with sockets that are being handled by the TCP offload engine
  of a T4 chip (you need t4_tom.ko module loaded after cxgbe, and an
  "ifconfig +toe" on the cxgbe interface).
- Does not require any modification to the application.
- Not enabled by default.  Use hw.t4nex.<X>.toe.ddp="1" to enable it.
2012-08-17 00:49:29 +00:00
Navdeep Parhar
1f1b5a0f6f Add a routine (t4_set_tcb_field) to update arbitrary parts of a hardware
TCB.  Filters are programmed by modifying the TCB too (via a different
routine) and the reply to any TCB update is delivered via a
CPL_SET_TCB_RPL.  Figure out whether the reply is for a filter-write or
something else and route it appropriately.

MFC after:	2 weeks
2012-08-16 20:15:29 +00:00
Navdeep Parhar
2a5f6b0e65 cxgbe(4): update to firmware interface 1.5.2.0; updates to shared code. 2012-06-22 07:51:15 +00:00
Navdeep Parhar
09fe63205c - Updated TOE support in the kernel.
- Stateful TCP offload drivers for Terminator 3 and 4 (T3 and T4) ASICs.
  These are available as t3_tom and t4_tom modules that augment cxgb(4)
  and cxgbe(4) respectively.  The cxgb/cxgbe drivers continue to work as
  usual with or without these extra features.

- iWARP driver for Terminator 3 ASIC (kernel verbs).  T4 iWARP in the
  works and will follow soon.

Build-tested with make universe.

30s overview
============
What interfaces support TCP offload?  Look for TOE4 and/or TOE6 in the
capabilities of an interface:
# ifconfig -m | grep TOE

Enable/disable TCP offload on an interface (just like any other ifnet
capability):
# ifconfig cxgbe0 toe
# ifconfig cxgbe0 -toe

Which connections are offloaded?  Look for toe4 and/or toe6 in the
output of netstat and sockstat:
# netstat -np tcp | grep toe
# sockstat -46c | grep toe

Reviewed by:	bz, gnn
Sponsored by:	Chelsio communications.
MFC after:	~3 months (after 9.1, and after ensuring MFC is feasible)
2012-06-19 07:34:13 +00:00