d/freebsd-nq
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
b0819f9877
freebsd-nq/sys/dev/cxgbe/t4_sge.c

4505 lines
118 KiB
C
Raw Normal View History

cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
/*-
* Copyright (c) 2011 Chelsio Communications, Inc.
* All rights reserved.
* Written by: Navdeep Parhar <np@FreeBSD.org>
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
*
* THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
* FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
* OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
* LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
* OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
* SUCH DAMAGE.
*/
#include <sys/cdefs.h>
__FBSDID("$FreeBSD$");
#include "opt_inet.h"
cxgbe(4): support for IPv6 TSO and LRO. Submitted by: bz (this is a modified version of that patch)
2012-06-29 19:51:06 +00:00
#include "opt_inet6.h"
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
#include <sys/types.h>
Include necessary headers that now are available due to pollution via if_var.h. Sponsored by: Netflix Sponsored by: Nginx, Inc.
2013-10-28 07:29:16 +00:00
#include <sys/eventhandler.h>
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
#include <sys/mbuf.h>
#include <sys/socket.h>
#include <sys/kernel.h>
- Updated TOE support in the kernel. - Stateful TCP offload drivers for Terminator 3 and 4 (T3 and T4) ASICs. These are available as t3_tom and t4_tom modules that augment cxgb(4) and cxgbe(4) respectively. The cxgb/cxgbe drivers continue to work as usual with or without these extra features. - iWARP driver for Terminator 3 ASIC (kernel verbs). T4 iWARP in the works and will follow soon. Build-tested with make universe. 30s overview ============ What interfaces support TCP offload? Look for TOE4 and/or TOE6 in the capabilities of an interface: # ifconfig -m | grep TOE Enable/disable TCP offload on an interface (just like any other ifnet capability): # ifconfig cxgbe0 toe # ifconfig cxgbe0 -toe Which connections are offloaded? Look for toe4 and/or toe6 in the output of netstat and sockstat: # netstat -np tcp | grep toe # sockstat -46c | grep toe Reviewed by: bz, gnn Sponsored by: Chelsio communications. MFC after: ~3 months (after 9.1, and after ensuring MFC is feasible)
2012-06-19 07:34:13 +00:00
#include <sys/kdb.h>
Resume tx immediately in response to an SGE egress update from the hardware. MFC after: 1 week
2011-03-05 03:06:38 +00:00
#include <sys/malloc.h>
#include <sys/queue.h>
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
#include <sys/sbuf.h>
Resume tx immediately in response to an SGE egress update from the hardware. MFC after: 1 week
2011-03-05 03:06:38 +00:00
#include <sys/taskqueue.h>
Merge r254386 from user/np/cxl_tuning. Add an INET|INET6 check missing in said revision. r254386: Flush inactive LRO entries periodically.
2013-08-29 06:26:22 +00:00
#include <sys/time.h>
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
#include <sys/sysctl.h>
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
#include <sys/smp.h>
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
#include <net/bpf.h>
#include <net/ethernet.h>
#include <net/if.h>
#include <net/if_vlan_var.h>
#include <netinet/in.h>
#include <netinet/ip.h>
cxgbe(4): support for IPv6 TSO and LRO. Submitted by: bz (this is a modified version of that patch)
2012-06-29 19:51:06 +00:00
#include <netinet/ip6.h>
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
#include <netinet/tcp.h>
Fix kernel build on amd64 after r256118, since the machine/md_var.h header is not implicitly included there. So include it explicitly. Approved by: re (delphij) Pointy hat to: dim MFC after: 3 days X-MFC-With: r256118
2013-10-07 22:30:03 +00:00
#include <machine/md_var.h>
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
#include <vm/vm.h>
#include <vm/pmap.h>
cxgbe(4): netmap support for Terminator 5 (T5) based 10G/40G cards. Netmap gets its own hardware-assisted virtual interface and won't take over or disrupt the "normal" interface in any way. You can use both simultaneously. For kernels with DEV_NETMAP, cxgbe(4) carves out an ncxl<N> interface (note the 'n' prefix) in the hardware to accompany each cxl<N> interface. These two ifnet's per port share the same wire but really are separate interfaces in the hardware and software. Each gets its own L2 MAC addresses (unicast and multicast), MTU, checksum caps, etc. You should run netmap on the 'n' interfaces only, that's what they are for. With this, pkt-gen is able to transmit > 45Mpps out of a single 40G port of a T580 card. 2 port tx is at ~56Mpps total (28M + 28M) as of now. Single port receive is at 33Mpps but this is very much a work in progress. I expect it to be closer to 40Mpps once done. In any case the current effort can already saturate multiple 10G ports of a T5 card at the smallest legal packet size. T4 gear is totally untested. trantor:~# ./pkt-gen -i ncxl0 -f tx -D 00:07:43:ab:cd:ef 881.952141 main [1621] interface is ncxl0 881.952250 extract_ip_range [275] range is 10.0.0.1:0 to 10.0.0.1:0 881.952253 extract_ip_range [275] range is 10.1.0.1:0 to 10.1.0.1:0 881.962540 main [1804] mapped 334980KB at 0x801dff000 Sending on netmap:ncxl0: 4 queues, 1 threads and 1 cpus. 10.0.0.1 -> 10.1.0.1 (00:00:00:00:00:00 -> 00:07:43:ab:cd:ef) 881.962562 main [1882] Sending 512 packets every 0.000000000 s 881.962563 main [1884] Wait 2 secs for phy reset 884.088516 main [1886] Ready... 884.088535 nm_open [457] overriding ifname ncxl0 ringid 0x0 flags 0x1 884.088607 sender_body [996] start 884.093246 sender_body [1064] drop copy 885.090435 main_thread [1418] 45206353 pps (45289533 pkts in 1001840 usec) 886.091600 main_thread [1418] 45322792 pps (45375593 pkts in 1001165 usec) 887.092435 main_thread [1418] 45313992 pps (45351784 pkts in 1000834 usec) 888.094434 main_thread [1418] 45315765 pps (45406397 pkts in 1002000 usec) 889.095434 main_thread [1418] 45333218 pps (45378551 pkts in 1001000 usec) 890.097434 main_thread [1418] 45315247 pps (45405877 pkts in 1002000 usec) 891.099434 main_thread [1418] 45326515 pps (45417168 pkts in 1002000 usec) 892.101434 main_thread [1418] 45333039 pps (45423705 pkts in 1002000 usec) 893.103434 main_thread [1418] 45324105 pps (45414708 pkts in 1001999 usec) 894.105434 main_thread [1418] 45318042 pps (45408723 pkts in 1002001 usec) 895.106434 main_thread [1418] 45332430 pps (45377762 pkts in 1001000 usec) 896.107434 main_thread [1418] 45338072 pps (45383410 pkts in 1001000 usec) ... Relnotes: Yes Sponsored by: Chelsio Communications.
2014-05-27 18:18:41 +00:00
#ifdef DEV_NETMAP
#include <machine/bus.h>
#include <sys/selinfo.h>
#include <net/if_var.h>
#include <net/netmap.h>
#include <dev/netmap/netmap_kern.h>
#endif
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
#include "common/common.h"
#include "common/t4_regs.h"
#include "common/t4_regs_values.h"
#include "common/t4_msg.h"
cxgbe(4): Add support for Chelsio's Terminator 5 (aka T5) ASIC. This includes support for the NIC and TOE features of the 40G, 10G, and 1G/100M cards based on the T5. The ASIC is mostly backward compatible with the Terminator 4 so cxgbe(4) has been updated instead of writing a brand new driver. T5 cards will show up as cxl (short for cxlgb) ports attached to the t5nex bus driver. Sponsored by: Chelsio
2013-03-30 02:26:20 +00:00
#ifdef T4_PKT_TIMESTAMP
#define RX_COPY_THRESHOLD (MINCLSIZE - 8)
#else
#define RX_COPY_THRESHOLD MINCLSIZE
#endif
Convert some fixed parameters to tunables (with reasonable default values). - cong_drop specifies what to do on congestion: nothing, backpressure, or drop. - fl_pktshift specifies the padding before Ethernet payload. - fl_pad specifies the boundary upto which to pad Ethernet payload. - spg_len controls the length of the status page. MFC after: 2 weeks
2012-08-14 21:47:41 +00:00
/*
* Ethernet frames are DMA'd at this byte offset into the freelist buffer.
* 0-7 are valid values.
*/
cxgbe(4): netmap support for Terminator 5 (T5) based 10G/40G cards. Netmap gets its own hardware-assisted virtual interface and won't take over or disrupt the "normal" interface in any way. You can use both simultaneously. For kernels with DEV_NETMAP, cxgbe(4) carves out an ncxl<N> interface (note the 'n' prefix) in the hardware to accompany each cxl<N> interface. These two ifnet's per port share the same wire but really are separate interfaces in the hardware and software. Each gets its own L2 MAC addresses (unicast and multicast), MTU, checksum caps, etc. You should run netmap on the 'n' interfaces only, that's what they are for. With this, pkt-gen is able to transmit > 45Mpps out of a single 40G port of a T580 card. 2 port tx is at ~56Mpps total (28M + 28M) as of now. Single port receive is at 33Mpps but this is very much a work in progress. I expect it to be closer to 40Mpps once done. In any case the current effort can already saturate multiple 10G ports of a T5 card at the smallest legal packet size. T4 gear is totally untested. trantor:~# ./pkt-gen -i ncxl0 -f tx -D 00:07:43:ab:cd:ef 881.952141 main [1621] interface is ncxl0 881.952250 extract_ip_range [275] range is 10.0.0.1:0 to 10.0.0.1:0 881.952253 extract_ip_range [275] range is 10.1.0.1:0 to 10.1.0.1:0 881.962540 main [1804] mapped 334980KB at 0x801dff000 Sending on netmap:ncxl0: 4 queues, 1 threads and 1 cpus. 10.0.0.1 -> 10.1.0.1 (00:00:00:00:00:00 -> 00:07:43:ab:cd:ef) 881.962562 main [1882] Sending 512 packets every 0.000000000 s 881.962563 main [1884] Wait 2 secs for phy reset 884.088516 main [1886] Ready... 884.088535 nm_open [457] overriding ifname ncxl0 ringid 0x0 flags 0x1 884.088607 sender_body [996] start 884.093246 sender_body [1064] drop copy 885.090435 main_thread [1418] 45206353 pps (45289533 pkts in 1001840 usec) 886.091600 main_thread [1418] 45322792 pps (45375593 pkts in 1001165 usec) 887.092435 main_thread [1418] 45313992 pps (45351784 pkts in 1000834 usec) 888.094434 main_thread [1418] 45315765 pps (45406397 pkts in 1002000 usec) 889.095434 main_thread [1418] 45333218 pps (45378551 pkts in 1001000 usec) 890.097434 main_thread [1418] 45315247 pps (45405877 pkts in 1002000 usec) 891.099434 main_thread [1418] 45326515 pps (45417168 pkts in 1002000 usec) 892.101434 main_thread [1418] 45333039 pps (45423705 pkts in 1002000 usec) 893.103434 main_thread [1418] 45324105 pps (45414708 pkts in 1001999 usec) 894.105434 main_thread [1418] 45318042 pps (45408723 pkts in 1002001 usec) 895.106434 main_thread [1418] 45332430 pps (45377762 pkts in 1001000 usec) 896.107434 main_thread [1418] 45338072 pps (45383410 pkts in 1001000 usec) ... Relnotes: Yes Sponsored by: Chelsio Communications.
2014-05-27 18:18:41 +00:00
int fl_pktshift = 2;
Convert some fixed parameters to tunables (with reasonable default values). - cong_drop specifies what to do on congestion: nothing, backpressure, or drop. - fl_pktshift specifies the padding before Ethernet payload. - fl_pad specifies the boundary upto which to pad Ethernet payload. - spg_len controls the length of the status page. MFC after: 2 weeks
2012-08-14 21:47:41 +00:00
TUNABLE_INT("hw.cxgbe.fl_pktshift", &fl_pktshift);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
Convert some fixed parameters to tunables (with reasonable default values). - cong_drop specifies what to do on congestion: nothing, backpressure, or drop. - fl_pktshift specifies the padding before Ethernet payload. - fl_pad specifies the boundary upto which to pad Ethernet payload. - spg_len controls the length of the status page. MFC after: 2 weeks
2012-08-14 21:47:41 +00:00
/*
* Pad ethernet payload up to this boundary.
* -1: driver should figure out a good value.
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
* 0: disable padding.
* Any power of 2 from 32 to 4096 (both inclusive) is also a valid value.
Convert some fixed parameters to tunables (with reasonable default values). - cong_drop specifies what to do on congestion: nothing, backpressure, or drop. - fl_pktshift specifies the padding before Ethernet payload. - fl_pad specifies the boundary upto which to pad Ethernet payload. - spg_len controls the length of the status page. MFC after: 2 weeks
2012-08-14 21:47:41 +00:00
*/
cxgbe(4): netmap support for Terminator 5 (T5) based 10G/40G cards. Netmap gets its own hardware-assisted virtual interface and won't take over or disrupt the "normal" interface in any way. You can use both simultaneously. For kernels with DEV_NETMAP, cxgbe(4) carves out an ncxl<N> interface (note the 'n' prefix) in the hardware to accompany each cxl<N> interface. These two ifnet's per port share the same wire but really are separate interfaces in the hardware and software. Each gets its own L2 MAC addresses (unicast and multicast), MTU, checksum caps, etc. You should run netmap on the 'n' interfaces only, that's what they are for. With this, pkt-gen is able to transmit > 45Mpps out of a single 40G port of a T580 card. 2 port tx is at ~56Mpps total (28M + 28M) as of now. Single port receive is at 33Mpps but this is very much a work in progress. I expect it to be closer to 40Mpps once done. In any case the current effort can already saturate multiple 10G ports of a T5 card at the smallest legal packet size. T4 gear is totally untested. trantor:~# ./pkt-gen -i ncxl0 -f tx -D 00:07:43:ab:cd:ef 881.952141 main [1621] interface is ncxl0 881.952250 extract_ip_range [275] range is 10.0.0.1:0 to 10.0.0.1:0 881.952253 extract_ip_range [275] range is 10.1.0.1:0 to 10.1.0.1:0 881.962540 main [1804] mapped 334980KB at 0x801dff000 Sending on netmap:ncxl0: 4 queues, 1 threads and 1 cpus. 10.0.0.1 -> 10.1.0.1 (00:00:00:00:00:00 -> 00:07:43:ab:cd:ef) 881.962562 main [1882] Sending 512 packets every 0.000000000 s 881.962563 main [1884] Wait 2 secs for phy reset 884.088516 main [1886] Ready... 884.088535 nm_open [457] overriding ifname ncxl0 ringid 0x0 flags 0x1 884.088607 sender_body [996] start 884.093246 sender_body [1064] drop copy 885.090435 main_thread [1418] 45206353 pps (45289533 pkts in 1001840 usec) 886.091600 main_thread [1418] 45322792 pps (45375593 pkts in 1001165 usec) 887.092435 main_thread [1418] 45313992 pps (45351784 pkts in 1000834 usec) 888.094434 main_thread [1418] 45315765 pps (45406397 pkts in 1002000 usec) 889.095434 main_thread [1418] 45333218 pps (45378551 pkts in 1001000 usec) 890.097434 main_thread [1418] 45315247 pps (45405877 pkts in 1002000 usec) 891.099434 main_thread [1418] 45326515 pps (45417168 pkts in 1002000 usec) 892.101434 main_thread [1418] 45333039 pps (45423705 pkts in 1002000 usec) 893.103434 main_thread [1418] 45324105 pps (45414708 pkts in 1001999 usec) 894.105434 main_thread [1418] 45318042 pps (45408723 pkts in 1002001 usec) 895.106434 main_thread [1418] 45332430 pps (45377762 pkts in 1001000 usec) 896.107434 main_thread [1418] 45338072 pps (45383410 pkts in 1001000 usec) ... Relnotes: Yes Sponsored by: Chelsio Communications.
2014-05-27 18:18:41 +00:00
int fl_pad = -1;
Convert some fixed parameters to tunables (with reasonable default values). - cong_drop specifies what to do on congestion: nothing, backpressure, or drop. - fl_pktshift specifies the padding before Ethernet payload. - fl_pad specifies the boundary upto which to pad Ethernet payload. - spg_len controls the length of the status page. MFC after: 2 weeks
2012-08-14 21:47:41 +00:00
TUNABLE_INT("hw.cxgbe.fl_pad", &fl_pad);
/*
* Status page length.
* -1: driver should figure out a good value.
* 64 or 128 are the only other valid values.
*/
cxgbe(4): netmap support for Terminator 5 (T5) based 10G/40G cards. Netmap gets its own hardware-assisted virtual interface and won't take over or disrupt the "normal" interface in any way. You can use both simultaneously. For kernels with DEV_NETMAP, cxgbe(4) carves out an ncxl<N> interface (note the 'n' prefix) in the hardware to accompany each cxl<N> interface. These two ifnet's per port share the same wire but really are separate interfaces in the hardware and software. Each gets its own L2 MAC addresses (unicast and multicast), MTU, checksum caps, etc. You should run netmap on the 'n' interfaces only, that's what they are for. With this, pkt-gen is able to transmit > 45Mpps out of a single 40G port of a T580 card. 2 port tx is at ~56Mpps total (28M + 28M) as of now. Single port receive is at 33Mpps but this is very much a work in progress. I expect it to be closer to 40Mpps once done. In any case the current effort can already saturate multiple 10G ports of a T5 card at the smallest legal packet size. T4 gear is totally untested. trantor:~# ./pkt-gen -i ncxl0 -f tx -D 00:07:43:ab:cd:ef 881.952141 main [1621] interface is ncxl0 881.952250 extract_ip_range [275] range is 10.0.0.1:0 to 10.0.0.1:0 881.952253 extract_ip_range [275] range is 10.1.0.1:0 to 10.1.0.1:0 881.962540 main [1804] mapped 334980KB at 0x801dff000 Sending on netmap:ncxl0: 4 queues, 1 threads and 1 cpus. 10.0.0.1 -> 10.1.0.1 (00:00:00:00:00:00 -> 00:07:43:ab:cd:ef) 881.962562 main [1882] Sending 512 packets every 0.000000000 s 881.962563 main [1884] Wait 2 secs for phy reset 884.088516 main [1886] Ready... 884.088535 nm_open [457] overriding ifname ncxl0 ringid 0x0 flags 0x1 884.088607 sender_body [996] start 884.093246 sender_body [1064] drop copy 885.090435 main_thread [1418] 45206353 pps (45289533 pkts in 1001840 usec) 886.091600 main_thread [1418] 45322792 pps (45375593 pkts in 1001165 usec) 887.092435 main_thread [1418] 45313992 pps (45351784 pkts in 1000834 usec) 888.094434 main_thread [1418] 45315765 pps (45406397 pkts in 1002000 usec) 889.095434 main_thread [1418] 45333218 pps (45378551 pkts in 1001000 usec) 890.097434 main_thread [1418] 45315247 pps (45405877 pkts in 1002000 usec) 891.099434 main_thread [1418] 45326515 pps (45417168 pkts in 1002000 usec) 892.101434 main_thread [1418] 45333039 pps (45423705 pkts in 1002000 usec) 893.103434 main_thread [1418] 45324105 pps (45414708 pkts in 1001999 usec) 894.105434 main_thread [1418] 45318042 pps (45408723 pkts in 1002001 usec) 895.106434 main_thread [1418] 45332430 pps (45377762 pkts in 1001000 usec) 896.107434 main_thread [1418] 45338072 pps (45383410 pkts in 1001000 usec) ... Relnotes: Yes Sponsored by: Chelsio Communications.
2014-05-27 18:18:41 +00:00
int spg_len = -1;
Convert some fixed parameters to tunables (with reasonable default values). - cong_drop specifies what to do on congestion: nothing, backpressure, or drop. - fl_pktshift specifies the padding before Ethernet payload. - fl_pad specifies the boundary upto which to pad Ethernet payload. - spg_len controls the length of the status page. MFC after: 2 weeks
2012-08-14 21:47:41 +00:00
TUNABLE_INT("hw.cxgbe.spg_len", &spg_len);
/*
* Congestion drops.
* -1: no congestion feedback (not recommended).
* 0: backpressure the channel instead of dropping packets right away.
* 1: no backpressure, drop packets for the congested queue immediately.
*/
static int cong_drop = 0;
TUNABLE_INT("hw.cxgbe.cong_drop", &cong_drop);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
/*
* Deliver multiple frames in the same free list buffer if they fit.
* -1: let the driver decide whether to enable buffer packing or not.
* 0: disable buffer packing.
* 1: enable buffer packing.
*/
static int buffer_packing = -1;
TUNABLE_INT("hw.cxgbe.buffer_packing", &buffer_packing);
/*
* Start next frame in a packed buffer at this boundary.
* -1: driver should figure out a good value.
* T4:
* ---
* if fl_pad != 0
* value specified here will be overridden by fl_pad.
* else
* power of 2 from 32 to 4096 (both inclusive) is a valid value here.
* T5:
* ---
* 16, or a power of 2 from 64 to 4096 (both inclusive) is a valid value.
*/
static int fl_pack = -1;
static int t4_fl_pack;
static int t5_fl_pack;
TUNABLE_INT("hw.cxgbe.fl_pack", &fl_pack);
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
/*
* Allow the driver to create mbuf(s) in a cluster allocated for rx.
* 0: never; always allocate mbufs from the zone_mbuf UMA zone.
* 1: ok to create mbuf(s) within a cluster if there is room.
*/
static int allow_mbufs_in_cluster = 1;
TUNABLE_INT("hw.cxgbe.allow_mbufs_in_cluster", &allow_mbufs_in_cluster);
/*
* Largest rx cluster size that the driver is allowed to allocate.
*/
static int largest_rx_cluster = MJUM16BYTES;
TUNABLE_INT("hw.cxgbe.largest_rx_cluster", &largest_rx_cluster);
/*
* Size of cluster allocation that's most likely to succeed. The driver will
* fall back to this size if it fails to allocate clusters larger than this.
*/
static int safest_rx_cluster = PAGE_SIZE;
TUNABLE_INT("hw.cxgbe.safest_rx_cluster", &safest_rx_cluster);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
/* Used to track coalesced tx work request */
struct txpkts {
uint64_t *flitp; /* ptr to flit where next pkt should start */
uint8_t npkt; /* # of packets in this work request */
uint8_t nflits; /* # of flits used by this work request */
uint16_t plen; /* total payload (sum of all packets) */
};
/* A packet's SGL. This + m_pkthdr has all info needed for tx */
struct sgl {
int nsegs; /* # of segments in the SGL, 0 means imm. tx */
int nflits; /* # of flits needed for the SGL */
bus_dma_segment_t seg[TX_SGL_SEGS];
};
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
static int service_iq(struct sge_iq *, int);
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
static struct mbuf *get_fl_payload(struct adapter *, struct sge_fl *, uint32_t,
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
int *);
static int t4_eth_rx(struct sge_iq *, const struct rss_header *, struct mbuf *);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
static inline void init_iq(struct sge_iq *, struct adapter *, int, int, int,
Remove unused item. cxgbe's rx queue's lock was removed a long time ago. MFC after: 3 days
2012-10-10 16:52:39 +00:00
int);
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
static inline void init_fl(struct adapter *, struct sge_fl *, int, int, int,
char *);
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
static inline void init_eq(struct sge_eq *, int, int, uint8_t, uint16_t,
char *);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
static int alloc_ring(struct adapter *, size_t, bus_dma_tag_t *, bus_dmamap_t *,
bus_addr_t *, void **);
static int free_ring(struct adapter *, bus_dma_tag_t, bus_dmamap_t, bus_addr_t,
void *);
static int alloc_iq_fl(struct port_info *, struct sge_iq *, struct sge_fl *,
- Enable per-channel congestion notification. - Enable PCIe relaxed ordering for all egress queues and rx data buffers. MFC after: 3 days
2011-05-18 22:09:04 +00:00
int, int);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
static int free_iq_fl(struct port_info *, struct sge_iq *, struct sge_fl *);
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
static void add_fl_sysctls(struct sysctl_ctx_list *, struct sysctl_oid *,
struct sge_fl *);
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
static int alloc_fwq(struct adapter *);
static int free_fwq(struct adapter *);
static int alloc_mgmtq(struct adapter *);
static int free_mgmtq(struct adapter *);
static int alloc_rxq(struct port_info *, struct sge_rxq *, int, int,
struct sysctl_oid *);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
static int free_rxq(struct port_info *, struct sge_rxq *);
- Updated TOE support in the kernel. - Stateful TCP offload drivers for Terminator 3 and 4 (T3 and T4) ASICs. These are available as t3_tom and t4_tom modules that augment cxgb(4) and cxgbe(4) respectively. The cxgb/cxgbe drivers continue to work as usual with or without these extra features. - iWARP driver for Terminator 3 ASIC (kernel verbs). T4 iWARP in the works and will follow soon. Build-tested with make universe. 30s overview ============ What interfaces support TCP offload? Look for TOE4 and/or TOE6 in the capabilities of an interface: # ifconfig -m | grep TOE Enable/disable TCP offload on an interface (just like any other ifnet capability): # ifconfig cxgbe0 toe # ifconfig cxgbe0 -toe Which connections are offloaded? Look for toe4 and/or toe6 in the output of netstat and sockstat: # netstat -np tcp | grep toe # sockstat -46c | grep toe Reviewed by: bz, gnn Sponsored by: Chelsio communications. MFC after: ~3 months (after 9.1, and after ensuring MFC is feasible)
2012-06-19 07:34:13 +00:00
#ifdef TCP_OFFLOAD
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
static int alloc_ofld_rxq(struct port_info *, struct sge_ofld_rxq *, int, int,
struct sysctl_oid *);
static int free_ofld_rxq(struct port_info *, struct sge_ofld_rxq *);
#endif
cxgbe(4): netmap support for Terminator 5 (T5) based 10G/40G cards. Netmap gets its own hardware-assisted virtual interface and won't take over or disrupt the "normal" interface in any way. You can use both simultaneously. For kernels with DEV_NETMAP, cxgbe(4) carves out an ncxl<N> interface (note the 'n' prefix) in the hardware to accompany each cxl<N> interface. These two ifnet's per port share the same wire but really are separate interfaces in the hardware and software. Each gets its own L2 MAC addresses (unicast and multicast), MTU, checksum caps, etc. You should run netmap on the 'n' interfaces only, that's what they are for. With this, pkt-gen is able to transmit > 45Mpps out of a single 40G port of a T580 card. 2 port tx is at ~56Mpps total (28M + 28M) as of now. Single port receive is at 33Mpps but this is very much a work in progress. I expect it to be closer to 40Mpps once done. In any case the current effort can already saturate multiple 10G ports of a T5 card at the smallest legal packet size. T4 gear is totally untested. trantor:~# ./pkt-gen -i ncxl0 -f tx -D 00:07:43:ab:cd:ef 881.952141 main [1621] interface is ncxl0 881.952250 extract_ip_range [275] range is 10.0.0.1:0 to 10.0.0.1:0 881.952253 extract_ip_range [275] range is 10.1.0.1:0 to 10.1.0.1:0 881.962540 main [1804] mapped 334980KB at 0x801dff000 Sending on netmap:ncxl0: 4 queues, 1 threads and 1 cpus. 10.0.0.1 -> 10.1.0.1 (00:00:00:00:00:00 -> 00:07:43:ab:cd:ef) 881.962562 main [1882] Sending 512 packets every 0.000000000 s 881.962563 main [1884] Wait 2 secs for phy reset 884.088516 main [1886] Ready... 884.088535 nm_open [457] overriding ifname ncxl0 ringid 0x0 flags 0x1 884.088607 sender_body [996] start 884.093246 sender_body [1064] drop copy 885.090435 main_thread [1418] 45206353 pps (45289533 pkts in 1001840 usec) 886.091600 main_thread [1418] 45322792 pps (45375593 pkts in 1001165 usec) 887.092435 main_thread [1418] 45313992 pps (45351784 pkts in 1000834 usec) 888.094434 main_thread [1418] 45315765 pps (45406397 pkts in 1002000 usec) 889.095434 main_thread [1418] 45333218 pps (45378551 pkts in 1001000 usec) 890.097434 main_thread [1418] 45315247 pps (45405877 pkts in 1002000 usec) 891.099434 main_thread [1418] 45326515 pps (45417168 pkts in 1002000 usec) 892.101434 main_thread [1418] 45333039 pps (45423705 pkts in 1002000 usec) 893.103434 main_thread [1418] 45324105 pps (45414708 pkts in 1001999 usec) 894.105434 main_thread [1418] 45318042 pps (45408723 pkts in 1002001 usec) 895.106434 main_thread [1418] 45332430 pps (45377762 pkts in 1001000 usec) 896.107434 main_thread [1418] 45338072 pps (45383410 pkts in 1001000 usec) ... Relnotes: Yes Sponsored by: Chelsio Communications.
2014-05-27 18:18:41 +00:00
#ifdef DEV_NETMAP
static int alloc_nm_rxq(struct port_info *, struct sge_nm_rxq *, int, int,
struct sysctl_oid *);
static int free_nm_rxq(struct port_info *, struct sge_nm_rxq *);
static int alloc_nm_txq(struct port_info *, struct sge_nm_txq *, int, int,
struct sysctl_oid *);
static int free_nm_txq(struct port_info *, struct sge_nm_txq *);
#endif
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
static int ctrl_eq_alloc(struct adapter *, struct sge_eq *);
static int eth_eq_alloc(struct adapter *, struct port_info *, struct sge_eq *);
- Updated TOE support in the kernel. - Stateful TCP offload drivers for Terminator 3 and 4 (T3 and T4) ASICs. These are available as t3_tom and t4_tom modules that augment cxgb(4) and cxgbe(4) respectively. The cxgb/cxgbe drivers continue to work as usual with or without these extra features. - iWARP driver for Terminator 3 ASIC (kernel verbs). T4 iWARP in the works and will follow soon. Build-tested with make universe. 30s overview ============ What interfaces support TCP offload? Look for TOE4 and/or TOE6 in the capabilities of an interface: # ifconfig -m | grep TOE Enable/disable TCP offload on an interface (just like any other ifnet capability): # ifconfig cxgbe0 toe # ifconfig cxgbe0 -toe Which connections are offloaded? Look for toe4 and/or toe6 in the output of netstat and sockstat: # netstat -np tcp | grep toe # sockstat -46c | grep toe Reviewed by: bz, gnn Sponsored by: Chelsio communications. MFC after: ~3 months (after 9.1, and after ensuring MFC is feasible)
2012-06-19 07:34:13 +00:00
#ifdef TCP_OFFLOAD
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
static int ofld_eq_alloc(struct adapter *, struct port_info *, struct sge_eq *);
#endif
static int alloc_eq(struct adapter *, struct port_info *, struct sge_eq *);
static int free_eq(struct adapter *, struct sge_eq *);
static int alloc_wrq(struct adapter *, struct port_info *, struct sge_wrq *,
struct sysctl_oid *);
static int free_wrq(struct adapter *, struct sge_wrq *);
static int alloc_txq(struct port_info *, struct sge_txq *, int,
struct sysctl_oid *);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
static int free_txq(struct port_info *, struct sge_txq *);
static void oneseg_dma_callback(void *, bus_dma_segment_t *, int, int);
static inline bool is_new_response(const struct sge_iq *, struct rsp_ctrl **);
static inline void iq_next(struct sge_iq *);
static inline void ring_fl_db(struct adapter *, struct sge_fl *);
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
static int refill_fl(struct adapter *, struct sge_fl *, int);
static void refill_sfl(void *);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
static int alloc_fl_sdesc(struct sge_fl *);
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
static void free_fl_sdesc(struct adapter *, struct sge_fl *);
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
static void find_best_refill_source(struct adapter *, struct sge_fl *, int);
static void find_safe_refill_source(struct adapter *, struct sge_fl *);
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
static void add_fl_to_sfl(struct adapter *, struct sge_fl *);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
static int get_pkt_sgl(struct sge_txq *, struct mbuf **, struct sgl *, int);
static int free_pkt_sgl(struct sge_txq *, struct sgl *);
static int write_txpkt_wr(struct port_info *, struct sge_txq *, struct mbuf *,
struct sgl *);
static int add_to_txpkts(struct port_info *, struct sge_txq *, struct txpkts *,
struct mbuf *, struct sgl *);
static void write_txpkts_wr(struct sge_txq *, struct txpkts *);
static inline void write_ulp_cpl_sgl(struct port_info *, struct sge_txq *,
struct txpkts *, struct mbuf *, struct sgl *);
static int write_sgl_to_txd(struct sge_eq *, struct sgl *, caddr_t *);
static inline void copy_to_txd(struct sge_eq *, caddr_t, caddr_t *, int);
- Move all Ethernet specific items from sge_eq to sge_txq. sge_eq is now a suitable base for all kinds of egress queues. - Add control queues (sge_ctrlq) and allocate one of these per hardware channel. They can be used to program filters and steer traffic (and more). MFC after: 1 week
2011-04-19 22:08:28 +00:00
static inline void ring_eq_db(struct adapter *, struct sge_eq *);
Calculate how many descriptors can be reclaimed before calling reclaim_tx_descs
2011-03-05 03:54:37 +00:00
static inline int reclaimable(struct sge_eq *);
- Move all Ethernet specific items from sge_eq to sge_txq. sge_eq is now a suitable base for all kinds of egress queues. - Add control queues (sge_ctrlq) and allocate one of these per hardware channel. They can be used to program filters and steer traffic (and more). MFC after: 1 week
2011-04-19 22:08:28 +00:00
static int reclaim_tx_descs(struct sge_txq *, int, int);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
static void write_eqflush_wr(struct sge_eq *);
static __be64 get_flit(bus_dma_segment_t *, int, int);
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
static int handle_sge_egr_update(struct sge_iq *, const struct rss_header *,
struct mbuf *);
Allow for a different handler for each type of firmware message. MFC after: 2 weeks
2012-08-16 18:31:50 +00:00
static int handle_fw_msg(struct sge_iq *, const struct rss_header *,
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
struct mbuf *);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
- Specialized ingress queues that take interrupts for other ingress queues. Try to have a set of these per port when possible, fall back to sharing a common pool between all ports otherwise. - One control queue per port (used to be one per hardware channel). - t4_eth_rx now handles Ethernet rx only. - sysctls to display pidx/cidx for some queues. MFC after: 1 week
2011-05-30 21:34:44 +00:00
static int sysctl_uint16(SYSCTL_HANDLER_ARGS);
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
static int sysctl_bufsizes(SYSCTL_HANDLER_ARGS);
- Move all Ethernet specific items from sge_eq to sge_txq. sge_eq is now a suitable base for all kinds of egress queues. - Add control queues (sge_ctrlq) and allocate one of these per hardware channel. They can be used to program filters and steer traffic (and more). MFC after: 1 week
2011-04-19 22:08:28 +00:00
cxgbe shouldn't directly know of the UMA zones where network buffers come from. MFC after: 1 week
2011-03-08 03:04:07 +00:00
/*
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
* Called on MOD_LOAD. Validates and calculates the SGE tunables.
cxgbe shouldn't directly know of the UMA zones where network buffers come from. MFC after: 1 week
2011-03-08 03:04:07 +00:00
*/
void
t4_sge_modload(void)
{
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
int pad;
cxgbe shouldn't directly know of the UMA zones where network buffers come from. MFC after: 1 week
2011-03-08 03:04:07 +00:00
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
/* set pad to a reasonable powerof2 between 16 and 4096 (inclusive) */
#if defined(__i386__) || defined(__amd64__)
pad = max(cpu_clflush_line_size, 16);
#else
pad = max(CACHE_LINE_SIZE, 16);
#endif
pad = min(pad, 4096);
Better way to determine the status page length and rx pad boundary.
2012-06-23 22:12:27 +00:00
Convert some fixed parameters to tunables (with reasonable default values). - cong_drop specifies what to do on congestion: nothing, backpressure, or drop. - fl_pktshift specifies the padding before Ethernet payload. - fl_pad specifies the boundary upto which to pad Ethernet payload. - spg_len controls the length of the status page. MFC after: 2 weeks
2012-08-14 21:47:41 +00:00
if (fl_pktshift < 0 || fl_pktshift > 7) {
printf("Invalid hw.cxgbe.fl_pktshift value (%d),"
" using 2 instead.\n", fl_pktshift);
fl_pktshift = 2;
}
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
if (fl_pad != 0 &&
(fl_pad < 32 || fl_pad > 4096 || !powerof2(fl_pad))) {
Convert some fixed parameters to tunables (with reasonable default values). - cong_drop specifies what to do on congestion: nothing, backpressure, or drop. - fl_pktshift specifies the padding before Ethernet payload. - fl_pad specifies the boundary upto which to pad Ethernet payload. - spg_len controls the length of the status page. MFC after: 2 weeks
2012-08-14 21:47:41 +00:00
if (fl_pad != -1) {
printf("Invalid hw.cxgbe.fl_pad value (%d),"
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
" using %d instead.\n", fl_pad, max(pad, 32));
Convert some fixed parameters to tunables (with reasonable default values). - cong_drop specifies what to do on congestion: nothing, backpressure, or drop. - fl_pktshift specifies the padding before Ethernet payload. - fl_pad specifies the boundary upto which to pad Ethernet payload. - spg_len controls the length of the status page. MFC after: 2 weeks
2012-08-14 21:47:41 +00:00
}
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
fl_pad = max(pad, 32);
Convert some fixed parameters to tunables (with reasonable default values). - cong_drop specifies what to do on congestion: nothing, backpressure, or drop. - fl_pktshift specifies the padding before Ethernet payload. - fl_pad specifies the boundary upto which to pad Ethernet payload. - spg_len controls the length of the status page. MFC after: 2 weeks
2012-08-14 21:47:41 +00:00
}
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
/*
* T4 has the same pad and pack boundary. If a pad boundary is set,
* pack boundary must be set to the same value. Otherwise take the
* specified value or auto-calculate something reasonable.
*/
if (fl_pad)
t4_fl_pack = fl_pad;
else if (fl_pack < 32 || fl_pack > 4096 || !powerof2(fl_pack))
t4_fl_pack = max(pad, 32);
else
t4_fl_pack = fl_pack;
/* T5's pack boundary is independent of the pad boundary. */
if (fl_pack < 16 || fl_pack == 32 || fl_pack > 4096 ||
!powerof2(fl_pack))
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
t5_fl_pack = max(pad, CACHE_LINE_SIZE);
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
else
t5_fl_pack = fl_pack;
Convert some fixed parameters to tunables (with reasonable default values). - cong_drop specifies what to do on congestion: nothing, backpressure, or drop. - fl_pktshift specifies the padding before Ethernet payload. - fl_pad specifies the boundary upto which to pad Ethernet payload. - spg_len controls the length of the status page. MFC after: 2 weeks
2012-08-14 21:47:41 +00:00
if (spg_len != 64 && spg_len != 128) {
int len;
#if defined(__i386__) || defined(__amd64__)
len = cpu_clflush_line_size > 64 ? 128 : 64;
#else
len = 64;
#endif
if (spg_len != -1) {
printf("Invalid hw.cxgbe.spg_len value (%d),"
" using %d instead.\n", spg_len, len);
}
spg_len = len;
}
if (cong_drop < -1 || cong_drop > 1) {
printf("Invalid hw.cxgbe.cong_drop value (%d),"
" using 0 instead.\n", cong_drop);
cong_drop = 0;
}
cxgbe shouldn't directly know of the UMA zones where network buffers come from. MFC after: 1 week
2011-03-08 03:04:07 +00:00
}
cxgbe(4): Add support for Chelsio's Terminator 5 (aka T5) ASIC. This includes support for the NIC and TOE features of the 40G, 10G, and 1G/100M cards based on the T5. The ASIC is mostly backward compatible with the Terminator 4 so cxgbe(4) has been updated instead of writing a brand new driver. T5 cards will show up as cxl (short for cxlgb) ports attached to the t5nex bus driver. Sponsored by: Chelsio
2013-03-30 02:26:20 +00:00
void
t4_init_sge_cpl_handlers(struct adapter *sc)
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
{
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
cxgbe(4): Add support for Chelsio's Terminator 5 (aka T5) ASIC. This includes support for the NIC and TOE features of the 40G, 10G, and 1G/100M cards based on the T5. The ASIC is mostly backward compatible with the Terminator 4 so cxgbe(4) has been updated instead of writing a brand new driver. T5 cards will show up as cxl (short for cxlgb) ports attached to the t5nex bus driver. Sponsored by: Chelsio
2013-03-30 02:26:20 +00:00
t4_register_cpl_handler(sc, CPL_FW4_MSG, handle_fw_msg);
t4_register_cpl_handler(sc, CPL_FW6_MSG, handle_fw_msg);
t4_register_cpl_handler(sc, CPL_SGE_EGR_UPDATE, handle_sge_egr_update);
t4_register_cpl_handler(sc, CPL_RX_PKT, t4_eth_rx);
t4_register_fw_msg_handler(sc, FW6_TYPE_CMD_RPL, t4_handle_fw_rpl);
}
Auto-reduce the holdoff timers that are greater than the maximum value allowed by the hardware. MFC after: 3 days
2013-04-11 22:46:39 +00:00
/*
* adap->params.vpd.cclk must be set up before this is called.
*/
cxgbe(4): Add support for Chelsio's Terminator 5 (aka T5) ASIC. This includes support for the NIC and TOE features of the 40G, 10G, and 1G/100M cards based on the T5. The ASIC is mostly backward compatible with the Terminator 4 so cxgbe(4) has been updated instead of writing a brand new driver. T5 cards will show up as cxl (short for cxlgb) ports attached to the t5nex bus driver. Sponsored by: Chelsio
2013-03-30 02:26:20 +00:00
void
t4_tweak_chip_settings(struct adapter *sc)
{
int i;
uint32_t v, m;
int intr_timer[SGE_NTIMERS] = {1, 5, 10, 50, 100, 200};
Auto-reduce the holdoff timers that are greater than the maximum value allowed by the hardware. MFC after: 3 days
2013-04-11 22:46:39 +00:00
int timer_max = M_TIMERVALUE0 * 1000 / sc->params.vpd.cclk;
cxgbe(4): Add support for Chelsio's Terminator 5 (aka T5) ASIC. This includes support for the NIC and TOE features of the 40G, 10G, and 1G/100M cards based on the T5. The ASIC is mostly backward compatible with the Terminator 4 so cxgbe(4) has been updated instead of writing a brand new driver. T5 cards will show up as cxl (short for cxlgb) ports attached to the t5nex bus driver. Sponsored by: Chelsio
2013-03-30 02:26:20 +00:00
int intr_pktcount[SGE_NCOUNTERS] = {1, 8, 16, 32}; /* 63 max */
uint16_t indsz = min(RX_COPY_THRESHOLD - 1, M_INDICATESIZE);
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
static int sge_flbuf_sizes[] = {
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
MCLBYTES,
#if MJUMPAGESIZE != MCLBYTES
MJUMPAGESIZE,
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
MJUMPAGESIZE - CL_METADATA_SIZE,
MJUMPAGESIZE - 2 * MSIZE - CL_METADATA_SIZE,
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
#endif
MJUM9BYTES,
MJUM16BYTES,
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
MCLBYTES - MSIZE - CL_METADATA_SIZE,
MJUM9BYTES - CL_METADATA_SIZE,
MJUM16BYTES - CL_METADATA_SIZE,
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
};
cxgbe(4): Add support for Chelsio's Terminator 5 (aka T5) ASIC. This includes support for the NIC and TOE features of the 40G, 10G, and 1G/100M cards based on the T5. The ASIC is mostly backward compatible with the Terminator 4 so cxgbe(4) has been updated instead of writing a brand new driver. T5 cards will show up as cxl (short for cxlgb) ports attached to the t5nex bus driver. Sponsored by: Chelsio
2013-03-30 02:26:20 +00:00
KASSERT(sc->flags & MASTER_PF,
("%s: trying to change chip settings when not master.", __func__));
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
m = V_PKTSHIFT(M_PKTSHIFT) | F_RXPKTCPLMODE | F_EGRSTATUSPAGESIZE;
cxgbe(4): Add support for Chelsio's Terminator 5 (aka T5) ASIC. This includes support for the NIC and TOE features of the 40G, 10G, and 1G/100M cards based on the T5. The ASIC is mostly backward compatible with the Terminator 4 so cxgbe(4) has been updated instead of writing a brand new driver. T5 cards will show up as cxl (short for cxlgb) ports attached to the t5nex bus driver. Sponsored by: Chelsio
2013-03-30 02:26:20 +00:00
v = V_PKTSHIFT(fl_pktshift) | F_RXPKTCPLMODE |
Better way to determine the status page length and rx pad boundary.
2012-06-23 22:12:27 +00:00
V_EGRSTATUSPAGESIZE(spg_len == 128);
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
if (is_t4(sc) && (fl_pad || buffer_packing)) {
/* t4_fl_pack has the correct value even when fl_pad = 0 */
m |= V_INGPADBOUNDARY(M_INGPADBOUNDARY);
v |= V_INGPADBOUNDARY(ilog2(t4_fl_pack) - 5);
} else if (is_t5(sc) && fl_pad) {
m |= V_INGPADBOUNDARY(M_INGPADBOUNDARY);
v |= V_INGPADBOUNDARY(ilog2(fl_pad) - 5);
}
cxgbe(4): Add support for Chelsio's Terminator 5 (aka T5) ASIC. This includes support for the NIC and TOE features of the 40G, 10G, and 1G/100M cards based on the T5. The ASIC is mostly backward compatible with the Terminator 4 so cxgbe(4) has been updated instead of writing a brand new driver. T5 cards will show up as cxl (short for cxlgb) ports attached to the t5nex bus driver. Sponsored by: Chelsio
2013-03-30 02:26:20 +00:00
t4_set_reg_field(sc, A_SGE_CONTROL, m, v);
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
if (is_t5(sc) && buffer_packing) {
m = V_INGPACKBOUNDARY(M_INGPACKBOUNDARY);
if (t5_fl_pack == 16)
v = V_INGPACKBOUNDARY(0);
else
v = V_INGPACKBOUNDARY(ilog2(t5_fl_pack) - 5);
t4_set_reg_field(sc, A_SGE_CONTROL2, m, v);
}
cxgbe(4): Add support for Chelsio's Terminator 5 (aka T5) ASIC. This includes support for the NIC and TOE features of the 40G, 10G, and 1G/100M cards based on the T5. The ASIC is mostly backward compatible with the Terminator 4 so cxgbe(4) has been updated instead of writing a brand new driver. T5 cards will show up as cxl (short for cxlgb) ports attached to the t5nex bus driver. Sponsored by: Chelsio
2013-03-30 02:26:20 +00:00
v = V_HOSTPAGESIZEPF0(PAGE_SHIFT - 10) |
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
V_HOSTPAGESIZEPF1(PAGE_SHIFT - 10) |
V_HOSTPAGESIZEPF2(PAGE_SHIFT - 10) |
V_HOSTPAGESIZEPF3(PAGE_SHIFT - 10) |
V_HOSTPAGESIZEPF4(PAGE_SHIFT - 10) |
V_HOSTPAGESIZEPF5(PAGE_SHIFT - 10) |
V_HOSTPAGESIZEPF6(PAGE_SHIFT - 10) |
V_HOSTPAGESIZEPF7(PAGE_SHIFT - 10);
cxgbe(4): Add support for Chelsio's Terminator 5 (aka T5) ASIC. This includes support for the NIC and TOE features of the 40G, 10G, and 1G/100M cards based on the T5. The ASIC is mostly backward compatible with the Terminator 4 so cxgbe(4) has been updated instead of writing a brand new driver. T5 cards will show up as cxl (short for cxlgb) ports attached to the t5nex bus driver. Sponsored by: Chelsio
2013-03-30 02:26:20 +00:00
t4_write_reg(sc, A_SGE_HOST_PAGE_SIZE, v);
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
KASSERT(nitems(sge_flbuf_sizes) <= SGE_FLBUF_SIZES,
("%s: hw buffer size table too big", __func__));
for (i = 0; i < min(nitems(sge_flbuf_sizes), SGE_FLBUF_SIZES); i++) {
cxgbe(4): Add support for Chelsio's Terminator 5 (aka T5) ASIC. This includes support for the NIC and TOE features of the 40G, 10G, and 1G/100M cards based on the T5. The ASIC is mostly backward compatible with the Terminator 4 so cxgbe(4) has been updated instead of writing a brand new driver. T5 cards will show up as cxl (short for cxlgb) ports attached to the t5nex bus driver. Sponsored by: Chelsio
2013-03-30 02:26:20 +00:00
t4_write_reg(sc, A_SGE_FL_BUFFER_SIZE0 + (4 * i),
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
sge_flbuf_sizes[i]);
cxgbe(4): Add support for Chelsio's Terminator 5 (aka T5) ASIC. This includes support for the NIC and TOE features of the 40G, 10G, and 1G/100M cards based on the T5. The ASIC is mostly backward compatible with the Terminator 4 so cxgbe(4) has been updated instead of writing a brand new driver. T5 cards will show up as cxl (short for cxlgb) ports attached to the t5nex bus driver. Sponsored by: Chelsio
2013-03-30 02:26:20 +00:00
}
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
cxgbe(4): Add support for Chelsio's Terminator 5 (aka T5) ASIC. This includes support for the NIC and TOE features of the 40G, 10G, and 1G/100M cards based on the T5. The ASIC is mostly backward compatible with the Terminator 4 so cxgbe(4) has been updated instead of writing a brand new driver. T5 cards will show up as cxl (short for cxlgb) ports attached to the t5nex bus driver. Sponsored by: Chelsio
2013-03-30 02:26:20 +00:00
v = V_THRESHOLD_0(intr_pktcount[0]) | V_THRESHOLD_1(intr_pktcount[1]) |
V_THRESHOLD_2(intr_pktcount[2]) | V_THRESHOLD_3(intr_pktcount[3]);
t4_write_reg(sc, A_SGE_INGRESS_RX_THRESHOLD, v);
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
Auto-reduce the holdoff timers that are greater than the maximum value allowed by the hardware. MFC after: 3 days
2013-04-11 22:46:39 +00:00
KASSERT(intr_timer[0] <= timer_max,
("%s: not a single usable timer (%d, %d)", __func__, intr_timer[0],
timer_max));
for (i = 1; i < nitems(intr_timer); i++) {
KASSERT(intr_timer[i] >= intr_timer[i - 1],
("%s: timers not listed in increasing order (%d)",
__func__, i));
while (intr_timer[i] > timer_max) {
if (i == nitems(intr_timer) - 1) {
intr_timer[i] = timer_max;
break;
}
intr_timer[i] += intr_timer[i - 1];
intr_timer[i] /= 2;
}
}
cxgbe(4): Add support for Chelsio's Terminator 5 (aka T5) ASIC. This includes support for the NIC and TOE features of the 40G, 10G, and 1G/100M cards based on the T5. The ASIC is mostly backward compatible with the Terminator 4 so cxgbe(4) has been updated instead of writing a brand new driver. T5 cards will show up as cxl (short for cxlgb) ports attached to the t5nex bus driver. Sponsored by: Chelsio
2013-03-30 02:26:20 +00:00
v = V_TIMERVALUE0(us_to_core_ticks(sc, intr_timer[0])) |
V_TIMERVALUE1(us_to_core_ticks(sc, intr_timer[1]));
t4_write_reg(sc, A_SGE_TIMER_VALUE_0_AND_1, v);
v = V_TIMERVALUE2(us_to_core_ticks(sc, intr_timer[2])) |
V_TIMERVALUE3(us_to_core_ticks(sc, intr_timer[3]));
t4_write_reg(sc, A_SGE_TIMER_VALUE_2_AND_3, v);
v = V_TIMERVALUE4(us_to_core_ticks(sc, intr_timer[4])) |
V_TIMERVALUE5(us_to_core_ticks(sc, intr_timer[5]));
t4_write_reg(sc, A_SGE_TIMER_VALUE_4_AND_5, v);
if (cong_drop == 0) {
m = F_TUNNELCNGDROP0 | F_TUNNELCNGDROP1 | F_TUNNELCNGDROP2 |
F_TUNNELCNGDROP3;
t4_set_reg_field(sc, A_TP_PARA_REG3, m, 0);
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
}
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
cxgbe(4): Add support for Chelsio's Terminator 5 (aka T5) ASIC. This includes support for the NIC and TOE features of the 40G, 10G, and 1G/100M cards based on the T5. The ASIC is mostly backward compatible with the Terminator 4 so cxgbe(4) has been updated instead of writing a brand new driver. T5 cards will show up as cxl (short for cxlgb) ports attached to the t5nex bus driver. Sponsored by: Chelsio
2013-03-30 02:26:20 +00:00
/* 4K, 16K, 64K, 256K DDP "page sizes" */
v = V_HPZ0(0) | V_HPZ1(2) | V_HPZ2(4) | V_HPZ3(6);
t4_write_reg(sc, A_ULP_RX_TDDP_PSZ, v);
m = v = F_TDDPTAGTCB;
t4_set_reg_field(sc, A_ULP_RX_CTL, m, v);
m = V_INDICATESIZE(M_INDICATESIZE) | F_REARMDDPOFFSET |
F_RESETDDPOFFSET;
v = V_INDICATESIZE(indsz) | F_REARMDDPOFFSET | F_RESETDDPOFFSET;
t4_set_reg_field(sc, A_TP_PARA_REG5, m, v);
}
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
/*
* SGE wants the buffer to be at least 64B and then a multiple of the pad
* boundary or 16, whichever is greater.
*/
static inline int
hwsz_ok(int hwsz)
{
int mask = max(fl_pad, 16) - 1;
return (hwsz >= 64 && (hwsz & mask) == 0);
}
cxgbe(4): Add support for Chelsio's Terminator 5 (aka T5) ASIC. This includes support for the NIC and TOE features of the 40G, 10G, and 1G/100M cards based on the T5. The ASIC is mostly backward compatible with the Terminator 4 so cxgbe(4) has been updated instead of writing a brand new driver. T5 cards will show up as cxl (short for cxlgb) ports attached to the t5nex bus driver. Sponsored by: Chelsio
2013-03-30 02:26:20 +00:00
/*
* XXX: driver really should be able to deal with unexpected settings.
*/
int
t4_read_chip_settings(struct adapter *sc)
{
struct sge *s = &sc->sge;
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
int i, j, n, rc = 0;
cxgbe(4): Add support for Chelsio's Terminator 5 (aka T5) ASIC. This includes support for the NIC and TOE features of the 40G, 10G, and 1G/100M cards based on the T5. The ASIC is mostly backward compatible with the Terminator 4 so cxgbe(4) has been updated instead of writing a brand new driver. T5 cards will show up as cxl (short for cxlgb) ports attached to the t5nex bus driver. Sponsored by: Chelsio
2013-03-30 02:26:20 +00:00
uint32_t m, v, r;
uint16_t indsz = min(RX_COPY_THRESHOLD - 1, M_INDICATESIZE);
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
static int sw_buf_sizes[] = { /* Sorted by size */
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
MCLBYTES,
#if MJUMPAGESIZE != MCLBYTES
MJUMPAGESIZE,
#endif
MJUM9BYTES,
MJUM16BYTES
};
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
struct sw_zone_info *swz, *safe_swz;
struct hw_buf_info *hwb;
cxgbe(4): Add support for Chelsio's Terminator 5 (aka T5) ASIC. This includes support for the NIC and TOE features of the 40G, 10G, and 1G/100M cards based on the T5. The ASIC is mostly backward compatible with the Terminator 4 so cxgbe(4) has been updated instead of writing a brand new driver. T5 cards will show up as cxl (short for cxlgb) ports attached to the t5nex bus driver. Sponsored by: Chelsio
2013-03-30 02:26:20 +00:00
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
m = V_PKTSHIFT(M_PKTSHIFT) | F_RXPKTCPLMODE | F_EGRSTATUSPAGESIZE;
cxgbe(4): Add support for Chelsio's Terminator 5 (aka T5) ASIC. This includes support for the NIC and TOE features of the 40G, 10G, and 1G/100M cards based on the T5. The ASIC is mostly backward compatible with the Terminator 4 so cxgbe(4) has been updated instead of writing a brand new driver. T5 cards will show up as cxl (short for cxlgb) ports attached to the t5nex bus driver. Sponsored by: Chelsio
2013-03-30 02:26:20 +00:00
v = V_PKTSHIFT(fl_pktshift) | F_RXPKTCPLMODE |
V_EGRSTATUSPAGESIZE(spg_len == 128);
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
if (is_t4(sc) && (fl_pad || buffer_packing)) {
m |= V_INGPADBOUNDARY(M_INGPADBOUNDARY);
v |= V_INGPADBOUNDARY(ilog2(t4_fl_pack) - 5);
} else if (is_t5(sc) && fl_pad) {
m |= V_INGPADBOUNDARY(M_INGPADBOUNDARY);
v |= V_INGPADBOUNDARY(ilog2(fl_pad) - 5);
}
cxgbe(4): Add support for Chelsio's Terminator 5 (aka T5) ASIC. This includes support for the NIC and TOE features of the 40G, 10G, and 1G/100M cards based on the T5. The ASIC is mostly backward compatible with the Terminator 4 so cxgbe(4) has been updated instead of writing a brand new driver. T5 cards will show up as cxl (short for cxlgb) ports attached to the t5nex bus driver. Sponsored by: Chelsio
2013-03-30 02:26:20 +00:00
r = t4_read_reg(sc, A_SGE_CONTROL);
if ((r & m) != v) {
device_printf(sc->dev, "invalid SGE_CONTROL(0x%x)\n", r);
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
rc = EINVAL;
}
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
if (is_t5(sc) && buffer_packing) {
m = V_INGPACKBOUNDARY(M_INGPACKBOUNDARY);
if (t5_fl_pack == 16)
v = V_INGPACKBOUNDARY(0);
else
v = V_INGPACKBOUNDARY(ilog2(t5_fl_pack) - 5);
r = t4_read_reg(sc, A_SGE_CONTROL2);
if ((r & m) != v) {
device_printf(sc->dev,
"invalid SGE_CONTROL2(0x%x)\n", r);
rc = EINVAL;
}
}
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
s->pack_boundary = is_t4(sc) ? t4_fl_pack : t5_fl_pack;
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
cxgbe(4): Add support for Chelsio's Terminator 5 (aka T5) ASIC. This includes support for the NIC and TOE features of the 40G, 10G, and 1G/100M cards based on the T5. The ASIC is mostly backward compatible with the Terminator 4 so cxgbe(4) has been updated instead of writing a brand new driver. T5 cards will show up as cxl (short for cxlgb) ports attached to the t5nex bus driver. Sponsored by: Chelsio
2013-03-30 02:26:20 +00:00
v = V_HOSTPAGESIZEPF0(PAGE_SHIFT - 10) |
V_HOSTPAGESIZEPF1(PAGE_SHIFT - 10) |
V_HOSTPAGESIZEPF2(PAGE_SHIFT - 10) |
V_HOSTPAGESIZEPF3(PAGE_SHIFT - 10) |
V_HOSTPAGESIZEPF4(PAGE_SHIFT - 10) |
V_HOSTPAGESIZEPF5(PAGE_SHIFT - 10) |
V_HOSTPAGESIZEPF6(PAGE_SHIFT - 10) |
V_HOSTPAGESIZEPF7(PAGE_SHIFT - 10);
r = t4_read_reg(sc, A_SGE_HOST_PAGE_SIZE);
if (r != v) {
device_printf(sc->dev, "invalid SGE_HOST_PAGE_SIZE(0x%x)\n", r);
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
rc = EINVAL;
}
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
/* Filter out unusable hw buffer sizes entirely (mark with -2). */
hwb = &s->hw_buf_info[0];
for (i = 0; i < nitems(s->hw_buf_info); i++, hwb++) {
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
r = t4_read_reg(sc, A_SGE_FL_BUFFER_SIZE0 + (4 * i));
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
hwb->size = r;
hwb->zidx = hwsz_ok(r) ? -1 : -2;
hwb->next = -1;
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
}
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
/*
* Create a sorted list in decreasing order of hw buffer sizes (and so
* increasing order of spare area) for each software zone.
*/
n = 0; /* no usable buffer size to begin with */
swz = &s->sw_zone_info[0];
safe_swz = NULL;
for (i = 0; i < SW_ZONE_SIZES; i++, swz++) {
int8_t head = -1, tail = -1;
swz->size = sw_buf_sizes[i];
swz->zone = m_getzone(swz->size);
swz->type = m_gettype(swz->size);
if (swz->size == safest_rx_cluster)
safe_swz = swz;
hwb = &s->hw_buf_info[0];
for (j = 0; j < SGE_FLBUF_SIZES; j++, hwb++) {
if (hwb->zidx != -1 || hwb->size > swz->size)
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
continue;
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
hwb->zidx = i;
if (head == -1)
head = tail = j;
else if (hwb->size < s->hw_buf_info[tail].size) {
s->hw_buf_info[tail].next = j;
tail = j;
} else {
int8_t *cur;
struct hw_buf_info *t;
for (cur = &head; *cur != -1; cur = &t->next) {
t = &s->hw_buf_info[*cur];
if (hwb->size == t->size) {
hwb->zidx = -2;
break;
}
if (hwb->size > t->size) {
hwb->next = *cur;
*cur = j;
break;
}
}
}
}
swz->head_hwidx = head;
swz->tail_hwidx = tail;
if (tail != -1) {
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
n++;
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
if (swz->size - s->hw_buf_info[tail].size >=
CL_METADATA_SIZE)
sc->flags |= BUF_PACKING_OK;
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
}
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
}
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
if (n == 0) {
device_printf(sc->dev, "no usable SGE FL buffer size.\n");
rc = EINVAL;
}
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
s->safe_hwidx1 = -1;
s->safe_hwidx2 = -1;
if (safe_swz != NULL) {
s->safe_hwidx1 = safe_swz->head_hwidx;
for (i = safe_swz->head_hwidx; i != -1; i = hwb->next) {
int spare;
hwb = &s->hw_buf_info[i];
spare = safe_swz->size - hwb->size;
if (spare < CL_METADATA_SIZE)
continue;
if (s->safe_hwidx2 == -1 ||
spare == CL_METADATA_SIZE + MSIZE)
s->safe_hwidx2 = i;
if (spare >= CL_METADATA_SIZE + MSIZE)
break;
}
}
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
cxgbe(4): Add support for Chelsio's Terminator 5 (aka T5) ASIC. This includes support for the NIC and TOE features of the 40G, 10G, and 1G/100M cards based on the T5. The ASIC is mostly backward compatible with the Terminator 4 so cxgbe(4) has been updated instead of writing a brand new driver. T5 cards will show up as cxl (short for cxlgb) ports attached to the t5nex bus driver. Sponsored by: Chelsio
2013-03-30 02:26:20 +00:00
r = t4_read_reg(sc, A_SGE_INGRESS_RX_THRESHOLD);
s->counter_val[0] = G_THRESHOLD_0(r);
s->counter_val[1] = G_THRESHOLD_1(r);
s->counter_val[2] = G_THRESHOLD_2(r);
s->counter_val[3] = G_THRESHOLD_3(r);
r = t4_read_reg(sc, A_SGE_TIMER_VALUE_0_AND_1);
s->timer_val[0] = G_TIMERVALUE0(r) / core_ticks_per_usec(sc);
s->timer_val[1] = G_TIMERVALUE1(r) / core_ticks_per_usec(sc);
r = t4_read_reg(sc, A_SGE_TIMER_VALUE_2_AND_3);
s->timer_val[2] = G_TIMERVALUE2(r) / core_ticks_per_usec(sc);
s->timer_val[3] = G_TIMERVALUE3(r) / core_ticks_per_usec(sc);
r = t4_read_reg(sc, A_SGE_TIMER_VALUE_4_AND_5);
s->timer_val[4] = G_TIMERVALUE4(r) / core_ticks_per_usec(sc);
s->timer_val[5] = G_TIMERVALUE5(r) / core_ticks_per_usec(sc);
if (cong_drop == 0) {
m = F_TUNNELCNGDROP0 | F_TUNNELCNGDROP1 | F_TUNNELCNGDROP2 |
F_TUNNELCNGDROP3;
r = t4_read_reg(sc, A_TP_PARA_REG3);
if (r & m) {
device_printf(sc->dev,
"invalid TP_PARA_REG3(0x%x)\n", r);
rc = EINVAL;
}
}
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
cxgbe(4): Add support for Chelsio's Terminator 5 (aka T5) ASIC. This includes support for the NIC and TOE features of the 40G, 10G, and 1G/100M cards based on the T5. The ASIC is mostly backward compatible with the Terminator 4 so cxgbe(4) has been updated instead of writing a brand new driver. T5 cards will show up as cxl (short for cxlgb) ports attached to the t5nex bus driver. Sponsored by: Chelsio
2013-03-30 02:26:20 +00:00
v = V_HPZ0(0) | V_HPZ1(2) | V_HPZ2(4) | V_HPZ3(6);
r = t4_read_reg(sc, A_ULP_RX_TDDP_PSZ);
if (r != v) {
device_printf(sc->dev, "invalid ULP_RX_TDDP_PSZ(0x%x)\n", r);
rc = EINVAL;
}
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
cxgbe(4): Add support for Chelsio's Terminator 5 (aka T5) ASIC. This includes support for the NIC and TOE features of the 40G, 10G, and 1G/100M cards based on the T5. The ASIC is mostly backward compatible with the Terminator 4 so cxgbe(4) has been updated instead of writing a brand new driver. T5 cards will show up as cxl (short for cxlgb) ports attached to the t5nex bus driver. Sponsored by: Chelsio
2013-03-30 02:26:20 +00:00
m = v = F_TDDPTAGTCB;
r = t4_read_reg(sc, A_ULP_RX_CTL);
if ((r & m) != v) {
device_printf(sc->dev, "invalid ULP_RX_CTL(0x%x)\n", r);
rc = EINVAL;
}
Allow lazy fill up of freelists. MFC after: 3 days
2011-06-04 23:31:33 +00:00
cxgbe(4): Add support for Chelsio's Terminator 5 (aka T5) ASIC. This includes support for the NIC and TOE features of the 40G, 10G, and 1G/100M cards based on the T5. The ASIC is mostly backward compatible with the Terminator 4 so cxgbe(4) has been updated instead of writing a brand new driver. T5 cards will show up as cxl (short for cxlgb) ports attached to the t5nex bus driver. Sponsored by: Chelsio
2013-03-30 02:26:20 +00:00
m = V_INDICATESIZE(M_INDICATESIZE) | F_REARMDDPOFFSET |
F_RESETDDPOFFSET;
v = V_INDICATESIZE(indsz) | F_REARMDDPOFFSET | F_RESETDDPOFFSET;
r = t4_read_reg(sc, A_TP_PARA_REG5);
if ((r & m) != v) {
device_printf(sc->dev, "invalid TP_PARA_REG5(0x%x)\n", r);
rc = EINVAL;
}
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
cxgbe(4): Add support for Chelsio's Terminator 5 (aka T5) ASIC. This includes support for the NIC and TOE features of the 40G, 10G, and 1G/100M cards based on the T5. The ASIC is mostly backward compatible with the Terminator 4 so cxgbe(4) has been updated instead of writing a brand new driver. T5 cards will show up as cxl (short for cxlgb) ports attached to the t5nex bus driver. Sponsored by: Chelsio
2013-03-30 02:26:20 +00:00
r = t4_read_reg(sc, A_SGE_CONM_CTRL);
s->fl_starve_threshold = G_EGRTHRESHOLD(r) * 2 + 1;
cxgbe(4): The T5 allows for a different freelist starvation threshold for queues with buffer packing. Use the correct value to calculate a freelist's low water mark. MFC after: 1 week
2014-02-06 03:21:43 +00:00
if (is_t4(sc))
s->fl_starve_threshold2 = s->fl_starve_threshold;
else
s->fl_starve_threshold2 = G_EGRTHRESHOLDPACKING(r) * 2 + 1;
cxgbe(4): Add support for Chelsio's Terminator 5 (aka T5) ASIC. This includes support for the NIC and TOE features of the 40G, 10G, and 1G/100M cards based on the T5. The ASIC is mostly backward compatible with the Terminator 4 so cxgbe(4) has been updated instead of writing a brand new driver. T5 cards will show up as cxl (short for cxlgb) ports attached to the t5nex bus driver. Sponsored by: Chelsio
2013-03-30 02:26:20 +00:00
cxgbe(4): Store the log2 of the # of doorbells per BAR2 page for both ingress and egress queues, and for both T4 and T5. These values are used by the T4/T5 iWARP driver.
2013-10-14 23:32:56 +00:00
/* egress queues: log2 of # of doorbells per BAR2 page */
r = t4_read_reg(sc, A_SGE_EGRESS_QUEUES_PER_PAGE_PF);
r >>= S_QUEUESPERPAGEPF0 +
(S_QUEUESPERPAGEPF1 - S_QUEUESPERPAGEPF0) * sc->pf;
s->eq_s_qpp = r & M_QUEUESPERPAGEPF0;
/* ingress queues: log2 of # of doorbells per BAR2 page */
r = t4_read_reg(sc, A_SGE_INGRESS_QUEUES_PER_PAGE_PF);
r >>= S_QUEUESPERPAGEPF0 +
(S_QUEUESPERPAGEPF1 - S_QUEUESPERPAGEPF0) * sc->pf;
s->iq_s_qpp = r & M_QUEUESPERPAGEPF0;
cxgbe(4): Add support for Chelsio's Terminator 5 (aka T5) ASIC. This includes support for the NIC and TOE features of the 40G, 10G, and 1G/100M cards based on the T5. The ASIC is mostly backward compatible with the Terminator 4 so cxgbe(4) has been updated instead of writing a brand new driver. T5 cards will show up as cxl (short for cxlgb) ports attached to the t5nex bus driver. Sponsored by: Chelsio
2013-03-30 02:26:20 +00:00
- Read all TP parameters in one place. - Read the filter mode, calculate various shifts, and use them properly during active open (in select_ntuple). MFC after: 1 day
2013-07-04 17:55:52 +00:00
t4_init_tp_params(sc);
cxgbe(4): Add support for Chelsio's Terminator 5 (aka T5) ASIC. This includes support for the NIC and TOE features of the 40G, 10G, and 1G/100M cards based on the T5. The ASIC is mostly backward compatible with the Terminator 4 so cxgbe(4) has been updated instead of writing a brand new driver. T5 cards will show up as cxl (short for cxlgb) ports attached to the t5nex bus driver. Sponsored by: Chelsio
2013-03-30 02:26:20 +00:00
t4_read_mtu_tbl(sc, sc->params.mtus, NULL);
t4_load_mtus(sc, sc->params.mtus, sc->params.a_wnd, sc->params.b_wnd);
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
return (rc);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
}
int
t4_create_dma_tag(struct adapter *sc)
{
int rc;
rc = bus_dma_tag_create(bus_get_dma_tag(sc->dev), 1, 0,
BUS_SPACE_MAXADDR, BUS_SPACE_MAXADDR, NULL, NULL, BUS_SPACE_MAXSIZE,
BUS_SPACE_UNRESTRICTED, BUS_SPACE_MAXSIZE, BUS_DMA_ALLOCNOW, NULL,
NULL, &sc->dmat);
if (rc != 0) {
device_printf(sc->dev,
"failed to create main DMA tag: %d\n", rc);
}
return (rc);
}
Fix the sysctl that displays whether buffer packing is enabled or not.
2013-08-30 02:13:36 +00:00
static inline int
enable_buffer_packing(struct adapter *sc)
{
if (sc->flags & BUF_PACKING_OK &&
((is_t5(sc) && buffer_packing) || /* 1 or -1 both ok for T5 */
(is_t4(sc) && buffer_packing == 1)))
return (1);
return (0);
}
Display SGE tunables in the sysctl tree. dev.t5nex.0.fl_pktshift: payload DMA offset in rx buffer (bytes) dev.t5nex.0.fl_pad: payload pad boundary (bytes) dev.t5nex.0.spg_len: status page size (bytes) dev.t5nex.0.cong_drop: congestion drop setting Discussed with: scottl
2013-07-31 05:12:51 +00:00
void
t4_sge_sysctls(struct adapter *sc, struct sysctl_ctx_list *ctx,
struct sysctl_oid_list *children)
{
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
SYSCTL_ADD_PROC(ctx, children, OID_AUTO, "buffer_sizes",
CTLTYPE_STRING | CTLFLAG_RD, &sc->sge, 0, sysctl_bufsizes, "A",
"freelist buffer sizes");
Display SGE tunables in the sysctl tree. dev.t5nex.0.fl_pktshift: payload DMA offset in rx buffer (bytes) dev.t5nex.0.fl_pad: payload pad boundary (bytes) dev.t5nex.0.spg_len: status page size (bytes) dev.t5nex.0.cong_drop: congestion drop setting Discussed with: scottl
2013-07-31 05:12:51 +00:00
SYSCTL_ADD_INT(ctx, children, OID_AUTO, "fl_pktshift", CTLFLAG_RD,
NULL, fl_pktshift, "payload DMA offset in rx buffer (bytes)");
SYSCTL_ADD_INT(ctx, children, OID_AUTO, "fl_pad", CTLFLAG_RD,
NULL, fl_pad, "payload pad boundary (bytes)");
SYSCTL_ADD_INT(ctx, children, OID_AUTO, "spg_len", CTLFLAG_RD,
NULL, spg_len, "status page size (bytes)");
SYSCTL_ADD_INT(ctx, children, OID_AUTO, "cong_drop", CTLFLAG_RD,
NULL, cong_drop, "congestion drop setting");
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
SYSCTL_ADD_INT(ctx, children, OID_AUTO, "buffer_packing", CTLFLAG_RD,
Fix the sysctl that displays whether buffer packing is enabled or not.
2013-08-30 02:13:36 +00:00
NULL, enable_buffer_packing(sc),
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
"pack multiple frames in one fl buffer");
SYSCTL_ADD_INT(ctx, children, OID_AUTO, "fl_pack", CTLFLAG_RD,
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
NULL, sc->sge.pack_boundary, "payload pack boundary (bytes)");
Display SGE tunables in the sysctl tree. dev.t5nex.0.fl_pktshift: payload DMA offset in rx buffer (bytes) dev.t5nex.0.fl_pad: payload pad boundary (bytes) dev.t5nex.0.spg_len: status page size (bytes) dev.t5nex.0.cong_drop: congestion drop setting Discussed with: scottl
2013-07-31 05:12:51 +00:00
}
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
int
t4_destroy_dma_tag(struct adapter *sc)
{
if (sc->dmat)
bus_dma_tag_destroy(sc->dmat);
return (0);
}
/*
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
* Allocate and initialize the firmware event queue and the management queue.
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
*
* Returns errno on failure. Resources allocated up to that point may still be
* allocated. Caller is responsible for cleanup in case this function fails.
*/
int
- Move all Ethernet specific items from sge_eq to sge_txq. sge_eq is now a suitable base for all kinds of egress queues. - Add control queues (sge_ctrlq) and allocate one of these per hardware channel. They can be used to program filters and steer traffic (and more). MFC after: 1 week
2011-04-19 22:08:28 +00:00
t4_setup_adapter_queues(struct adapter *sc)
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
{
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
int rc;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
ADAPTER_LOCK_ASSERT_NOTOWNED(sc);
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
sysctl_ctx_init(&sc->ctx);
sc->flags |= ADAP_SYSCTL_CTX;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
- Specialized ingress queues that take interrupts for other ingress queues. Try to have a set of these per port when possible, fall back to sharing a common pool between all ports otherwise. - One control queue per port (used to be one per hardware channel). - t4_eth_rx now handles Ethernet rx only. - sysctls to display pidx/cidx for some queues. MFC after: 1 week
2011-05-30 21:34:44 +00:00
/*
* Firmware event queue
*/
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
rc = alloc_fwq(sc);
There is no need to report the same error twice. MFC after: 3 days
2012-10-10 16:54:14 +00:00
if (rc != 0)
- Move all Ethernet specific items from sge_eq to sge_txq. sge_eq is now a suitable base for all kinds of egress queues. - Add control queues (sge_ctrlq) and allocate one of these per hardware channel. They can be used to program filters and steer traffic (and more). MFC after: 1 week
2011-04-19 22:08:28 +00:00
return (rc);
/*
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
* Management queue. This is just a control queue that uses the fwq as
* its associated iq.
- Move all Ethernet specific items from sge_eq to sge_txq. sge_eq is now a suitable base for all kinds of egress queues. - Add control queues (sge_ctrlq) and allocate one of these per hardware channel. They can be used to program filters and steer traffic (and more). MFC after: 1 week
2011-04-19 22:08:28 +00:00
*/
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
rc = alloc_mgmtq(sc);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
return (rc);
}
/*
* Idempotent
*/
int
- Move all Ethernet specific items from sge_eq to sge_txq. sge_eq is now a suitable base for all kinds of egress queues. - Add control queues (sge_ctrlq) and allocate one of these per hardware channel. They can be used to program filters and steer traffic (and more). MFC after: 1 week
2011-04-19 22:08:28 +00:00
t4_teardown_adapter_queues(struct adapter *sc)
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
{
ADAPTER_LOCK_ASSERT_NOTOWNED(sc);
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
/* Do this before freeing the queue */
if (sc->flags & ADAP_SYSCTL_CTX) {
- Move all Ethernet specific items from sge_eq to sge_txq. sge_eq is now a suitable base for all kinds of egress queues. - Add control queues (sge_ctrlq) and allocate one of these per hardware channel. They can be used to program filters and steer traffic (and more). MFC after: 1 week
2011-04-19 22:08:28 +00:00
sysctl_ctx_free(&sc->ctx);
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
sc->flags &= ~ADAP_SYSCTL_CTX;
- Move all Ethernet specific items from sge_eq to sge_txq. sge_eq is now a suitable base for all kinds of egress queues. - Add control queues (sge_ctrlq) and allocate one of these per hardware channel. They can be used to program filters and steer traffic (and more). MFC after: 1 week
2011-04-19 22:08:28 +00:00
}
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
free_mgmtq(sc);
free_fwq(sc);
return (0);
}
cxgbe(4): netmap support for Terminator 5 (T5) based 10G/40G cards. Netmap gets its own hardware-assisted virtual interface and won't take over or disrupt the "normal" interface in any way. You can use both simultaneously. For kernels with DEV_NETMAP, cxgbe(4) carves out an ncxl<N> interface (note the 'n' prefix) in the hardware to accompany each cxl<N> interface. These two ifnet's per port share the same wire but really are separate interfaces in the hardware and software. Each gets its own L2 MAC addresses (unicast and multicast), MTU, checksum caps, etc. You should run netmap on the 'n' interfaces only, that's what they are for. With this, pkt-gen is able to transmit > 45Mpps out of a single 40G port of a T580 card. 2 port tx is at ~56Mpps total (28M + 28M) as of now. Single port receive is at 33Mpps but this is very much a work in progress. I expect it to be closer to 40Mpps once done. In any case the current effort can already saturate multiple 10G ports of a T5 card at the smallest legal packet size. T4 gear is totally untested. trantor:~# ./pkt-gen -i ncxl0 -f tx -D 00:07:43:ab:cd:ef 881.952141 main [1621] interface is ncxl0 881.952250 extract_ip_range [275] range is 10.0.0.1:0 to 10.0.0.1:0 881.952253 extract_ip_range [275] range is 10.1.0.1:0 to 10.1.0.1:0 881.962540 main [1804] mapped 334980KB at 0x801dff000 Sending on netmap:ncxl0: 4 queues, 1 threads and 1 cpus. 10.0.0.1 -> 10.1.0.1 (00:00:00:00:00:00 -> 00:07:43:ab:cd:ef) 881.962562 main [1882] Sending 512 packets every 0.000000000 s 881.962563 main [1884] Wait 2 secs for phy reset 884.088516 main [1886] Ready... 884.088535 nm_open [457] overriding ifname ncxl0 ringid 0x0 flags 0x1 884.088607 sender_body [996] start 884.093246 sender_body [1064] drop copy 885.090435 main_thread [1418] 45206353 pps (45289533 pkts in 1001840 usec) 886.091600 main_thread [1418] 45322792 pps (45375593 pkts in 1001165 usec) 887.092435 main_thread [1418] 45313992 pps (45351784 pkts in 1000834 usec) 888.094434 main_thread [1418] 45315765 pps (45406397 pkts in 1002000 usec) 889.095434 main_thread [1418] 45333218 pps (45378551 pkts in 1001000 usec) 890.097434 main_thread [1418] 45315247 pps (45405877 pkts in 1002000 usec) 891.099434 main_thread [1418] 45326515 pps (45417168 pkts in 1002000 usec) 892.101434 main_thread [1418] 45333039 pps (45423705 pkts in 1002000 usec) 893.103434 main_thread [1418] 45324105 pps (45414708 pkts in 1001999 usec) 894.105434 main_thread [1418] 45318042 pps (45408723 pkts in 1002001 usec) 895.106434 main_thread [1418] 45332430 pps (45377762 pkts in 1001000 usec) 896.107434 main_thread [1418] 45338072 pps (45383410 pkts in 1001000 usec) ... Relnotes: Yes Sponsored by: Chelsio Communications.
2014-05-27 18:18:41 +00:00
static inline int
port_intr_count(struct port_info *pi)
{
int rc = 0;
if (pi->flags & INTR_RXQ)
rc += pi->nrxq;
#ifdef TCP_OFFLOAD
if (pi->flags & INTR_OFLD_RXQ)
rc += pi->nofldrxq;
#endif
#ifdef DEV_NETMAP
if (pi->flags & INTR_NM_RXQ)
rc += pi->nnmrxq;
#endif
return (rc);
}
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
static inline int
first_vector(struct port_info *pi)
{
struct adapter *sc = pi->adapter;
int rc = T4_EXTRA_INTR, i;
- Move all Ethernet specific items from sge_eq to sge_txq. sge_eq is now a suitable base for all kinds of egress queues. - Add control queues (sge_ctrlq) and allocate one of these per hardware channel. They can be used to program filters and steer traffic (and more). MFC after: 1 week
2011-04-19 22:08:28 +00:00
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
if (sc->intr_count == 1)
return (0);
- Specialized ingress queues that take interrupts for other ingress queues. Try to have a set of these per port when possible, fall back to sharing a common pool between all ports otherwise. - One control queue per port (used to be one per hardware channel). - t4_eth_rx now handles Ethernet rx only. - sysctls to display pidx/cidx for some queues. MFC after: 1 week
2011-05-30 21:34:44 +00:00
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
for_each_port(sc, i) {
if (i == pi->port_id)
break;
cxgbe(4): netmap support for Terminator 5 (T5) based 10G/40G cards. Netmap gets its own hardware-assisted virtual interface and won't take over or disrupt the "normal" interface in any way. You can use both simultaneously. For kernels with DEV_NETMAP, cxgbe(4) carves out an ncxl<N> interface (note the 'n' prefix) in the hardware to accompany each cxl<N> interface. These two ifnet's per port share the same wire but really are separate interfaces in the hardware and software. Each gets its own L2 MAC addresses (unicast and multicast), MTU, checksum caps, etc. You should run netmap on the 'n' interfaces only, that's what they are for. With this, pkt-gen is able to transmit > 45Mpps out of a single 40G port of a T580 card. 2 port tx is at ~56Mpps total (28M + 28M) as of now. Single port receive is at 33Mpps but this is very much a work in progress. I expect it to be closer to 40Mpps once done. In any case the current effort can already saturate multiple 10G ports of a T5 card at the smallest legal packet size. T4 gear is totally untested. trantor:~# ./pkt-gen -i ncxl0 -f tx -D 00:07:43:ab:cd:ef 881.952141 main [1621] interface is ncxl0 881.952250 extract_ip_range [275] range is 10.0.0.1:0 to 10.0.0.1:0 881.952253 extract_ip_range [275] range is 10.1.0.1:0 to 10.1.0.1:0 881.962540 main [1804] mapped 334980KB at 0x801dff000 Sending on netmap:ncxl0: 4 queues, 1 threads and 1 cpus. 10.0.0.1 -> 10.1.0.1 (00:00:00:00:00:00 -> 00:07:43:ab:cd:ef) 881.962562 main [1882] Sending 512 packets every 0.000000000 s 881.962563 main [1884] Wait 2 secs for phy reset 884.088516 main [1886] Ready... 884.088535 nm_open [457] overriding ifname ncxl0 ringid 0x0 flags 0x1 884.088607 sender_body [996] start 884.093246 sender_body [1064] drop copy 885.090435 main_thread [1418] 45206353 pps (45289533 pkts in 1001840 usec) 886.091600 main_thread [1418] 45322792 pps (45375593 pkts in 1001165 usec) 887.092435 main_thread [1418] 45313992 pps (45351784 pkts in 1000834 usec) 888.094434 main_thread [1418] 45315765 pps (45406397 pkts in 1002000 usec) 889.095434 main_thread [1418] 45333218 pps (45378551 pkts in 1001000 usec) 890.097434 main_thread [1418] 45315247 pps (45405877 pkts in 1002000 usec) 891.099434 main_thread [1418] 45326515 pps (45417168 pkts in 1002000 usec) 892.101434 main_thread [1418] 45333039 pps (45423705 pkts in 1002000 usec) 893.103434 main_thread [1418] 45324105 pps (45414708 pkts in 1001999 usec) 894.105434 main_thread [1418] 45318042 pps (45408723 pkts in 1002001 usec) 895.106434 main_thread [1418] 45332430 pps (45377762 pkts in 1001000 usec) 896.107434 main_thread [1418] 45338072 pps (45383410 pkts in 1001000 usec) ... Relnotes: Yes Sponsored by: Chelsio Communications.
2014-05-27 18:18:41 +00:00
rc += port_intr_count(sc->port[i]);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
}
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
return (rc);
}
/*
* Given an arbitrary "index," come up with an iq that can be used by other
* queues (of this port) for interrupt forwarding, SGE egress updates, etc.
* The iq returned is guaranteed to be something that takes direct interrupts.
*/
static struct sge_iq *
port_intr_iq(struct port_info *pi, int idx)
{
struct adapter *sc = pi->adapter;
struct sge *s = &sc->sge;
struct sge_iq *iq = NULL;
cxgbe(4): netmap support for Terminator 5 (T5) based 10G/40G cards. Netmap gets its own hardware-assisted virtual interface and won't take over or disrupt the "normal" interface in any way. You can use both simultaneously. For kernels with DEV_NETMAP, cxgbe(4) carves out an ncxl<N> interface (note the 'n' prefix) in the hardware to accompany each cxl<N> interface. These two ifnet's per port share the same wire but really are separate interfaces in the hardware and software. Each gets its own L2 MAC addresses (unicast and multicast), MTU, checksum caps, etc. You should run netmap on the 'n' interfaces only, that's what they are for. With this, pkt-gen is able to transmit > 45Mpps out of a single 40G port of a T580 card. 2 port tx is at ~56Mpps total (28M + 28M) as of now. Single port receive is at 33Mpps but this is very much a work in progress. I expect it to be closer to 40Mpps once done. In any case the current effort can already saturate multiple 10G ports of a T5 card at the smallest legal packet size. T4 gear is totally untested. trantor:~# ./pkt-gen -i ncxl0 -f tx -D 00:07:43:ab:cd:ef 881.952141 main [1621] interface is ncxl0 881.952250 extract_ip_range [275] range is 10.0.0.1:0 to 10.0.0.1:0 881.952253 extract_ip_range [275] range is 10.1.0.1:0 to 10.1.0.1:0 881.962540 main [1804] mapped 334980KB at 0x801dff000 Sending on netmap:ncxl0: 4 queues, 1 threads and 1 cpus. 10.0.0.1 -> 10.1.0.1 (00:00:00:00:00:00 -> 00:07:43:ab:cd:ef) 881.962562 main [1882] Sending 512 packets every 0.000000000 s 881.962563 main [1884] Wait 2 secs for phy reset 884.088516 main [1886] Ready... 884.088535 nm_open [457] overriding ifname ncxl0 ringid 0x0 flags 0x1 884.088607 sender_body [996] start 884.093246 sender_body [1064] drop copy 885.090435 main_thread [1418] 45206353 pps (45289533 pkts in 1001840 usec) 886.091600 main_thread [1418] 45322792 pps (45375593 pkts in 1001165 usec) 887.092435 main_thread [1418] 45313992 pps (45351784 pkts in 1000834 usec) 888.094434 main_thread [1418] 45315765 pps (45406397 pkts in 1002000 usec) 889.095434 main_thread [1418] 45333218 pps (45378551 pkts in 1001000 usec) 890.097434 main_thread [1418] 45315247 pps (45405877 pkts in 1002000 usec) 891.099434 main_thread [1418] 45326515 pps (45417168 pkts in 1002000 usec) 892.101434 main_thread [1418] 45333039 pps (45423705 pkts in 1002000 usec) 893.103434 main_thread [1418] 45324105 pps (45414708 pkts in 1001999 usec) 894.105434 main_thread [1418] 45318042 pps (45408723 pkts in 1002001 usec) 895.106434 main_thread [1418] 45332430 pps (45377762 pkts in 1001000 usec) 896.107434 main_thread [1418] 45338072 pps (45383410 pkts in 1001000 usec) ... Relnotes: Yes Sponsored by: Chelsio Communications.
2014-05-27 18:18:41 +00:00
int nintr, i;
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
if (sc->intr_count == 1)
return (&sc->sge.fwq);
cxgbe(4): netmap support for Terminator 5 (T5) based 10G/40G cards. Netmap gets its own hardware-assisted virtual interface and won't take over or disrupt the "normal" interface in any way. You can use both simultaneously. For kernels with DEV_NETMAP, cxgbe(4) carves out an ncxl<N> interface (note the 'n' prefix) in the hardware to accompany each cxl<N> interface. These two ifnet's per port share the same wire but really are separate interfaces in the hardware and software. Each gets its own L2 MAC addresses (unicast and multicast), MTU, checksum caps, etc. You should run netmap on the 'n' interfaces only, that's what they are for. With this, pkt-gen is able to transmit > 45Mpps out of a single 40G port of a T580 card. 2 port tx is at ~56Mpps total (28M + 28M) as of now. Single port receive is at 33Mpps but this is very much a work in progress. I expect it to be closer to 40Mpps once done. In any case the current effort can already saturate multiple 10G ports of a T5 card at the smallest legal packet size. T4 gear is totally untested. trantor:~# ./pkt-gen -i ncxl0 -f tx -D 00:07:43:ab:cd:ef 881.952141 main [1621] interface is ncxl0 881.952250 extract_ip_range [275] range is 10.0.0.1:0 to 10.0.0.1:0 881.952253 extract_ip_range [275] range is 10.1.0.1:0 to 10.1.0.1:0 881.962540 main [1804] mapped 334980KB at 0x801dff000 Sending on netmap:ncxl0: 4 queues, 1 threads and 1 cpus. 10.0.0.1 -> 10.1.0.1 (00:00:00:00:00:00 -> 00:07:43:ab:cd:ef) 881.962562 main [1882] Sending 512 packets every 0.000000000 s 881.962563 main [1884] Wait 2 secs for phy reset 884.088516 main [1886] Ready... 884.088535 nm_open [457] overriding ifname ncxl0 ringid 0x0 flags 0x1 884.088607 sender_body [996] start 884.093246 sender_body [1064] drop copy 885.090435 main_thread [1418] 45206353 pps (45289533 pkts in 1001840 usec) 886.091600 main_thread [1418] 45322792 pps (45375593 pkts in 1001165 usec) 887.092435 main_thread [1418] 45313992 pps (45351784 pkts in 1000834 usec) 888.094434 main_thread [1418] 45315765 pps (45406397 pkts in 1002000 usec) 889.095434 main_thread [1418] 45333218 pps (45378551 pkts in 1001000 usec) 890.097434 main_thread [1418] 45315247 pps (45405877 pkts in 1002000 usec) 891.099434 main_thread [1418] 45326515 pps (45417168 pkts in 1002000 usec) 892.101434 main_thread [1418] 45333039 pps (45423705 pkts in 1002000 usec) 893.103434 main_thread [1418] 45324105 pps (45414708 pkts in 1001999 usec) 894.105434 main_thread [1418] 45318042 pps (45408723 pkts in 1002001 usec) 895.106434 main_thread [1418] 45332430 pps (45377762 pkts in 1001000 usec) 896.107434 main_thread [1418] 45338072 pps (45383410 pkts in 1001000 usec) ... Relnotes: Yes Sponsored by: Chelsio Communications.
2014-05-27 18:18:41 +00:00
nintr = port_intr_count(pi);
KASSERT(nintr != 0,
("%s: pi %p has no exclusive interrupts, total interrupts = %d",
__func__, pi, sc->intr_count));
#ifdef DEV_NETMAP
/* Exclude netmap queues as they can't take anyone else's interrupts */
if (pi->flags & INTR_NM_RXQ)
nintr -= pi->nnmrxq;
KASSERT(nintr > 0,
("%s: pi %p has nintr %d after netmap adjustment of %d", __func__,
pi, nintr, pi->nnmrxq));
#endif
i = idx % nintr;
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
cxgbe(4): netmap support for Terminator 5 (T5) based 10G/40G cards. Netmap gets its own hardware-assisted virtual interface and won't take over or disrupt the "normal" interface in any way. You can use both simultaneously. For kernels with DEV_NETMAP, cxgbe(4) carves out an ncxl<N> interface (note the 'n' prefix) in the hardware to accompany each cxl<N> interface. These two ifnet's per port share the same wire but really are separate interfaces in the hardware and software. Each gets its own L2 MAC addresses (unicast and multicast), MTU, checksum caps, etc. You should run netmap on the 'n' interfaces only, that's what they are for. With this, pkt-gen is able to transmit > 45Mpps out of a single 40G port of a T580 card. 2 port tx is at ~56Mpps total (28M + 28M) as of now. Single port receive is at 33Mpps but this is very much a work in progress. I expect it to be closer to 40Mpps once done. In any case the current effort can already saturate multiple 10G ports of a T5 card at the smallest legal packet size. T4 gear is totally untested. trantor:~# ./pkt-gen -i ncxl0 -f tx -D 00:07:43:ab:cd:ef 881.952141 main [1621] interface is ncxl0 881.952250 extract_ip_range [275] range is 10.0.0.1:0 to 10.0.0.1:0 881.952253 extract_ip_range [275] range is 10.1.0.1:0 to 10.1.0.1:0 881.962540 main [1804] mapped 334980KB at 0x801dff000 Sending on netmap:ncxl0: 4 queues, 1 threads and 1 cpus. 10.0.0.1 -> 10.1.0.1 (00:00:00:00:00:00 -> 00:07:43:ab:cd:ef) 881.962562 main [1882] Sending 512 packets every 0.000000000 s 881.962563 main [1884] Wait 2 secs for phy reset 884.088516 main [1886] Ready... 884.088535 nm_open [457] overriding ifname ncxl0 ringid 0x0 flags 0x1 884.088607 sender_body [996] start 884.093246 sender_body [1064] drop copy 885.090435 main_thread [1418] 45206353 pps (45289533 pkts in 1001840 usec) 886.091600 main_thread [1418] 45322792 pps (45375593 pkts in 1001165 usec) 887.092435 main_thread [1418] 45313992 pps (45351784 pkts in 1000834 usec) 888.094434 main_thread [1418] 45315765 pps (45406397 pkts in 1002000 usec) 889.095434 main_thread [1418] 45333218 pps (45378551 pkts in 1001000 usec) 890.097434 main_thread [1418] 45315247 pps (45405877 pkts in 1002000 usec) 891.099434 main_thread [1418] 45326515 pps (45417168 pkts in 1002000 usec) 892.101434 main_thread [1418] 45333039 pps (45423705 pkts in 1002000 usec) 893.103434 main_thread [1418] 45324105 pps (45414708 pkts in 1001999 usec) 894.105434 main_thread [1418] 45318042 pps (45408723 pkts in 1002001 usec) 895.106434 main_thread [1418] 45332430 pps (45377762 pkts in 1001000 usec) 896.107434 main_thread [1418] 45338072 pps (45383410 pkts in 1001000 usec) ... Relnotes: Yes Sponsored by: Chelsio Communications.
2014-05-27 18:18:41 +00:00
if (pi->flags & INTR_RXQ) {
if (i < pi->nrxq) {
iq = &s->rxq[pi->first_rxq + i].iq;
goto done;
}
i -= pi->nrxq;
}
#ifdef TCP_OFFLOAD
if (pi->flags & INTR_OFLD_RXQ) {
if (i < pi->nofldrxq) {
iq = &s->ofld_rxq[pi->first_ofld_rxq + i].iq;
goto done;
}
i -= pi->nofldrxq;
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
}
#endif
cxgbe(4): netmap support for Terminator 5 (T5) based 10G/40G cards. Netmap gets its own hardware-assisted virtual interface and won't take over or disrupt the "normal" interface in any way. You can use both simultaneously. For kernels with DEV_NETMAP, cxgbe(4) carves out an ncxl<N> interface (note the 'n' prefix) in the hardware to accompany each cxl<N> interface. These two ifnet's per port share the same wire but really are separate interfaces in the hardware and software. Each gets its own L2 MAC addresses (unicast and multicast), MTU, checksum caps, etc. You should run netmap on the 'n' interfaces only, that's what they are for. With this, pkt-gen is able to transmit > 45Mpps out of a single 40G port of a T580 card. 2 port tx is at ~56Mpps total (28M + 28M) as of now. Single port receive is at 33Mpps but this is very much a work in progress. I expect it to be closer to 40Mpps once done. In any case the current effort can already saturate multiple 10G ports of a T5 card at the smallest legal packet size. T4 gear is totally untested. trantor:~# ./pkt-gen -i ncxl0 -f tx -D 00:07:43:ab:cd:ef 881.952141 main [1621] interface is ncxl0 881.952250 extract_ip_range [275] range is 10.0.0.1:0 to 10.0.0.1:0 881.952253 extract_ip_range [275] range is 10.1.0.1:0 to 10.1.0.1:0 881.962540 main [1804] mapped 334980KB at 0x801dff000 Sending on netmap:ncxl0: 4 queues, 1 threads and 1 cpus. 10.0.0.1 -> 10.1.0.1 (00:00:00:00:00:00 -> 00:07:43:ab:cd:ef) 881.962562 main [1882] Sending 512 packets every 0.000000000 s 881.962563 main [1884] Wait 2 secs for phy reset 884.088516 main [1886] Ready... 884.088535 nm_open [457] overriding ifname ncxl0 ringid 0x0 flags 0x1 884.088607 sender_body [996] start 884.093246 sender_body [1064] drop copy 885.090435 main_thread [1418] 45206353 pps (45289533 pkts in 1001840 usec) 886.091600 main_thread [1418] 45322792 pps (45375593 pkts in 1001165 usec) 887.092435 main_thread [1418] 45313992 pps (45351784 pkts in 1000834 usec) 888.094434 main_thread [1418] 45315765 pps (45406397 pkts in 1002000 usec) 889.095434 main_thread [1418] 45333218 pps (45378551 pkts in 1001000 usec) 890.097434 main_thread [1418] 45315247 pps (45405877 pkts in 1002000 usec) 891.099434 main_thread [1418] 45326515 pps (45417168 pkts in 1002000 usec) 892.101434 main_thread [1418] 45333039 pps (45423705 pkts in 1002000 usec) 893.103434 main_thread [1418] 45324105 pps (45414708 pkts in 1001999 usec) 894.105434 main_thread [1418] 45318042 pps (45408723 pkts in 1002001 usec) 895.106434 main_thread [1418] 45332430 pps (45377762 pkts in 1001000 usec) 896.107434 main_thread [1418] 45338072 pps (45383410 pkts in 1001000 usec) ... Relnotes: Yes Sponsored by: Chelsio Communications.
2014-05-27 18:18:41 +00:00
panic("%s: pi %p, intr_flags 0x%lx, idx %d, total intr %d\n", __func__,
pi, pi->flags & INTR_ALL, idx, nintr);
done:
MPASS(iq != NULL);
KASSERT(iq->flags & IQ_INTR,
("%s: iq %p (port %p, intr_flags 0x%lx, idx %d)", __func__, iq, pi,
pi->flags & INTR_ALL, idx));
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
return (iq);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
}
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
/* Maximum payload that can be delivered with a single iq descriptor */
The size of the buffers in an Ethernet freelist has to be higher than the interface's MTU. Initialize such freelists with correct values. This wasn't a problem for common MTUs (1500 and 9000) as the buffers (2048 and 9216 in size) happened to have enough spare room. I ran into it when playing around with unusual MTUs. MFC after: 2 weeks
2012-08-15 01:03:13 +00:00
static inline int
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
mtu_to_max_payload(struct adapter *sc, int mtu, const int toe)
The size of the buffers in an Ethernet freelist has to be higher than the interface's MTU. Initialize such freelists with correct values. This wasn't a problem for common MTUs (1500 and 9000) as the buffers (2048 and 9216 in size) happened to have enough spare room. I ran into it when playing around with unusual MTUs. MFC after: 2 weeks
2012-08-15 01:03:13 +00:00
{
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
int payload;
The size of the buffers in an Ethernet freelist has to be higher than the interface's MTU. Initialize such freelists with correct values. This wasn't a problem for common MTUs (1500 and 9000) as the buffers (2048 and 9216 in size) happened to have enough spare room. I ran into it when playing around with unusual MTUs. MFC after: 2 weeks
2012-08-15 01:03:13 +00:00
- Make note of interface MTU change if the rx queues exist, and not just when the interface is up. - Add a tunable to control the TOE's rx coalesce feature (enabled by default as it always has been). Consider the interface MTU or the coalesce size when deciding which cluster zone to use to fill the offload rx queue's free list. The tunable is: dev.{t4nex,t5nex}.<N>.toe.rx_coalesce MFC after: 1 day
2013-07-04 21:19:01 +00:00
#ifdef TCP_OFFLOAD
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
if (toe) {
payload = sc->tt.rx_coalesce ?
G_RXCOALESCESIZE(t4_read_reg(sc, A_TP_PARA_REG2)) : mtu;
} else {
#endif
/* large enough even when hw VLAN extraction is disabled */
payload = fl_pktshift + ETHER_HDR_LEN + ETHER_VLAN_ENCAP_LEN +
mtu;
#ifdef TCP_OFFLOAD
}
#endif
payload = roundup2(payload, fl_pad);
- Make note of interface MTU change if the rx queues exist, and not just when the interface is up. - Add a tunable to control the TOE's rx coalesce feature (enabled by default as it always has been). Consider the interface MTU or the coalesce size when deciding which cluster zone to use to fill the offload rx queue's free list. The tunable is: dev.{t4nex,t5nex}.<N>.toe.rx_coalesce MFC after: 1 day
2013-07-04 21:19:01 +00:00
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
return (payload);
- Make note of interface MTU change if the rx queues exist, and not just when the interface is up. - Add a tunable to control the TOE's rx coalesce feature (enabled by default as it always has been). Consider the interface MTU or the coalesce size when deciding which cluster zone to use to fill the offload rx queue's free list. The tunable is: dev.{t4nex,t5nex}.<N>.toe.rx_coalesce MFC after: 1 day
2013-07-04 21:19:01 +00:00
}
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
int
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
t4_setup_port_queues(struct port_info *pi)
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
{
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
int rc = 0, i, j, intr_idx, iqid;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
struct sge_rxq *rxq;
struct sge_txq *txq;
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
struct sge_wrq *ctrlq;
- Updated TOE support in the kernel. - Stateful TCP offload drivers for Terminator 3 and 4 (T3 and T4) ASICs. These are available as t3_tom and t4_tom modules that augment cxgb(4) and cxgbe(4) respectively. The cxgb/cxgbe drivers continue to work as usual with or without these extra features. - iWARP driver for Terminator 3 ASIC (kernel verbs). T4 iWARP in the works and will follow soon. Build-tested with make universe. 30s overview ============ What interfaces support TCP offload? Look for TOE4 and/or TOE6 in the capabilities of an interface: # ifconfig -m | grep TOE Enable/disable TCP offload on an interface (just like any other ifnet capability): # ifconfig cxgbe0 toe # ifconfig cxgbe0 -toe Which connections are offloaded? Look for toe4 and/or toe6 in the output of netstat and sockstat: # netstat -np tcp | grep toe # sockstat -46c | grep toe Reviewed by: bz, gnn Sponsored by: Chelsio communications. MFC after: ~3 months (after 9.1, and after ensuring MFC is feasible)
2012-06-19 07:34:13 +00:00
#ifdef TCP_OFFLOAD
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
struct sge_ofld_rxq *ofld_rxq;
struct sge_wrq *ofld_txq;
cxgbe(4): netmap support for Terminator 5 (T5) based 10G/40G cards. Netmap gets its own hardware-assisted virtual interface and won't take over or disrupt the "normal" interface in any way. You can use both simultaneously. For kernels with DEV_NETMAP, cxgbe(4) carves out an ncxl<N> interface (note the 'n' prefix) in the hardware to accompany each cxl<N> interface. These two ifnet's per port share the same wire but really are separate interfaces in the hardware and software. Each gets its own L2 MAC addresses (unicast and multicast), MTU, checksum caps, etc. You should run netmap on the 'n' interfaces only, that's what they are for. With this, pkt-gen is able to transmit > 45Mpps out of a single 40G port of a T580 card. 2 port tx is at ~56Mpps total (28M + 28M) as of now. Single port receive is at 33Mpps but this is very much a work in progress. I expect it to be closer to 40Mpps once done. In any case the current effort can already saturate multiple 10G ports of a T5 card at the smallest legal packet size. T4 gear is totally untested. trantor:~# ./pkt-gen -i ncxl0 -f tx -D 00:07:43:ab:cd:ef 881.952141 main [1621] interface is ncxl0 881.952250 extract_ip_range [275] range is 10.0.0.1:0 to 10.0.0.1:0 881.952253 extract_ip_range [275] range is 10.1.0.1:0 to 10.1.0.1:0 881.962540 main [1804] mapped 334980KB at 0x801dff000 Sending on netmap:ncxl0: 4 queues, 1 threads and 1 cpus. 10.0.0.1 -> 10.1.0.1 (00:00:00:00:00:00 -> 00:07:43:ab:cd:ef) 881.962562 main [1882] Sending 512 packets every 0.000000000 s 881.962563 main [1884] Wait 2 secs for phy reset 884.088516 main [1886] Ready... 884.088535 nm_open [457] overriding ifname ncxl0 ringid 0x0 flags 0x1 884.088607 sender_body [996] start 884.093246 sender_body [1064] drop copy 885.090435 main_thread [1418] 45206353 pps (45289533 pkts in 1001840 usec) 886.091600 main_thread [1418] 45322792 pps (45375593 pkts in 1001165 usec) 887.092435 main_thread [1418] 45313992 pps (45351784 pkts in 1000834 usec) 888.094434 main_thread [1418] 45315765 pps (45406397 pkts in 1002000 usec) 889.095434 main_thread [1418] 45333218 pps (45378551 pkts in 1001000 usec) 890.097434 main_thread [1418] 45315247 pps (45405877 pkts in 1002000 usec) 891.099434 main_thread [1418] 45326515 pps (45417168 pkts in 1002000 usec) 892.101434 main_thread [1418] 45333039 pps (45423705 pkts in 1002000 usec) 893.103434 main_thread [1418] 45324105 pps (45414708 pkts in 1001999 usec) 894.105434 main_thread [1418] 45318042 pps (45408723 pkts in 1002001 usec) 895.106434 main_thread [1418] 45332430 pps (45377762 pkts in 1001000 usec) 896.107434 main_thread [1418] 45338072 pps (45383410 pkts in 1001000 usec) ... Relnotes: Yes Sponsored by: Chelsio Communications.
2014-05-27 18:18:41 +00:00
#endif
#ifdef DEV_NETMAP
struct sge_nm_rxq *nm_rxq;
struct sge_nm_txq *nm_txq;
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
#endif
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
char name[16];
struct adapter *sc = pi->adapter;
- Make note of interface MTU change if the rx queues exist, and not just when the interface is up. - Add a tunable to control the TOE's rx coalesce feature (enabled by default as it always has been). Consider the interface MTU or the coalesce size when deciding which cluster zone to use to fill the offload rx queue's free list. The tunable is: dev.{t4nex,t5nex}.<N>.toe.rx_coalesce MFC after: 1 day
2013-07-04 21:19:01 +00:00
struct ifnet *ifp = pi->ifp;
- Updated TOE support in the kernel. - Stateful TCP offload drivers for Terminator 3 and 4 (T3 and T4) ASICs. These are available as t3_tom and t4_tom modules that augment cxgb(4) and cxgbe(4) respectively. The cxgb/cxgbe drivers continue to work as usual with or without these extra features. - iWARP driver for Terminator 3 ASIC (kernel verbs). T4 iWARP in the works and will follow soon. Build-tested with make universe. 30s overview ============ What interfaces support TCP offload? Look for TOE4 and/or TOE6 in the capabilities of an interface: # ifconfig -m | grep TOE Enable/disable TCP offload on an interface (just like any other ifnet capability): # ifconfig cxgbe0 toe # ifconfig cxgbe0 -toe Which connections are offloaded? Look for toe4 and/or toe6 in the output of netstat and sockstat: # netstat -np tcp | grep toe # sockstat -46c | grep toe Reviewed by: bz, gnn Sponsored by: Chelsio communications. MFC after: ~3 months (after 9.1, and after ensuring MFC is feasible)
2012-06-19 07:34:13 +00:00
struct sysctl_oid *oid = device_get_sysctl_tree(pi->dev);
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
struct sysctl_oid_list *children = SYSCTL_CHILDREN(oid);
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
int maxp, pack, mtu = ifp->if_mtu;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
/* Interrupt vector to start from (when using multiple vectors) */
intr_idx = first_vector(pi);
/*
cxgbe(4): netmap support for Terminator 5 (T5) based 10G/40G cards. Netmap gets its own hardware-assisted virtual interface and won't take over or disrupt the "normal" interface in any way. You can use both simultaneously. For kernels with DEV_NETMAP, cxgbe(4) carves out an ncxl<N> interface (note the 'n' prefix) in the hardware to accompany each cxl<N> interface. These two ifnet's per port share the same wire but really are separate interfaces in the hardware and software. Each gets its own L2 MAC addresses (unicast and multicast), MTU, checksum caps, etc. You should run netmap on the 'n' interfaces only, that's what they are for. With this, pkt-gen is able to transmit > 45Mpps out of a single 40G port of a T580 card. 2 port tx is at ~56Mpps total (28M + 28M) as of now. Single port receive is at 33Mpps but this is very much a work in progress. I expect it to be closer to 40Mpps once done. In any case the current effort can already saturate multiple 10G ports of a T5 card at the smallest legal packet size. T4 gear is totally untested. trantor:~# ./pkt-gen -i ncxl0 -f tx -D 00:07:43:ab:cd:ef 881.952141 main [1621] interface is ncxl0 881.952250 extract_ip_range [275] range is 10.0.0.1:0 to 10.0.0.1:0 881.952253 extract_ip_range [275] range is 10.1.0.1:0 to 10.1.0.1:0 881.962540 main [1804] mapped 334980KB at 0x801dff000 Sending on netmap:ncxl0: 4 queues, 1 threads and 1 cpus. 10.0.0.1 -> 10.1.0.1 (00:00:00:00:00:00 -> 00:07:43:ab:cd:ef) 881.962562 main [1882] Sending 512 packets every 0.000000000 s 881.962563 main [1884] Wait 2 secs for phy reset 884.088516 main [1886] Ready... 884.088535 nm_open [457] overriding ifname ncxl0 ringid 0x0 flags 0x1 884.088607 sender_body [996] start 884.093246 sender_body [1064] drop copy 885.090435 main_thread [1418] 45206353 pps (45289533 pkts in 1001840 usec) 886.091600 main_thread [1418] 45322792 pps (45375593 pkts in 1001165 usec) 887.092435 main_thread [1418] 45313992 pps (45351784 pkts in 1000834 usec) 888.094434 main_thread [1418] 45315765 pps (45406397 pkts in 1002000 usec) 889.095434 main_thread [1418] 45333218 pps (45378551 pkts in 1001000 usec) 890.097434 main_thread [1418] 45315247 pps (45405877 pkts in 1002000 usec) 891.099434 main_thread [1418] 45326515 pps (45417168 pkts in 1002000 usec) 892.101434 main_thread [1418] 45333039 pps (45423705 pkts in 1002000 usec) 893.103434 main_thread [1418] 45324105 pps (45414708 pkts in 1001999 usec) 894.105434 main_thread [1418] 45318042 pps (45408723 pkts in 1002001 usec) 895.106434 main_thread [1418] 45332430 pps (45377762 pkts in 1001000 usec) 896.107434 main_thread [1418] 45338072 pps (45383410 pkts in 1001000 usec) ... Relnotes: Yes Sponsored by: Chelsio Communications.
2014-05-27 18:18:41 +00:00
* First pass over all NIC and TOE rx queues:
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
* a) initialize iq and fl
* b) allocate queue iff it will take direct interrupts.
*/
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
maxp = mtu_to_max_payload(sc, mtu, 0);
Fix the sysctl that displays whether buffer packing is enabled or not.
2013-08-30 02:13:36 +00:00
pack = enable_buffer_packing(sc);
cxgbe(4): netmap support for Terminator 5 (T5) based 10G/40G cards. Netmap gets its own hardware-assisted virtual interface and won't take over or disrupt the "normal" interface in any way. You can use both simultaneously. For kernels with DEV_NETMAP, cxgbe(4) carves out an ncxl<N> interface (note the 'n' prefix) in the hardware to accompany each cxl<N> interface. These two ifnet's per port share the same wire but really are separate interfaces in the hardware and software. Each gets its own L2 MAC addresses (unicast and multicast), MTU, checksum caps, etc. You should run netmap on the 'n' interfaces only, that's what they are for. With this, pkt-gen is able to transmit > 45Mpps out of a single 40G port of a T580 card. 2 port tx is at ~56Mpps total (28M + 28M) as of now. Single port receive is at 33Mpps but this is very much a work in progress. I expect it to be closer to 40Mpps once done. In any case the current effort can already saturate multiple 10G ports of a T5 card at the smallest legal packet size. T4 gear is totally untested. trantor:~# ./pkt-gen -i ncxl0 -f tx -D 00:07:43:ab:cd:ef 881.952141 main [1621] interface is ncxl0 881.952250 extract_ip_range [275] range is 10.0.0.1:0 to 10.0.0.1:0 881.952253 extract_ip_range [275] range is 10.1.0.1:0 to 10.1.0.1:0 881.962540 main [1804] mapped 334980KB at 0x801dff000 Sending on netmap:ncxl0: 4 queues, 1 threads and 1 cpus. 10.0.0.1 -> 10.1.0.1 (00:00:00:00:00:00 -> 00:07:43:ab:cd:ef) 881.962562 main [1882] Sending 512 packets every 0.000000000 s 881.962563 main [1884] Wait 2 secs for phy reset 884.088516 main [1886] Ready... 884.088535 nm_open [457] overriding ifname ncxl0 ringid 0x0 flags 0x1 884.088607 sender_body [996] start 884.093246 sender_body [1064] drop copy 885.090435 main_thread [1418] 45206353 pps (45289533 pkts in 1001840 usec) 886.091600 main_thread [1418] 45322792 pps (45375593 pkts in 1001165 usec) 887.092435 main_thread [1418] 45313992 pps (45351784 pkts in 1000834 usec) 888.094434 main_thread [1418] 45315765 pps (45406397 pkts in 1002000 usec) 889.095434 main_thread [1418] 45333218 pps (45378551 pkts in 1001000 usec) 890.097434 main_thread [1418] 45315247 pps (45405877 pkts in 1002000 usec) 891.099434 main_thread [1418] 45326515 pps (45417168 pkts in 1002000 usec) 892.101434 main_thread [1418] 45333039 pps (45423705 pkts in 1002000 usec) 893.103434 main_thread [1418] 45324105 pps (45414708 pkts in 1001999 usec) 894.105434 main_thread [1418] 45318042 pps (45408723 pkts in 1002001 usec) 895.106434 main_thread [1418] 45332430 pps (45377762 pkts in 1001000 usec) 896.107434 main_thread [1418] 45338072 pps (45383410 pkts in 1001000 usec) ... Relnotes: Yes Sponsored by: Chelsio Communications.
2014-05-27 18:18:41 +00:00
if (pi->flags & INTR_RXQ) {
oid = SYSCTL_ADD_NODE(&pi->ctx, children, OID_AUTO, "rxq",
CTLFLAG_RD, NULL, "rx queues");
}
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
for_each_rxq(pi, i, rxq) {
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
init_iq(&rxq->iq, sc, pi->tmr_idx, pi->pktc_idx, pi->qsize_rxq,
Remove unused item. cxgbe's rx queue's lock was removed a long time ago. MFC after: 3 days
2012-10-10 16:52:39 +00:00
RX_IQ_ESIZE);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
snprintf(name, sizeof(name), "%s rxq%d-fl",
device_get_nameunit(pi->dev), i);
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
init_fl(sc, &rxq->fl, pi->qsize_rxq / 8, maxp, pack, name);
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
cxgbe(4): netmap support for Terminator 5 (T5) based 10G/40G cards. Netmap gets its own hardware-assisted virtual interface and won't take over or disrupt the "normal" interface in any way. You can use both simultaneously. For kernels with DEV_NETMAP, cxgbe(4) carves out an ncxl<N> interface (note the 'n' prefix) in the hardware to accompany each cxl<N> interface. These two ifnet's per port share the same wire but really are separate interfaces in the hardware and software. Each gets its own L2 MAC addresses (unicast and multicast), MTU, checksum caps, etc. You should run netmap on the 'n' interfaces only, that's what they are for. With this, pkt-gen is able to transmit > 45Mpps out of a single 40G port of a T580 card. 2 port tx is at ~56Mpps total (28M + 28M) as of now. Single port receive is at 33Mpps but this is very much a work in progress. I expect it to be closer to 40Mpps once done. In any case the current effort can already saturate multiple 10G ports of a T5 card at the smallest legal packet size. T4 gear is totally untested. trantor:~# ./pkt-gen -i ncxl0 -f tx -D 00:07:43:ab:cd:ef 881.952141 main [1621] interface is ncxl0 881.952250 extract_ip_range [275] range is 10.0.0.1:0 to 10.0.0.1:0 881.952253 extract_ip_range [275] range is 10.1.0.1:0 to 10.1.0.1:0 881.962540 main [1804] mapped 334980KB at 0x801dff000 Sending on netmap:ncxl0: 4 queues, 1 threads and 1 cpus. 10.0.0.1 -> 10.1.0.1 (00:00:00:00:00:00 -> 00:07:43:ab:cd:ef) 881.962562 main [1882] Sending 512 packets every 0.000000000 s 881.962563 main [1884] Wait 2 secs for phy reset 884.088516 main [1886] Ready... 884.088535 nm_open [457] overriding ifname ncxl0 ringid 0x0 flags 0x1 884.088607 sender_body [996] start 884.093246 sender_body [1064] drop copy 885.090435 main_thread [1418] 45206353 pps (45289533 pkts in 1001840 usec) 886.091600 main_thread [1418] 45322792 pps (45375593 pkts in 1001165 usec) 887.092435 main_thread [1418] 45313992 pps (45351784 pkts in 1000834 usec) 888.094434 main_thread [1418] 45315765 pps (45406397 pkts in 1002000 usec) 889.095434 main_thread [1418] 45333218 pps (45378551 pkts in 1001000 usec) 890.097434 main_thread [1418] 45315247 pps (45405877 pkts in 1002000 usec) 891.099434 main_thread [1418] 45326515 pps (45417168 pkts in 1002000 usec) 892.101434 main_thread [1418] 45333039 pps (45423705 pkts in 1002000 usec) 893.103434 main_thread [1418] 45324105 pps (45414708 pkts in 1001999 usec) 894.105434 main_thread [1418] 45318042 pps (45408723 pkts in 1002001 usec) 895.106434 main_thread [1418] 45332430 pps (45377762 pkts in 1001000 usec) 896.107434 main_thread [1418] 45338072 pps (45383410 pkts in 1001000 usec) ... Relnotes: Yes Sponsored by: Chelsio Communications.
2014-05-27 18:18:41 +00:00
if (pi->flags & INTR_RXQ) {
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
rxq->iq.flags |= IQ_INTR;
rc = alloc_rxq(pi, rxq, intr_idx, i, oid);
if (rc != 0)
goto done;
intr_idx++;
}
}
- Updated TOE support in the kernel. - Stateful TCP offload drivers for Terminator 3 and 4 (T3 and T4) ASICs. These are available as t3_tom and t4_tom modules that augment cxgb(4) and cxgbe(4) respectively. The cxgb/cxgbe drivers continue to work as usual with or without these extra features. - iWARP driver for Terminator 3 ASIC (kernel verbs). T4 iWARP in the works and will follow soon. Build-tested with make universe. 30s overview ============ What interfaces support TCP offload? Look for TOE4 and/or TOE6 in the capabilities of an interface: # ifconfig -m | grep TOE Enable/disable TCP offload on an interface (just like any other ifnet capability): # ifconfig cxgbe0 toe # ifconfig cxgbe0 -toe Which connections are offloaded? Look for toe4 and/or toe6 in the output of netstat and sockstat: # netstat -np tcp | grep toe # sockstat -46c | grep toe Reviewed by: bz, gnn Sponsored by: Chelsio communications. MFC after: ~3 months (after 9.1, and after ensuring MFC is feasible)
2012-06-19 07:34:13 +00:00
#ifdef TCP_OFFLOAD
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
maxp = mtu_to_max_payload(sc, mtu, 1);
cxgbe(4): netmap support for Terminator 5 (T5) based 10G/40G cards. Netmap gets its own hardware-assisted virtual interface and won't take over or disrupt the "normal" interface in any way. You can use both simultaneously. For kernels with DEV_NETMAP, cxgbe(4) carves out an ncxl<N> interface (note the 'n' prefix) in the hardware to accompany each cxl<N> interface. These two ifnet's per port share the same wire but really are separate interfaces in the hardware and software. Each gets its own L2 MAC addresses (unicast and multicast), MTU, checksum caps, etc. You should run netmap on the 'n' interfaces only, that's what they are for. With this, pkt-gen is able to transmit > 45Mpps out of a single 40G port of a T580 card. 2 port tx is at ~56Mpps total (28M + 28M) as of now. Single port receive is at 33Mpps but this is very much a work in progress. I expect it to be closer to 40Mpps once done. In any case the current effort can already saturate multiple 10G ports of a T5 card at the smallest legal packet size. T4 gear is totally untested. trantor:~# ./pkt-gen -i ncxl0 -f tx -D 00:07:43:ab:cd:ef 881.952141 main [1621] interface is ncxl0 881.952250 extract_ip_range [275] range is 10.0.0.1:0 to 10.0.0.1:0 881.952253 extract_ip_range [275] range is 10.1.0.1:0 to 10.1.0.1:0 881.962540 main [1804] mapped 334980KB at 0x801dff000 Sending on netmap:ncxl0: 4 queues, 1 threads and 1 cpus. 10.0.0.1 -> 10.1.0.1 (00:00:00:00:00:00 -> 00:07:43:ab:cd:ef) 881.962562 main [1882] Sending 512 packets every 0.000000000 s 881.962563 main [1884] Wait 2 secs for phy reset 884.088516 main [1886] Ready... 884.088535 nm_open [457] overriding ifname ncxl0 ringid 0x0 flags 0x1 884.088607 sender_body [996] start 884.093246 sender_body [1064] drop copy 885.090435 main_thread [1418] 45206353 pps (45289533 pkts in 1001840 usec) 886.091600 main_thread [1418] 45322792 pps (45375593 pkts in 1001165 usec) 887.092435 main_thread [1418] 45313992 pps (45351784 pkts in 1000834 usec) 888.094434 main_thread [1418] 45315765 pps (45406397 pkts in 1002000 usec) 889.095434 main_thread [1418] 45333218 pps (45378551 pkts in 1001000 usec) 890.097434 main_thread [1418] 45315247 pps (45405877 pkts in 1002000 usec) 891.099434 main_thread [1418] 45326515 pps (45417168 pkts in 1002000 usec) 892.101434 main_thread [1418] 45333039 pps (45423705 pkts in 1002000 usec) 893.103434 main_thread [1418] 45324105 pps (45414708 pkts in 1001999 usec) 894.105434 main_thread [1418] 45318042 pps (45408723 pkts in 1002001 usec) 895.106434 main_thread [1418] 45332430 pps (45377762 pkts in 1001000 usec) 896.107434 main_thread [1418] 45338072 pps (45383410 pkts in 1001000 usec) ... Relnotes: Yes Sponsored by: Chelsio Communications.
2014-05-27 18:18:41 +00:00
if (is_offload(sc) && pi->flags & INTR_OFLD_RXQ) {
oid = SYSCTL_ADD_NODE(&pi->ctx, children, OID_AUTO, "ofld_rxq",
CTLFLAG_RD, NULL,
"rx queues for offloaded TCP connections");
}
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
for_each_ofld_rxq(pi, i, ofld_rxq) {
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
init_iq(&ofld_rxq->iq, sc, pi->tmr_idx, pi->pktc_idx,
Remove unused item. cxgbe's rx queue's lock was removed a long time ago. MFC after: 3 days
2012-10-10 16:52:39 +00:00
pi->qsize_rxq, RX_IQ_ESIZE);
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
snprintf(name, sizeof(name), "%s ofld_rxq%d-fl",
device_get_nameunit(pi->dev), i);
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
init_fl(sc, &ofld_rxq->fl, pi->qsize_rxq / 8, maxp, pack, name);
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
cxgbe(4): netmap support for Terminator 5 (T5) based 10G/40G cards. Netmap gets its own hardware-assisted virtual interface and won't take over or disrupt the "normal" interface in any way. You can use both simultaneously. For kernels with DEV_NETMAP, cxgbe(4) carves out an ncxl<N> interface (note the 'n' prefix) in the hardware to accompany each cxl<N> interface. These two ifnet's per port share the same wire but really are separate interfaces in the hardware and software. Each gets its own L2 MAC addresses (unicast and multicast), MTU, checksum caps, etc. You should run netmap on the 'n' interfaces only, that's what they are for. With this, pkt-gen is able to transmit > 45Mpps out of a single 40G port of a T580 card. 2 port tx is at ~56Mpps total (28M + 28M) as of now. Single port receive is at 33Mpps but this is very much a work in progress. I expect it to be closer to 40Mpps once done. In any case the current effort can already saturate multiple 10G ports of a T5 card at the smallest legal packet size. T4 gear is totally untested. trantor:~# ./pkt-gen -i ncxl0 -f tx -D 00:07:43:ab:cd:ef 881.952141 main [1621] interface is ncxl0 881.952250 extract_ip_range [275] range is 10.0.0.1:0 to 10.0.0.1:0 881.952253 extract_ip_range [275] range is 10.1.0.1:0 to 10.1.0.1:0 881.962540 main [1804] mapped 334980KB at 0x801dff000 Sending on netmap:ncxl0: 4 queues, 1 threads and 1 cpus. 10.0.0.1 -> 10.1.0.1 (00:00:00:00:00:00 -> 00:07:43:ab:cd:ef) 881.962562 main [1882] Sending 512 packets every 0.000000000 s 881.962563 main [1884] Wait 2 secs for phy reset 884.088516 main [1886] Ready... 884.088535 nm_open [457] overriding ifname ncxl0 ringid 0x0 flags 0x1 884.088607 sender_body [996] start 884.093246 sender_body [1064] drop copy 885.090435 main_thread [1418] 45206353 pps (45289533 pkts in 1001840 usec) 886.091600 main_thread [1418] 45322792 pps (45375593 pkts in 1001165 usec) 887.092435 main_thread [1418] 45313992 pps (45351784 pkts in 1000834 usec) 888.094434 main_thread [1418] 45315765 pps (45406397 pkts in 1002000 usec) 889.095434 main_thread [1418] 45333218 pps (45378551 pkts in 1001000 usec) 890.097434 main_thread [1418] 45315247 pps (45405877 pkts in 1002000 usec) 891.099434 main_thread [1418] 45326515 pps (45417168 pkts in 1002000 usec) 892.101434 main_thread [1418] 45333039 pps (45423705 pkts in 1002000 usec) 893.103434 main_thread [1418] 45324105 pps (45414708 pkts in 1001999 usec) 894.105434 main_thread [1418] 45318042 pps (45408723 pkts in 1002001 usec) 895.106434 main_thread [1418] 45332430 pps (45377762 pkts in 1001000 usec) 896.107434 main_thread [1418] 45338072 pps (45383410 pkts in 1001000 usec) ... Relnotes: Yes Sponsored by: Chelsio Communications.
2014-05-27 18:18:41 +00:00
if (pi->flags & INTR_OFLD_RXQ) {
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
ofld_rxq->iq.flags |= IQ_INTR;
cxgbe(4): netmap support for Terminator 5 (T5) based 10G/40G cards. Netmap gets its own hardware-assisted virtual interface and won't take over or disrupt the "normal" interface in any way. You can use both simultaneously. For kernels with DEV_NETMAP, cxgbe(4) carves out an ncxl<N> interface (note the 'n' prefix) in the hardware to accompany each cxl<N> interface. These two ifnet's per port share the same wire but really are separate interfaces in the hardware and software. Each gets its own L2 MAC addresses (unicast and multicast), MTU, checksum caps, etc. You should run netmap on the 'n' interfaces only, that's what they are for. With this, pkt-gen is able to transmit > 45Mpps out of a single 40G port of a T580 card. 2 port tx is at ~56Mpps total (28M + 28M) as of now. Single port receive is at 33Mpps but this is very much a work in progress. I expect it to be closer to 40Mpps once done. In any case the current effort can already saturate multiple 10G ports of a T5 card at the smallest legal packet size. T4 gear is totally untested. trantor:~# ./pkt-gen -i ncxl0 -f tx -D 00:07:43:ab:cd:ef 881.952141 main [1621] interface is ncxl0 881.952250 extract_ip_range [275] range is 10.0.0.1:0 to 10.0.0.1:0 881.952253 extract_ip_range [275] range is 10.1.0.1:0 to 10.1.0.1:0 881.962540 main [1804] mapped 334980KB at 0x801dff000 Sending on netmap:ncxl0: 4 queues, 1 threads and 1 cpus. 10.0.0.1 -> 10.1.0.1 (00:00:00:00:00:00 -> 00:07:43:ab:cd:ef) 881.962562 main [1882] Sending 512 packets every 0.000000000 s 881.962563 main [1884] Wait 2 secs for phy reset 884.088516 main [1886] Ready... 884.088535 nm_open [457] overriding ifname ncxl0 ringid 0x0 flags 0x1 884.088607 sender_body [996] start 884.093246 sender_body [1064] drop copy 885.090435 main_thread [1418] 45206353 pps (45289533 pkts in 1001840 usec) 886.091600 main_thread [1418] 45322792 pps (45375593 pkts in 1001165 usec) 887.092435 main_thread [1418] 45313992 pps (45351784 pkts in 1000834 usec) 888.094434 main_thread [1418] 45315765 pps (45406397 pkts in 1002000 usec) 889.095434 main_thread [1418] 45333218 pps (45378551 pkts in 1001000 usec) 890.097434 main_thread [1418] 45315247 pps (45405877 pkts in 1002000 usec) 891.099434 main_thread [1418] 45326515 pps (45417168 pkts in 1002000 usec) 892.101434 main_thread [1418] 45333039 pps (45423705 pkts in 1002000 usec) 893.103434 main_thread [1418] 45324105 pps (45414708 pkts in 1001999 usec) 894.105434 main_thread [1418] 45318042 pps (45408723 pkts in 1002001 usec) 895.106434 main_thread [1418] 45332430 pps (45377762 pkts in 1001000 usec) 896.107434 main_thread [1418] 45338072 pps (45383410 pkts in 1001000 usec) ... Relnotes: Yes Sponsored by: Chelsio Communications.
2014-05-27 18:18:41 +00:00
rc = alloc_ofld_rxq(pi, ofld_rxq, intr_idx, i, oid);
if (rc != 0)
goto done;
intr_idx++;
}
}
#endif
#ifdef DEV_NETMAP
/*
* We don't have buffers to back the netmap rx queues right now so we
* create the queues in a way that doesn't set off any congestion signal
* in the chip.
*/
if (pi->flags & INTR_NM_RXQ) {
oid = SYSCTL_ADD_NODE(&pi->ctx, children, OID_AUTO, "nm_rxq",
CTLFLAG_RD, NULL, "rx queues for netmap");
for_each_nm_rxq(pi, i, nm_rxq) {
rc = alloc_nm_rxq(pi, nm_rxq, intr_idx, i, oid);
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
if (rc != 0)
goto done;
intr_idx++;
}
}
#endif
/*
cxgbe(4): netmap support for Terminator 5 (T5) based 10G/40G cards. Netmap gets its own hardware-assisted virtual interface and won't take over or disrupt the "normal" interface in any way. You can use both simultaneously. For kernels with DEV_NETMAP, cxgbe(4) carves out an ncxl<N> interface (note the 'n' prefix) in the hardware to accompany each cxl<N> interface. These two ifnet's per port share the same wire but really are separate interfaces in the hardware and software. Each gets its own L2 MAC addresses (unicast and multicast), MTU, checksum caps, etc. You should run netmap on the 'n' interfaces only, that's what they are for. With this, pkt-gen is able to transmit > 45Mpps out of a single 40G port of a T580 card. 2 port tx is at ~56Mpps total (28M + 28M) as of now. Single port receive is at 33Mpps but this is very much a work in progress. I expect it to be closer to 40Mpps once done. In any case the current effort can already saturate multiple 10G ports of a T5 card at the smallest legal packet size. T4 gear is totally untested. trantor:~# ./pkt-gen -i ncxl0 -f tx -D 00:07:43:ab:cd:ef 881.952141 main [1621] interface is ncxl0 881.952250 extract_ip_range [275] range is 10.0.0.1:0 to 10.0.0.1:0 881.952253 extract_ip_range [275] range is 10.1.0.1:0 to 10.1.0.1:0 881.962540 main [1804] mapped 334980KB at 0x801dff000 Sending on netmap:ncxl0: 4 queues, 1 threads and 1 cpus. 10.0.0.1 -> 10.1.0.1 (00:00:00:00:00:00 -> 00:07:43:ab:cd:ef) 881.962562 main [1882] Sending 512 packets every 0.000000000 s 881.962563 main [1884] Wait 2 secs for phy reset 884.088516 main [1886] Ready... 884.088535 nm_open [457] overriding ifname ncxl0 ringid 0x0 flags 0x1 884.088607 sender_body [996] start 884.093246 sender_body [1064] drop copy 885.090435 main_thread [1418] 45206353 pps (45289533 pkts in 1001840 usec) 886.091600 main_thread [1418] 45322792 pps (45375593 pkts in 1001165 usec) 887.092435 main_thread [1418] 45313992 pps (45351784 pkts in 1000834 usec) 888.094434 main_thread [1418] 45315765 pps (45406397 pkts in 1002000 usec) 889.095434 main_thread [1418] 45333218 pps (45378551 pkts in 1001000 usec) 890.097434 main_thread [1418] 45315247 pps (45405877 pkts in 1002000 usec) 891.099434 main_thread [1418] 45326515 pps (45417168 pkts in 1002000 usec) 892.101434 main_thread [1418] 45333039 pps (45423705 pkts in 1002000 usec) 893.103434 main_thread [1418] 45324105 pps (45414708 pkts in 1001999 usec) 894.105434 main_thread [1418] 45318042 pps (45408723 pkts in 1002001 usec) 895.106434 main_thread [1418] 45332430 pps (45377762 pkts in 1001000 usec) 896.107434 main_thread [1418] 45338072 pps (45383410 pkts in 1001000 usec) ... Relnotes: Yes Sponsored by: Chelsio Communications.
2014-05-27 18:18:41 +00:00
* Second pass over all NIC and TOE rx queues. The queues forwarding
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
* their interrupts are allocated now.
*/
j = 0;
cxgbe(4): netmap support for Terminator 5 (T5) based 10G/40G cards. Netmap gets its own hardware-assisted virtual interface and won't take over or disrupt the "normal" interface in any way. You can use both simultaneously. For kernels with DEV_NETMAP, cxgbe(4) carves out an ncxl<N> interface (note the 'n' prefix) in the hardware to accompany each cxl<N> interface. These two ifnet's per port share the same wire but really are separate interfaces in the hardware and software. Each gets its own L2 MAC addresses (unicast and multicast), MTU, checksum caps, etc. You should run netmap on the 'n' interfaces only, that's what they are for. With this, pkt-gen is able to transmit > 45Mpps out of a single 40G port of a T580 card. 2 port tx is at ~56Mpps total (28M + 28M) as of now. Single port receive is at 33Mpps but this is very much a work in progress. I expect it to be closer to 40Mpps once done. In any case the current effort can already saturate multiple 10G ports of a T5 card at the smallest legal packet size. T4 gear is totally untested. trantor:~# ./pkt-gen -i ncxl0 -f tx -D 00:07:43:ab:cd:ef 881.952141 main [1621] interface is ncxl0 881.952250 extract_ip_range [275] range is 10.0.0.1:0 to 10.0.0.1:0 881.952253 extract_ip_range [275] range is 10.1.0.1:0 to 10.1.0.1:0 881.962540 main [1804] mapped 334980KB at 0x801dff000 Sending on netmap:ncxl0: 4 queues, 1 threads and 1 cpus. 10.0.0.1 -> 10.1.0.1 (00:00:00:00:00:00 -> 00:07:43:ab:cd:ef) 881.962562 main [1882] Sending 512 packets every 0.000000000 s 881.962563 main [1884] Wait 2 secs for phy reset 884.088516 main [1886] Ready... 884.088535 nm_open [457] overriding ifname ncxl0 ringid 0x0 flags 0x1 884.088607 sender_body [996] start 884.093246 sender_body [1064] drop copy 885.090435 main_thread [1418] 45206353 pps (45289533 pkts in 1001840 usec) 886.091600 main_thread [1418] 45322792 pps (45375593 pkts in 1001165 usec) 887.092435 main_thread [1418] 45313992 pps (45351784 pkts in 1000834 usec) 888.094434 main_thread [1418] 45315765 pps (45406397 pkts in 1002000 usec) 889.095434 main_thread [1418] 45333218 pps (45378551 pkts in 1001000 usec) 890.097434 main_thread [1418] 45315247 pps (45405877 pkts in 1002000 usec) 891.099434 main_thread [1418] 45326515 pps (45417168 pkts in 1002000 usec) 892.101434 main_thread [1418] 45333039 pps (45423705 pkts in 1002000 usec) 893.103434 main_thread [1418] 45324105 pps (45414708 pkts in 1001999 usec) 894.105434 main_thread [1418] 45318042 pps (45408723 pkts in 1002001 usec) 895.106434 main_thread [1418] 45332430 pps (45377762 pkts in 1001000 usec) 896.107434 main_thread [1418] 45338072 pps (45383410 pkts in 1001000 usec) ... Relnotes: Yes Sponsored by: Chelsio Communications.
2014-05-27 18:18:41 +00:00
if (!(pi->flags & INTR_RXQ)) {
oid = SYSCTL_ADD_NODE(&pi->ctx, children, OID_AUTO, "rxq",
CTLFLAG_RD, NULL, "rx queues");
for_each_rxq(pi, i, rxq) {
MPASS(!(rxq->iq.flags & IQ_INTR));
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
cxgbe(4): netmap support for Terminator 5 (T5) based 10G/40G cards. Netmap gets its own hardware-assisted virtual interface and won't take over or disrupt the "normal" interface in any way. You can use both simultaneously. For kernels with DEV_NETMAP, cxgbe(4) carves out an ncxl<N> interface (note the 'n' prefix) in the hardware to accompany each cxl<N> interface. These two ifnet's per port share the same wire but really are separate interfaces in the hardware and software. Each gets its own L2 MAC addresses (unicast and multicast), MTU, checksum caps, etc. You should run netmap on the 'n' interfaces only, that's what they are for. With this, pkt-gen is able to transmit > 45Mpps out of a single 40G port of a T580 card. 2 port tx is at ~56Mpps total (28M + 28M) as of now. Single port receive is at 33Mpps but this is very much a work in progress. I expect it to be closer to 40Mpps once done. In any case the current effort can already saturate multiple 10G ports of a T5 card at the smallest legal packet size. T4 gear is totally untested. trantor:~# ./pkt-gen -i ncxl0 -f tx -D 00:07:43:ab:cd:ef 881.952141 main [1621] interface is ncxl0 881.952250 extract_ip_range [275] range is 10.0.0.1:0 to 10.0.0.1:0 881.952253 extract_ip_range [275] range is 10.1.0.1:0 to 10.1.0.1:0 881.962540 main [1804] mapped 334980KB at 0x801dff000 Sending on netmap:ncxl0: 4 queues, 1 threads and 1 cpus. 10.0.0.1 -> 10.1.0.1 (00:00:00:00:00:00 -> 00:07:43:ab:cd:ef) 881.962562 main [1882] Sending 512 packets every 0.000000000 s 881.962563 main [1884] Wait 2 secs for phy reset 884.088516 main [1886] Ready... 884.088535 nm_open [457] overriding ifname ncxl0 ringid 0x0 flags 0x1 884.088607 sender_body [996] start 884.093246 sender_body [1064] drop copy 885.090435 main_thread [1418] 45206353 pps (45289533 pkts in 1001840 usec) 886.091600 main_thread [1418] 45322792 pps (45375593 pkts in 1001165 usec) 887.092435 main_thread [1418] 45313992 pps (45351784 pkts in 1000834 usec) 888.094434 main_thread [1418] 45315765 pps (45406397 pkts in 1002000 usec) 889.095434 main_thread [1418] 45333218 pps (45378551 pkts in 1001000 usec) 890.097434 main_thread [1418] 45315247 pps (45405877 pkts in 1002000 usec) 891.099434 main_thread [1418] 45326515 pps (45417168 pkts in 1002000 usec) 892.101434 main_thread [1418] 45333039 pps (45423705 pkts in 1002000 usec) 893.103434 main_thread [1418] 45324105 pps (45414708 pkts in 1001999 usec) 894.105434 main_thread [1418] 45318042 pps (45408723 pkts in 1002001 usec) 895.106434 main_thread [1418] 45332430 pps (45377762 pkts in 1001000 usec) 896.107434 main_thread [1418] 45338072 pps (45383410 pkts in 1001000 usec) ... Relnotes: Yes Sponsored by: Chelsio Communications.
2014-05-27 18:18:41 +00:00
intr_idx = port_intr_iq(pi, j)->abs_id;
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
cxgbe(4): netmap support for Terminator 5 (T5) based 10G/40G cards. Netmap gets its own hardware-assisted virtual interface and won't take over or disrupt the "normal" interface in any way. You can use both simultaneously. For kernels with DEV_NETMAP, cxgbe(4) carves out an ncxl<N> interface (note the 'n' prefix) in the hardware to accompany each cxl<N> interface. These two ifnet's per port share the same wire but really are separate interfaces in the hardware and software. Each gets its own L2 MAC addresses (unicast and multicast), MTU, checksum caps, etc. You should run netmap on the 'n' interfaces only, that's what they are for. With this, pkt-gen is able to transmit > 45Mpps out of a single 40G port of a T580 card. 2 port tx is at ~56Mpps total (28M + 28M) as of now. Single port receive is at 33Mpps but this is very much a work in progress. I expect it to be closer to 40Mpps once done. In any case the current effort can already saturate multiple 10G ports of a T5 card at the smallest legal packet size. T4 gear is totally untested. trantor:~# ./pkt-gen -i ncxl0 -f tx -D 00:07:43:ab:cd:ef 881.952141 main [1621] interface is ncxl0 881.952250 extract_ip_range [275] range is 10.0.0.1:0 to 10.0.0.1:0 881.952253 extract_ip_range [275] range is 10.1.0.1:0 to 10.1.0.1:0 881.962540 main [1804] mapped 334980KB at 0x801dff000 Sending on netmap:ncxl0: 4 queues, 1 threads and 1 cpus. 10.0.0.1 -> 10.1.0.1 (00:00:00:00:00:00 -> 00:07:43:ab:cd:ef) 881.962562 main [1882] Sending 512 packets every 0.000000000 s 881.962563 main [1884] Wait 2 secs for phy reset 884.088516 main [1886] Ready... 884.088535 nm_open [457] overriding ifname ncxl0 ringid 0x0 flags 0x1 884.088607 sender_body [996] start 884.093246 sender_body [1064] drop copy 885.090435 main_thread [1418] 45206353 pps (45289533 pkts in 1001840 usec) 886.091600 main_thread [1418] 45322792 pps (45375593 pkts in 1001165 usec) 887.092435 main_thread [1418] 45313992 pps (45351784 pkts in 1000834 usec) 888.094434 main_thread [1418] 45315765 pps (45406397 pkts in 1002000 usec) 889.095434 main_thread [1418] 45333218 pps (45378551 pkts in 1001000 usec) 890.097434 main_thread [1418] 45315247 pps (45405877 pkts in 1002000 usec) 891.099434 main_thread [1418] 45326515 pps (45417168 pkts in 1002000 usec) 892.101434 main_thread [1418] 45333039 pps (45423705 pkts in 1002000 usec) 893.103434 main_thread [1418] 45324105 pps (45414708 pkts in 1001999 usec) 894.105434 main_thread [1418] 45318042 pps (45408723 pkts in 1002001 usec) 895.106434 main_thread [1418] 45332430 pps (45377762 pkts in 1001000 usec) 896.107434 main_thread [1418] 45338072 pps (45383410 pkts in 1001000 usec) ... Relnotes: Yes Sponsored by: Chelsio Communications.
2014-05-27 18:18:41 +00:00
rc = alloc_rxq(pi, rxq, intr_idx, i, oid);
if (rc != 0)
goto done;
j++;
}
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
}
- Updated TOE support in the kernel. - Stateful TCP offload drivers for Terminator 3 and 4 (T3 and T4) ASICs. These are available as t3_tom and t4_tom modules that augment cxgb(4) and cxgbe(4) respectively. The cxgb/cxgbe drivers continue to work as usual with or without these extra features. - iWARP driver for Terminator 3 ASIC (kernel verbs). T4 iWARP in the works and will follow soon. Build-tested with make universe. 30s overview ============ What interfaces support TCP offload? Look for TOE4 and/or TOE6 in the capabilities of an interface: # ifconfig -m | grep TOE Enable/disable TCP offload on an interface (just like any other ifnet capability): # ifconfig cxgbe0 toe # ifconfig cxgbe0 -toe Which connections are offloaded? Look for toe4 and/or toe6 in the output of netstat and sockstat: # netstat -np tcp | grep toe # sockstat -46c | grep toe Reviewed by: bz, gnn Sponsored by: Chelsio communications. MFC after: ~3 months (after 9.1, and after ensuring MFC is feasible)
2012-06-19 07:34:13 +00:00
#ifdef TCP_OFFLOAD
cxgbe(4): netmap support for Terminator 5 (T5) based 10G/40G cards. Netmap gets its own hardware-assisted virtual interface and won't take over or disrupt the "normal" interface in any way. You can use both simultaneously. For kernels with DEV_NETMAP, cxgbe(4) carves out an ncxl<N> interface (note the 'n' prefix) in the hardware to accompany each cxl<N> interface. These two ifnet's per port share the same wire but really are separate interfaces in the hardware and software. Each gets its own L2 MAC addresses (unicast and multicast), MTU, checksum caps, etc. You should run netmap on the 'n' interfaces only, that's what they are for. With this, pkt-gen is able to transmit > 45Mpps out of a single 40G port of a T580 card. 2 port tx is at ~56Mpps total (28M + 28M) as of now. Single port receive is at 33Mpps but this is very much a work in progress. I expect it to be closer to 40Mpps once done. In any case the current effort can already saturate multiple 10G ports of a T5 card at the smallest legal packet size. T4 gear is totally untested. trantor:~# ./pkt-gen -i ncxl0 -f tx -D 00:07:43:ab:cd:ef 881.952141 main [1621] interface is ncxl0 881.952250 extract_ip_range [275] range is 10.0.0.1:0 to 10.0.0.1:0 881.952253 extract_ip_range [275] range is 10.1.0.1:0 to 10.1.0.1:0 881.962540 main [1804] mapped 334980KB at 0x801dff000 Sending on netmap:ncxl0: 4 queues, 1 threads and 1 cpus. 10.0.0.1 -> 10.1.0.1 (00:00:00:00:00:00 -> 00:07:43:ab:cd:ef) 881.962562 main [1882] Sending 512 packets every 0.000000000 s 881.962563 main [1884] Wait 2 secs for phy reset 884.088516 main [1886] Ready... 884.088535 nm_open [457] overriding ifname ncxl0 ringid 0x0 flags 0x1 884.088607 sender_body [996] start 884.093246 sender_body [1064] drop copy 885.090435 main_thread [1418] 45206353 pps (45289533 pkts in 1001840 usec) 886.091600 main_thread [1418] 45322792 pps (45375593 pkts in 1001165 usec) 887.092435 main_thread [1418] 45313992 pps (45351784 pkts in 1000834 usec) 888.094434 main_thread [1418] 45315765 pps (45406397 pkts in 1002000 usec) 889.095434 main_thread [1418] 45333218 pps (45378551 pkts in 1001000 usec) 890.097434 main_thread [1418] 45315247 pps (45405877 pkts in 1002000 usec) 891.099434 main_thread [1418] 45326515 pps (45417168 pkts in 1002000 usec) 892.101434 main_thread [1418] 45333039 pps (45423705 pkts in 1002000 usec) 893.103434 main_thread [1418] 45324105 pps (45414708 pkts in 1001999 usec) 894.105434 main_thread [1418] 45318042 pps (45408723 pkts in 1002001 usec) 895.106434 main_thread [1418] 45332430 pps (45377762 pkts in 1001000 usec) 896.107434 main_thread [1418] 45338072 pps (45383410 pkts in 1001000 usec) ... Relnotes: Yes Sponsored by: Chelsio Communications.
2014-05-27 18:18:41 +00:00
if (is_offload(sc) && !(pi->flags & INTR_OFLD_RXQ)) {
oid = SYSCTL_ADD_NODE(&pi->ctx, children, OID_AUTO, "ofld_rxq",
CTLFLAG_RD, NULL,
"rx queues for offloaded TCP connections");
for_each_ofld_rxq(pi, i, ofld_rxq) {
MPASS(!(ofld_rxq->iq.flags & IQ_INTR));
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
cxgbe(4): netmap support for Terminator 5 (T5) based 10G/40G cards. Netmap gets its own hardware-assisted virtual interface and won't take over or disrupt the "normal" interface in any way. You can use both simultaneously. For kernels with DEV_NETMAP, cxgbe(4) carves out an ncxl<N> interface (note the 'n' prefix) in the hardware to accompany each cxl<N> interface. These two ifnet's per port share the same wire but really are separate interfaces in the hardware and software. Each gets its own L2 MAC addresses (unicast and multicast), MTU, checksum caps, etc. You should run netmap on the 'n' interfaces only, that's what they are for. With this, pkt-gen is able to transmit > 45Mpps out of a single 40G port of a T580 card. 2 port tx is at ~56Mpps total (28M + 28M) as of now. Single port receive is at 33Mpps but this is very much a work in progress. I expect it to be closer to 40Mpps once done. In any case the current effort can already saturate multiple 10G ports of a T5 card at the smallest legal packet size. T4 gear is totally untested. trantor:~# ./pkt-gen -i ncxl0 -f tx -D 00:07:43:ab:cd:ef 881.952141 main [1621] interface is ncxl0 881.952250 extract_ip_range [275] range is 10.0.0.1:0 to 10.0.0.1:0 881.952253 extract_ip_range [275] range is 10.1.0.1:0 to 10.1.0.1:0 881.962540 main [1804] mapped 334980KB at 0x801dff000 Sending on netmap:ncxl0: 4 queues, 1 threads and 1 cpus. 10.0.0.1 -> 10.1.0.1 (00:00:00:00:00:00 -> 00:07:43:ab:cd:ef) 881.962562 main [1882] Sending 512 packets every 0.000000000 s 881.962563 main [1884] Wait 2 secs for phy reset 884.088516 main [1886] Ready... 884.088535 nm_open [457] overriding ifname ncxl0 ringid 0x0 flags 0x1 884.088607 sender_body [996] start 884.093246 sender_body [1064] drop copy 885.090435 main_thread [1418] 45206353 pps (45289533 pkts in 1001840 usec) 886.091600 main_thread [1418] 45322792 pps (45375593 pkts in 1001165 usec) 887.092435 main_thread [1418] 45313992 pps (45351784 pkts in 1000834 usec) 888.094434 main_thread [1418] 45315765 pps (45406397 pkts in 1002000 usec) 889.095434 main_thread [1418] 45333218 pps (45378551 pkts in 1001000 usec) 890.097434 main_thread [1418] 45315247 pps (45405877 pkts in 1002000 usec) 891.099434 main_thread [1418] 45326515 pps (45417168 pkts in 1002000 usec) 892.101434 main_thread [1418] 45333039 pps (45423705 pkts in 1002000 usec) 893.103434 main_thread [1418] 45324105 pps (45414708 pkts in 1001999 usec) 894.105434 main_thread [1418] 45318042 pps (45408723 pkts in 1002001 usec) 895.106434 main_thread [1418] 45332430 pps (45377762 pkts in 1001000 usec) 896.107434 main_thread [1418] 45338072 pps (45383410 pkts in 1001000 usec) ... Relnotes: Yes Sponsored by: Chelsio Communications.
2014-05-27 18:18:41 +00:00
intr_idx = port_intr_iq(pi, j)->abs_id;
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
cxgbe(4): netmap support for Terminator 5 (T5) based 10G/40G cards. Netmap gets its own hardware-assisted virtual interface and won't take over or disrupt the "normal" interface in any way. You can use both simultaneously. For kernels with DEV_NETMAP, cxgbe(4) carves out an ncxl<N> interface (note the 'n' prefix) in the hardware to accompany each cxl<N> interface. These two ifnet's per port share the same wire but really are separate interfaces in the hardware and software. Each gets its own L2 MAC addresses (unicast and multicast), MTU, checksum caps, etc. You should run netmap on the 'n' interfaces only, that's what they are for. With this, pkt-gen is able to transmit > 45Mpps out of a single 40G port of a T580 card. 2 port tx is at ~56Mpps total (28M + 28M) as of now. Single port receive is at 33Mpps but this is very much a work in progress. I expect it to be closer to 40Mpps once done. In any case the current effort can already saturate multiple 10G ports of a T5 card at the smallest legal packet size. T4 gear is totally untested. trantor:~# ./pkt-gen -i ncxl0 -f tx -D 00:07:43:ab:cd:ef 881.952141 main [1621] interface is ncxl0 881.952250 extract_ip_range [275] range is 10.0.0.1:0 to 10.0.0.1:0 881.952253 extract_ip_range [275] range is 10.1.0.1:0 to 10.1.0.1:0 881.962540 main [1804] mapped 334980KB at 0x801dff000 Sending on netmap:ncxl0: 4 queues, 1 threads and 1 cpus. 10.0.0.1 -> 10.1.0.1 (00:00:00:00:00:00 -> 00:07:43:ab:cd:ef) 881.962562 main [1882] Sending 512 packets every 0.000000000 s 881.962563 main [1884] Wait 2 secs for phy reset 884.088516 main [1886] Ready... 884.088535 nm_open [457] overriding ifname ncxl0 ringid 0x0 flags 0x1 884.088607 sender_body [996] start 884.093246 sender_body [1064] drop copy 885.090435 main_thread [1418] 45206353 pps (45289533 pkts in 1001840 usec) 886.091600 main_thread [1418] 45322792 pps (45375593 pkts in 1001165 usec) 887.092435 main_thread [1418] 45313992 pps (45351784 pkts in 1000834 usec) 888.094434 main_thread [1418] 45315765 pps (45406397 pkts in 1002000 usec) 889.095434 main_thread [1418] 45333218 pps (45378551 pkts in 1001000 usec) 890.097434 main_thread [1418] 45315247 pps (45405877 pkts in 1002000 usec) 891.099434 main_thread [1418] 45326515 pps (45417168 pkts in 1002000 usec) 892.101434 main_thread [1418] 45333039 pps (45423705 pkts in 1002000 usec) 893.103434 main_thread [1418] 45324105 pps (45414708 pkts in 1001999 usec) 894.105434 main_thread [1418] 45318042 pps (45408723 pkts in 1002001 usec) 895.106434 main_thread [1418] 45332430 pps (45377762 pkts in 1001000 usec) 896.107434 main_thread [1418] 45338072 pps (45383410 pkts in 1001000 usec) ... Relnotes: Yes Sponsored by: Chelsio Communications.
2014-05-27 18:18:41 +00:00
rc = alloc_ofld_rxq(pi, ofld_rxq, intr_idx, i, oid);
if (rc != 0)
goto done;
j++;
}
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
}
#endif
cxgbe(4): netmap support for Terminator 5 (T5) based 10G/40G cards. Netmap gets its own hardware-assisted virtual interface and won't take over or disrupt the "normal" interface in any way. You can use both simultaneously. For kernels with DEV_NETMAP, cxgbe(4) carves out an ncxl<N> interface (note the 'n' prefix) in the hardware to accompany each cxl<N> interface. These two ifnet's per port share the same wire but really are separate interfaces in the hardware and software. Each gets its own L2 MAC addresses (unicast and multicast), MTU, checksum caps, etc. You should run netmap on the 'n' interfaces only, that's what they are for. With this, pkt-gen is able to transmit > 45Mpps out of a single 40G port of a T580 card. 2 port tx is at ~56Mpps total (28M + 28M) as of now. Single port receive is at 33Mpps but this is very much a work in progress. I expect it to be closer to 40Mpps once done. In any case the current effort can already saturate multiple 10G ports of a T5 card at the smallest legal packet size. T4 gear is totally untested. trantor:~# ./pkt-gen -i ncxl0 -f tx -D 00:07:43:ab:cd:ef 881.952141 main [1621] interface is ncxl0 881.952250 extract_ip_range [275] range is 10.0.0.1:0 to 10.0.0.1:0 881.952253 extract_ip_range [275] range is 10.1.0.1:0 to 10.1.0.1:0 881.962540 main [1804] mapped 334980KB at 0x801dff000 Sending on netmap:ncxl0: 4 queues, 1 threads and 1 cpus. 10.0.0.1 -> 10.1.0.1 (00:00:00:00:00:00 -> 00:07:43:ab:cd:ef) 881.962562 main [1882] Sending 512 packets every 0.000000000 s 881.962563 main [1884] Wait 2 secs for phy reset 884.088516 main [1886] Ready... 884.088535 nm_open [457] overriding ifname ncxl0 ringid 0x0 flags 0x1 884.088607 sender_body [996] start 884.093246 sender_body [1064] drop copy 885.090435 main_thread [1418] 45206353 pps (45289533 pkts in 1001840 usec) 886.091600 main_thread [1418] 45322792 pps (45375593 pkts in 1001165 usec) 887.092435 main_thread [1418] 45313992 pps (45351784 pkts in 1000834 usec) 888.094434 main_thread [1418] 45315765 pps (45406397 pkts in 1002000 usec) 889.095434 main_thread [1418] 45333218 pps (45378551 pkts in 1001000 usec) 890.097434 main_thread [1418] 45315247 pps (45405877 pkts in 1002000 usec) 891.099434 main_thread [1418] 45326515 pps (45417168 pkts in 1002000 usec) 892.101434 main_thread [1418] 45333039 pps (45423705 pkts in 1002000 usec) 893.103434 main_thread [1418] 45324105 pps (45414708 pkts in 1001999 usec) 894.105434 main_thread [1418] 45318042 pps (45408723 pkts in 1002001 usec) 895.106434 main_thread [1418] 45332430 pps (45377762 pkts in 1001000 usec) 896.107434 main_thread [1418] 45338072 pps (45383410 pkts in 1001000 usec) ... Relnotes: Yes Sponsored by: Chelsio Communications.
2014-05-27 18:18:41 +00:00
#ifdef DEV_NETMAP
if (!(pi->flags & INTR_NM_RXQ))
CXGBE_UNIMPLEMENTED(__func__);
#endif
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
/*
* Now the tx queues. Only one pass needed.
*/
oid = SYSCTL_ADD_NODE(&pi->ctx, children, OID_AUTO, "txq", CTLFLAG_RD,
NULL, "tx queues");
j = 0;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
for_each_txq(pi, i, txq) {
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
iqid = port_intr_iq(pi, j)->cntxt_id;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
snprintf(name, sizeof(name), "%s txq%d",
device_get_nameunit(pi->dev), i);
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
init_eq(&txq->eq, EQ_ETH, pi->qsize_txq, pi->tx_chan, iqid,
name);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
rc = alloc_txq(pi, txq, i, oid);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
if (rc != 0)
goto done;
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
j++;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
}
- Updated TOE support in the kernel. - Stateful TCP offload drivers for Terminator 3 and 4 (T3 and T4) ASICs. These are available as t3_tom and t4_tom modules that augment cxgb(4) and cxgbe(4) respectively. The cxgb/cxgbe drivers continue to work as usual with or without these extra features. - iWARP driver for Terminator 3 ASIC (kernel verbs). T4 iWARP in the works and will follow soon. Build-tested with make universe. 30s overview ============ What interfaces support TCP offload? Look for TOE4 and/or TOE6 in the capabilities of an interface: # ifconfig -m | grep TOE Enable/disable TCP offload on an interface (just like any other ifnet capability): # ifconfig cxgbe0 toe # ifconfig cxgbe0 -toe Which connections are offloaded? Look for toe4 and/or toe6 in the output of netstat and sockstat: # netstat -np tcp | grep toe # sockstat -46c | grep toe Reviewed by: bz, gnn Sponsored by: Chelsio communications. MFC after: ~3 months (after 9.1, and after ensuring MFC is feasible)
2012-06-19 07:34:13 +00:00
#ifdef TCP_OFFLOAD
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
oid = SYSCTL_ADD_NODE(&pi->ctx, children, OID_AUTO, "ofld_txq",
CTLFLAG_RD, NULL, "tx queues for offloaded TCP connections");
for_each_ofld_txq(pi, i, ofld_txq) {
cxgbe(4): netmap support for Terminator 5 (T5) based 10G/40G cards. Netmap gets its own hardware-assisted virtual interface and won't take over or disrupt the "normal" interface in any way. You can use both simultaneously. For kernels with DEV_NETMAP, cxgbe(4) carves out an ncxl<N> interface (note the 'n' prefix) in the hardware to accompany each cxl<N> interface. These two ifnet's per port share the same wire but really are separate interfaces in the hardware and software. Each gets its own L2 MAC addresses (unicast and multicast), MTU, checksum caps, etc. You should run netmap on the 'n' interfaces only, that's what they are for. With this, pkt-gen is able to transmit > 45Mpps out of a single 40G port of a T580 card. 2 port tx is at ~56Mpps total (28M + 28M) as of now. Single port receive is at 33Mpps but this is very much a work in progress. I expect it to be closer to 40Mpps once done. In any case the current effort can already saturate multiple 10G ports of a T5 card at the smallest legal packet size. T4 gear is totally untested. trantor:~# ./pkt-gen -i ncxl0 -f tx -D 00:07:43:ab:cd:ef 881.952141 main [1621] interface is ncxl0 881.952250 extract_ip_range [275] range is 10.0.0.1:0 to 10.0.0.1:0 881.952253 extract_ip_range [275] range is 10.1.0.1:0 to 10.1.0.1:0 881.962540 main [1804] mapped 334980KB at 0x801dff000 Sending on netmap:ncxl0: 4 queues, 1 threads and 1 cpus. 10.0.0.1 -> 10.1.0.1 (00:00:00:00:00:00 -> 00:07:43:ab:cd:ef) 881.962562 main [1882] Sending 512 packets every 0.000000000 s 881.962563 main [1884] Wait 2 secs for phy reset 884.088516 main [1886] Ready... 884.088535 nm_open [457] overriding ifname ncxl0 ringid 0x0 flags 0x1 884.088607 sender_body [996] start 884.093246 sender_body [1064] drop copy 885.090435 main_thread [1418] 45206353 pps (45289533 pkts in 1001840 usec) 886.091600 main_thread [1418] 45322792 pps (45375593 pkts in 1001165 usec) 887.092435 main_thread [1418] 45313992 pps (45351784 pkts in 1000834 usec) 888.094434 main_thread [1418] 45315765 pps (45406397 pkts in 1002000 usec) 889.095434 main_thread [1418] 45333218 pps (45378551 pkts in 1001000 usec) 890.097434 main_thread [1418] 45315247 pps (45405877 pkts in 1002000 usec) 891.099434 main_thread [1418] 45326515 pps (45417168 pkts in 1002000 usec) 892.101434 main_thread [1418] 45333039 pps (45423705 pkts in 1002000 usec) 893.103434 main_thread [1418] 45324105 pps (45414708 pkts in 1001999 usec) 894.105434 main_thread [1418] 45318042 pps (45408723 pkts in 1002001 usec) 895.106434 main_thread [1418] 45332430 pps (45377762 pkts in 1001000 usec) 896.107434 main_thread [1418] 45338072 pps (45383410 pkts in 1001000 usec) ... Relnotes: Yes Sponsored by: Chelsio Communications.
2014-05-27 18:18:41 +00:00
struct sysctl_oid *oid2;
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
iqid = port_intr_iq(pi, j)->cntxt_id;
snprintf(name, sizeof(name), "%s ofld_txq%d",
device_get_nameunit(pi->dev), i);
init_eq(&ofld_txq->eq, EQ_OFLD, pi->qsize_txq, pi->tx_chan,
iqid, name);
snprintf(name, sizeof(name), "%d", i);
oid2 = SYSCTL_ADD_NODE(&pi->ctx, SYSCTL_CHILDREN(oid), OID_AUTO,
name, CTLFLAG_RD, NULL, "offload tx queue");
rc = alloc_wrq(sc, pi, ofld_txq, oid2);
if (rc != 0)
goto done;
j++;
}
#endif
cxgbe(4): netmap support for Terminator 5 (T5) based 10G/40G cards. Netmap gets its own hardware-assisted virtual interface and won't take over or disrupt the "normal" interface in any way. You can use both simultaneously. For kernels with DEV_NETMAP, cxgbe(4) carves out an ncxl<N> interface (note the 'n' prefix) in the hardware to accompany each cxl<N> interface. These two ifnet's per port share the same wire but really are separate interfaces in the hardware and software. Each gets its own L2 MAC addresses (unicast and multicast), MTU, checksum caps, etc. You should run netmap on the 'n' interfaces only, that's what they are for. With this, pkt-gen is able to transmit > 45Mpps out of a single 40G port of a T580 card. 2 port tx is at ~56Mpps total (28M + 28M) as of now. Single port receive is at 33Mpps but this is very much a work in progress. I expect it to be closer to 40Mpps once done. In any case the current effort can already saturate multiple 10G ports of a T5 card at the smallest legal packet size. T4 gear is totally untested. trantor:~# ./pkt-gen -i ncxl0 -f tx -D 00:07:43:ab:cd:ef 881.952141 main [1621] interface is ncxl0 881.952250 extract_ip_range [275] range is 10.0.0.1:0 to 10.0.0.1:0 881.952253 extract_ip_range [275] range is 10.1.0.1:0 to 10.1.0.1:0 881.962540 main [1804] mapped 334980KB at 0x801dff000 Sending on netmap:ncxl0: 4 queues, 1 threads and 1 cpus. 10.0.0.1 -> 10.1.0.1 (00:00:00:00:00:00 -> 00:07:43:ab:cd:ef) 881.962562 main [1882] Sending 512 packets every 0.000000000 s 881.962563 main [1884] Wait 2 secs for phy reset 884.088516 main [1886] Ready... 884.088535 nm_open [457] overriding ifname ncxl0 ringid 0x0 flags 0x1 884.088607 sender_body [996] start 884.093246 sender_body [1064] drop copy 885.090435 main_thread [1418] 45206353 pps (45289533 pkts in 1001840 usec) 886.091600 main_thread [1418] 45322792 pps (45375593 pkts in 1001165 usec) 887.092435 main_thread [1418] 45313992 pps (45351784 pkts in 1000834 usec) 888.094434 main_thread [1418] 45315765 pps (45406397 pkts in 1002000 usec) 889.095434 main_thread [1418] 45333218 pps (45378551 pkts in 1001000 usec) 890.097434 main_thread [1418] 45315247 pps (45405877 pkts in 1002000 usec) 891.099434 main_thread [1418] 45326515 pps (45417168 pkts in 1002000 usec) 892.101434 main_thread [1418] 45333039 pps (45423705 pkts in 1002000 usec) 893.103434 main_thread [1418] 45324105 pps (45414708 pkts in 1001999 usec) 894.105434 main_thread [1418] 45318042 pps (45408723 pkts in 1002001 usec) 895.106434 main_thread [1418] 45332430 pps (45377762 pkts in 1001000 usec) 896.107434 main_thread [1418] 45338072 pps (45383410 pkts in 1001000 usec) ... Relnotes: Yes Sponsored by: Chelsio Communications.
2014-05-27 18:18:41 +00:00
#ifdef DEV_NETMAP
oid = SYSCTL_ADD_NODE(&pi->ctx, children, OID_AUTO, "nm_txq",
CTLFLAG_RD, NULL, "tx queues for netmap use");
for_each_nm_txq(pi, i, nm_txq) {
iqid = pi->first_nm_rxq + (j % pi->nnmrxq);
rc = alloc_nm_txq(pi, nm_txq, iqid, i, oid);
if (rc != 0)
goto done;
j++;
}
#endif
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
/*
* Finally, the control queue.
*/
oid = SYSCTL_ADD_NODE(&pi->ctx, children, OID_AUTO, "ctrlq", CTLFLAG_RD,
NULL, "ctrl queue");
ctrlq = &sc->sge.ctrlq[pi->port_id];
iqid = port_intr_iq(pi, 0)->cntxt_id;
snprintf(name, sizeof(name), "%s ctrlq", device_get_nameunit(pi->dev));
init_eq(&ctrlq->eq, EQ_CTRL, CTRL_EQ_QSIZE, pi->tx_chan, iqid, name);
rc = alloc_wrq(sc, pi, ctrlq, oid);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
done:
if (rc)
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
t4_teardown_port_queues(pi);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
return (rc);
}
/*
* Idempotent
*/
int
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
t4_teardown_port_queues(struct port_info *pi)
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
{
int i;
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
struct adapter *sc = pi->adapter;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
struct sge_rxq *rxq;
struct sge_txq *txq;
- Updated TOE support in the kernel. - Stateful TCP offload drivers for Terminator 3 and 4 (T3 and T4) ASICs. These are available as t3_tom and t4_tom modules that augment cxgb(4) and cxgbe(4) respectively. The cxgb/cxgbe drivers continue to work as usual with or without these extra features. - iWARP driver for Terminator 3 ASIC (kernel verbs). T4 iWARP in the works and will follow soon. Build-tested with make universe. 30s overview ============ What interfaces support TCP offload? Look for TOE4 and/or TOE6 in the capabilities of an interface: # ifconfig -m | grep TOE Enable/disable TCP offload on an interface (just like any other ifnet capability): # ifconfig cxgbe0 toe # ifconfig cxgbe0 -toe Which connections are offloaded? Look for toe4 and/or toe6 in the output of netstat and sockstat: # netstat -np tcp | grep toe # sockstat -46c | grep toe Reviewed by: bz, gnn Sponsored by: Chelsio communications. MFC after: ~3 months (after 9.1, and after ensuring MFC is feasible)
2012-06-19 07:34:13 +00:00
#ifdef TCP_OFFLOAD
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
struct sge_ofld_rxq *ofld_rxq;
struct sge_wrq *ofld_txq;
#endif
cxgbe(4): netmap support for Terminator 5 (T5) based 10G/40G cards. Netmap gets its own hardware-assisted virtual interface and won't take over or disrupt the "normal" interface in any way. You can use both simultaneously. For kernels with DEV_NETMAP, cxgbe(4) carves out an ncxl<N> interface (note the 'n' prefix) in the hardware to accompany each cxl<N> interface. These two ifnet's per port share the same wire but really are separate interfaces in the hardware and software. Each gets its own L2 MAC addresses (unicast and multicast), MTU, checksum caps, etc. You should run netmap on the 'n' interfaces only, that's what they are for. With this, pkt-gen is able to transmit > 45Mpps out of a single 40G port of a T580 card. 2 port tx is at ~56Mpps total (28M + 28M) as of now. Single port receive is at 33Mpps but this is very much a work in progress. I expect it to be closer to 40Mpps once done. In any case the current effort can already saturate multiple 10G ports of a T5 card at the smallest legal packet size. T4 gear is totally untested. trantor:~# ./pkt-gen -i ncxl0 -f tx -D 00:07:43:ab:cd:ef 881.952141 main [1621] interface is ncxl0 881.952250 extract_ip_range [275] range is 10.0.0.1:0 to 10.0.0.1:0 881.952253 extract_ip_range [275] range is 10.1.0.1:0 to 10.1.0.1:0 881.962540 main [1804] mapped 334980KB at 0x801dff000 Sending on netmap:ncxl0: 4 queues, 1 threads and 1 cpus. 10.0.0.1 -> 10.1.0.1 (00:00:00:00:00:00 -> 00:07:43:ab:cd:ef) 881.962562 main [1882] Sending 512 packets every 0.000000000 s 881.962563 main [1884] Wait 2 secs for phy reset 884.088516 main [1886] Ready... 884.088535 nm_open [457] overriding ifname ncxl0 ringid 0x0 flags 0x1 884.088607 sender_body [996] start 884.093246 sender_body [1064] drop copy 885.090435 main_thread [1418] 45206353 pps (45289533 pkts in 1001840 usec) 886.091600 main_thread [1418] 45322792 pps (45375593 pkts in 1001165 usec) 887.092435 main_thread [1418] 45313992 pps (45351784 pkts in 1000834 usec) 888.094434 main_thread [1418] 45315765 pps (45406397 pkts in 1002000 usec) 889.095434 main_thread [1418] 45333218 pps (45378551 pkts in 1001000 usec) 890.097434 main_thread [1418] 45315247 pps (45405877 pkts in 1002000 usec) 891.099434 main_thread [1418] 45326515 pps (45417168 pkts in 1002000 usec) 892.101434 main_thread [1418] 45333039 pps (45423705 pkts in 1002000 usec) 893.103434 main_thread [1418] 45324105 pps (45414708 pkts in 1001999 usec) 894.105434 main_thread [1418] 45318042 pps (45408723 pkts in 1002001 usec) 895.106434 main_thread [1418] 45332430 pps (45377762 pkts in 1001000 usec) 896.107434 main_thread [1418] 45338072 pps (45383410 pkts in 1001000 usec) ... Relnotes: Yes Sponsored by: Chelsio Communications.
2014-05-27 18:18:41 +00:00
#ifdef DEV_NETMAP
struct sge_nm_rxq *nm_rxq;
struct sge_nm_txq *nm_txq;
#endif
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
/* Do this before freeing the queues */
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
if (pi->flags & PORT_SYSCTL_CTX) {
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
sysctl_ctx_free(&pi->ctx);
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
pi->flags &= ~PORT_SYSCTL_CTX;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
}
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
/*
* Take down all the tx queues first, as they reference the rx queues
* (for egress updates, etc.).
*/
free_wrq(sc, &sc->sge.ctrlq[pi->port_id]);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
for_each_txq(pi, i, txq) {
free_txq(pi, txq);
}
- Updated TOE support in the kernel. - Stateful TCP offload drivers for Terminator 3 and 4 (T3 and T4) ASICs. These are available as t3_tom and t4_tom modules that augment cxgb(4) and cxgbe(4) respectively. The cxgb/cxgbe drivers continue to work as usual with or without these extra features. - iWARP driver for Terminator 3 ASIC (kernel verbs). T4 iWARP in the works and will follow soon. Build-tested with make universe. 30s overview ============ What interfaces support TCP offload? Look for TOE4 and/or TOE6 in the capabilities of an interface: # ifconfig -m | grep TOE Enable/disable TCP offload on an interface (just like any other ifnet capability): # ifconfig cxgbe0 toe # ifconfig cxgbe0 -toe Which connections are offloaded? Look for toe4 and/or toe6 in the output of netstat and sockstat: # netstat -np tcp | grep toe # sockstat -46c | grep toe Reviewed by: bz, gnn Sponsored by: Chelsio communications. MFC after: ~3 months (after 9.1, and after ensuring MFC is feasible)
2012-06-19 07:34:13 +00:00
#ifdef TCP_OFFLOAD
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
for_each_ofld_txq(pi, i, ofld_txq) {
free_wrq(sc, ofld_txq);
}
#endif
cxgbe(4): netmap support for Terminator 5 (T5) based 10G/40G cards. Netmap gets its own hardware-assisted virtual interface and won't take over or disrupt the "normal" interface in any way. You can use both simultaneously. For kernels with DEV_NETMAP, cxgbe(4) carves out an ncxl<N> interface (note the 'n' prefix) in the hardware to accompany each cxl<N> interface. These two ifnet's per port share the same wire but really are separate interfaces in the hardware and software. Each gets its own L2 MAC addresses (unicast and multicast), MTU, checksum caps, etc. You should run netmap on the 'n' interfaces only, that's what they are for. With this, pkt-gen is able to transmit > 45Mpps out of a single 40G port of a T580 card. 2 port tx is at ~56Mpps total (28M + 28M) as of now. Single port receive is at 33Mpps but this is very much a work in progress. I expect it to be closer to 40Mpps once done. In any case the current effort can already saturate multiple 10G ports of a T5 card at the smallest legal packet size. T4 gear is totally untested. trantor:~# ./pkt-gen -i ncxl0 -f tx -D 00:07:43:ab:cd:ef 881.952141 main [1621] interface is ncxl0 881.952250 extract_ip_range [275] range is 10.0.0.1:0 to 10.0.0.1:0 881.952253 extract_ip_range [275] range is 10.1.0.1:0 to 10.1.0.1:0 881.962540 main [1804] mapped 334980KB at 0x801dff000 Sending on netmap:ncxl0: 4 queues, 1 threads and 1 cpus. 10.0.0.1 -> 10.1.0.1 (00:00:00:00:00:00 -> 00:07:43:ab:cd:ef) 881.962562 main [1882] Sending 512 packets every 0.000000000 s 881.962563 main [1884] Wait 2 secs for phy reset 884.088516 main [1886] Ready... 884.088535 nm_open [457] overriding ifname ncxl0 ringid 0x0 flags 0x1 884.088607 sender_body [996] start 884.093246 sender_body [1064] drop copy 885.090435 main_thread [1418] 45206353 pps (45289533 pkts in 1001840 usec) 886.091600 main_thread [1418] 45322792 pps (45375593 pkts in 1001165 usec) 887.092435 main_thread [1418] 45313992 pps (45351784 pkts in 1000834 usec) 888.094434 main_thread [1418] 45315765 pps (45406397 pkts in 1002000 usec) 889.095434 main_thread [1418] 45333218 pps (45378551 pkts in 1001000 usec) 890.097434 main_thread [1418] 45315247 pps (45405877 pkts in 1002000 usec) 891.099434 main_thread [1418] 45326515 pps (45417168 pkts in 1002000 usec) 892.101434 main_thread [1418] 45333039 pps (45423705 pkts in 1002000 usec) 893.103434 main_thread [1418] 45324105 pps (45414708 pkts in 1001999 usec) 894.105434 main_thread [1418] 45318042 pps (45408723 pkts in 1002001 usec) 895.106434 main_thread [1418] 45332430 pps (45377762 pkts in 1001000 usec) 896.107434 main_thread [1418] 45338072 pps (45383410 pkts in 1001000 usec) ... Relnotes: Yes Sponsored by: Chelsio Communications.
2014-05-27 18:18:41 +00:00
#ifdef DEV_NETMAP
for_each_nm_txq(pi, i, nm_txq)
free_nm_txq(pi, nm_txq);
#endif
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
/*
* Then take down the rx queues that forward their interrupts, as they
* reference other rx queues.
*/
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
for_each_rxq(pi, i, rxq) {
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
if ((rxq->iq.flags & IQ_INTR) == 0)
free_rxq(pi, rxq);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
}
- Updated TOE support in the kernel. - Stateful TCP offload drivers for Terminator 3 and 4 (T3 and T4) ASICs. These are available as t3_tom and t4_tom modules that augment cxgb(4) and cxgbe(4) respectively. The cxgb/cxgbe drivers continue to work as usual with or without these extra features. - iWARP driver for Terminator 3 ASIC (kernel verbs). T4 iWARP in the works and will follow soon. Build-tested with make universe. 30s overview ============ What interfaces support TCP offload? Look for TOE4 and/or TOE6 in the capabilities of an interface: # ifconfig -m | grep TOE Enable/disable TCP offload on an interface (just like any other ifnet capability): # ifconfig cxgbe0 toe # ifconfig cxgbe0 -toe Which connections are offloaded? Look for toe4 and/or toe6 in the output of netstat and sockstat: # netstat -np tcp | grep toe # sockstat -46c | grep toe Reviewed by: bz, gnn Sponsored by: Chelsio communications. MFC after: ~3 months (after 9.1, and after ensuring MFC is feasible)
2012-06-19 07:34:13 +00:00
#ifdef TCP_OFFLOAD
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
for_each_ofld_rxq(pi, i, ofld_rxq) {
if ((ofld_rxq->iq.flags & IQ_INTR) == 0)
free_ofld_rxq(pi, ofld_rxq);
}
#endif
cxgbe(4): netmap support for Terminator 5 (T5) based 10G/40G cards. Netmap gets its own hardware-assisted virtual interface and won't take over or disrupt the "normal" interface in any way. You can use both simultaneously. For kernels with DEV_NETMAP, cxgbe(4) carves out an ncxl<N> interface (note the 'n' prefix) in the hardware to accompany each cxl<N> interface. These two ifnet's per port share the same wire but really are separate interfaces in the hardware and software. Each gets its own L2 MAC addresses (unicast and multicast), MTU, checksum caps, etc. You should run netmap on the 'n' interfaces only, that's what they are for. With this, pkt-gen is able to transmit > 45Mpps out of a single 40G port of a T580 card. 2 port tx is at ~56Mpps total (28M + 28M) as of now. Single port receive is at 33Mpps but this is very much a work in progress. I expect it to be closer to 40Mpps once done. In any case the current effort can already saturate multiple 10G ports of a T5 card at the smallest legal packet size. T4 gear is totally untested. trantor:~# ./pkt-gen -i ncxl0 -f tx -D 00:07:43:ab:cd:ef 881.952141 main [1621] interface is ncxl0 881.952250 extract_ip_range [275] range is 10.0.0.1:0 to 10.0.0.1:0 881.952253 extract_ip_range [275] range is 10.1.0.1:0 to 10.1.0.1:0 881.962540 main [1804] mapped 334980KB at 0x801dff000 Sending on netmap:ncxl0: 4 queues, 1 threads and 1 cpus. 10.0.0.1 -> 10.1.0.1 (00:00:00:00:00:00 -> 00:07:43:ab:cd:ef) 881.962562 main [1882] Sending 512 packets every 0.000000000 s 881.962563 main [1884] Wait 2 secs for phy reset 884.088516 main [1886] Ready... 884.088535 nm_open [457] overriding ifname ncxl0 ringid 0x0 flags 0x1 884.088607 sender_body [996] start 884.093246 sender_body [1064] drop copy 885.090435 main_thread [1418] 45206353 pps (45289533 pkts in 1001840 usec) 886.091600 main_thread [1418] 45322792 pps (45375593 pkts in 1001165 usec) 887.092435 main_thread [1418] 45313992 pps (45351784 pkts in 1000834 usec) 888.094434 main_thread [1418] 45315765 pps (45406397 pkts in 1002000 usec) 889.095434 main_thread [1418] 45333218 pps (45378551 pkts in 1001000 usec) 890.097434 main_thread [1418] 45315247 pps (45405877 pkts in 1002000 usec) 891.099434 main_thread [1418] 45326515 pps (45417168 pkts in 1002000 usec) 892.101434 main_thread [1418] 45333039 pps (45423705 pkts in 1002000 usec) 893.103434 main_thread [1418] 45324105 pps (45414708 pkts in 1001999 usec) 894.105434 main_thread [1418] 45318042 pps (45408723 pkts in 1002001 usec) 895.106434 main_thread [1418] 45332430 pps (45377762 pkts in 1001000 usec) 896.107434 main_thread [1418] 45338072 pps (45383410 pkts in 1001000 usec) ... Relnotes: Yes Sponsored by: Chelsio Communications.
2014-05-27 18:18:41 +00:00
#ifdef DEV_NETMAP
for_each_nm_rxq(pi, i, nm_rxq)
free_nm_rxq(pi, nm_rxq);
#endif
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
/*
* Then take down the rx queues that take direct interrupts.
*/
for_each_rxq(pi, i, rxq) {
if (rxq->iq.flags & IQ_INTR)
free_rxq(pi, rxq);
}
- Updated TOE support in the kernel. - Stateful TCP offload drivers for Terminator 3 and 4 (T3 and T4) ASICs. These are available as t3_tom and t4_tom modules that augment cxgb(4) and cxgbe(4) respectively. The cxgb/cxgbe drivers continue to work as usual with or without these extra features. - iWARP driver for Terminator 3 ASIC (kernel verbs). T4 iWARP in the works and will follow soon. Build-tested with make universe. 30s overview ============ What interfaces support TCP offload? Look for TOE4 and/or TOE6 in the capabilities of an interface: # ifconfig -m | grep TOE Enable/disable TCP offload on an interface (just like any other ifnet capability): # ifconfig cxgbe0 toe # ifconfig cxgbe0 -toe Which connections are offloaded? Look for toe4 and/or toe6 in the output of netstat and sockstat: # netstat -np tcp | grep toe # sockstat -46c | grep toe Reviewed by: bz, gnn Sponsored by: Chelsio communications. MFC after: ~3 months (after 9.1, and after ensuring MFC is feasible)
2012-06-19 07:34:13 +00:00
#ifdef TCP_OFFLOAD
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
for_each_ofld_rxq(pi, i, ofld_rxq) {
if (ofld_rxq->iq.flags & IQ_INTR)
free_ofld_rxq(pi, ofld_rxq);
}
#endif
cxgbe(4): netmap support for Terminator 5 (T5) based 10G/40G cards. Netmap gets its own hardware-assisted virtual interface and won't take over or disrupt the "normal" interface in any way. You can use both simultaneously. For kernels with DEV_NETMAP, cxgbe(4) carves out an ncxl<N> interface (note the 'n' prefix) in the hardware to accompany each cxl<N> interface. These two ifnet's per port share the same wire but really are separate interfaces in the hardware and software. Each gets its own L2 MAC addresses (unicast and multicast), MTU, checksum caps, etc. You should run netmap on the 'n' interfaces only, that's what they are for. With this, pkt-gen is able to transmit > 45Mpps out of a single 40G port of a T580 card. 2 port tx is at ~56Mpps total (28M + 28M) as of now. Single port receive is at 33Mpps but this is very much a work in progress. I expect it to be closer to 40Mpps once done. In any case the current effort can already saturate multiple 10G ports of a T5 card at the smallest legal packet size. T4 gear is totally untested. trantor:~# ./pkt-gen -i ncxl0 -f tx -D 00:07:43:ab:cd:ef 881.952141 main [1621] interface is ncxl0 881.952250 extract_ip_range [275] range is 10.0.0.1:0 to 10.0.0.1:0 881.952253 extract_ip_range [275] range is 10.1.0.1:0 to 10.1.0.1:0 881.962540 main [1804] mapped 334980KB at 0x801dff000 Sending on netmap:ncxl0: 4 queues, 1 threads and 1 cpus. 10.0.0.1 -> 10.1.0.1 (00:00:00:00:00:00 -> 00:07:43:ab:cd:ef) 881.962562 main [1882] Sending 512 packets every 0.000000000 s 881.962563 main [1884] Wait 2 secs for phy reset 884.088516 main [1886] Ready... 884.088535 nm_open [457] overriding ifname ncxl0 ringid 0x0 flags 0x1 884.088607 sender_body [996] start 884.093246 sender_body [1064] drop copy 885.090435 main_thread [1418] 45206353 pps (45289533 pkts in 1001840 usec) 886.091600 main_thread [1418] 45322792 pps (45375593 pkts in 1001165 usec) 887.092435 main_thread [1418] 45313992 pps (45351784 pkts in 1000834 usec) 888.094434 main_thread [1418] 45315765 pps (45406397 pkts in 1002000 usec) 889.095434 main_thread [1418] 45333218 pps (45378551 pkts in 1001000 usec) 890.097434 main_thread [1418] 45315247 pps (45405877 pkts in 1002000 usec) 891.099434 main_thread [1418] 45326515 pps (45417168 pkts in 1002000 usec) 892.101434 main_thread [1418] 45333039 pps (45423705 pkts in 1002000 usec) 893.103434 main_thread [1418] 45324105 pps (45414708 pkts in 1001999 usec) 894.105434 main_thread [1418] 45318042 pps (45408723 pkts in 1002001 usec) 895.106434 main_thread [1418] 45332430 pps (45377762 pkts in 1001000 usec) 896.107434 main_thread [1418] 45338072 pps (45383410 pkts in 1001000 usec) ... Relnotes: Yes Sponsored by: Chelsio Communications.
2014-05-27 18:18:41 +00:00
#ifdef DEV_NETMAP
CXGBE_UNIMPLEMENTED(__func__);
#endif
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
return (0);
}
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
/*
* Deals with errors and the firmware event queue. All data rx queues forward
* their interrupt to the firmware event queue.
*/
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
void
t4_intr_all(void *arg)
{
struct adapter *sc = arg;
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
struct sge_iq *fwq = &sc->sge.fwq;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
t4_intr_err(arg);
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
if (atomic_cmpset_int(&fwq->state, IQS_IDLE, IQS_BUSY)) {
service_iq(fwq, 0);
atomic_cmpset_int(&fwq->state, IQS_BUSY, IQS_IDLE);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
}
}
/* Deals with error interrupts */
void
t4_intr_err(void *arg)
{
struct adapter *sc = arg;
- Specialized ingress queues that take interrupts for other ingress queues. Try to have a set of these per port when possible, fall back to sharing a common pool between all ports otherwise. - One control queue per port (used to be one per hardware channel). - t4_eth_rx now handles Ethernet rx only. - sysctls to display pidx/cidx for some queues. MFC after: 1 week
2011-05-30 21:34:44 +00:00
t4_write_reg(sc, MYPF_REG(A_PCIE_PF_CLI), 0);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
t4_slow_intr_handler(sc);
}
void
t4_intr_evt(void *arg)
Fix a couple of bad races that can occur when a cxgbe interface is taken down. The ingress queue lock was unused and has been removed as part of these changes. - An in-flight egress update from the SGE must be handled before the queue that requested it is destroyed. Wait for the update to arrive. - Interrupt handlers must stop processing rx events for a queue before the queue is destroyed. Events that have not yet been processed should be ignored once the queue disappears. MFC after: 1 week
2011-04-15 03:09:27 +00:00
{
struct sge_iq *iq = arg;
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
if (atomic_cmpset_int(&iq->state, IQS_IDLE, IQS_BUSY)) {
service_iq(iq, 0);
atomic_cmpset_int(&iq->state, IQS_BUSY, IQS_IDLE);
- Specialized ingress queues that take interrupts for other ingress queues. Try to have a set of these per port when possible, fall back to sharing a common pool between all ports otherwise. - One control queue per port (used to be one per hardware channel). - t4_eth_rx now handles Ethernet rx only. - sysctls to display pidx/cidx for some queues. MFC after: 1 week
2011-05-30 21:34:44 +00:00
}
Fix a couple of bad races that can occur when a cxgbe interface is taken down. The ingress queue lock was unused and has been removed as part of these changes. - An in-flight egress update from the SGE must be handled before the queue that requested it is destroyed. Wait for the update to arrive. - Interrupt handlers must stop processing rx events for a queue before the queue is destroyed. Events that have not yet been processed should be ignored once the queue disappears. MFC after: 1 week
2011-04-15 03:09:27 +00:00
}
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
void
t4_intr(void *arg)
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
{
struct sge_iq *iq = arg;
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
if (atomic_cmpset_int(&iq->state, IQS_IDLE, IQS_BUSY)) {
service_iq(iq, 0);
atomic_cmpset_int(&iq->state, IQS_BUSY, IQS_IDLE);
}
}
/*
* Deals with anything and everything on the given ingress queue.
*/
static int
service_iq(struct sge_iq *iq, int budget)
{
struct sge_iq *q;
- Updated TOE support in the kernel. - Stateful TCP offload drivers for Terminator 3 and 4 (T3 and T4) ASICs. These are available as t3_tom and t4_tom modules that augment cxgb(4) and cxgbe(4) respectively. The cxgb/cxgbe drivers continue to work as usual with or without these extra features. - iWARP driver for Terminator 3 ASIC (kernel verbs). T4 iWARP in the works and will follow soon. Build-tested with make universe. 30s overview ============ What interfaces support TCP offload? Look for TOE4 and/or TOE6 in the capabilities of an interface: # ifconfig -m | grep TOE Enable/disable TCP offload on an interface (just like any other ifnet capability): # ifconfig cxgbe0 toe # ifconfig cxgbe0 -toe Which connections are offloaded? Look for toe4 and/or toe6 in the output of netstat and sockstat: # netstat -np tcp | grep toe # sockstat -46c | grep toe Reviewed by: bz, gnn Sponsored by: Chelsio communications. MFC after: ~3 months (after 9.1, and after ensuring MFC is feasible)
2012-06-19 07:34:13 +00:00
struct sge_rxq *rxq = iq_to_rxq(iq); /* Use iff iq is part of rxq */
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
struct sge_fl *fl = &rxq->fl; /* Use iff IQ_HAS_FL */
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
struct adapter *sc = iq->adapter;
struct rsp_ctrl *ctrl;
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
const struct rss_header *rss;
int ndescs = 0, limit, fl_bufs_used = 0;
int rsp_type;
uint32_t lq;
struct mbuf *m0;
STAILQ_HEAD(, sge_iq) iql = STAILQ_HEAD_INITIALIZER(iql);
Merge r254386 from user/np/cxl_tuning. Add an INET|INET6 check missing in said revision. r254386: Flush inactive LRO entries periodically.
2013-08-29 06:26:22 +00:00
#if defined(INET) || defined(INET6)
const struct timeval lro_timeout = {0, sc->lro_timeout};
#endif
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
limit = budget ? budget : iq->qsize / 8;
KASSERT(iq->state == IQS_BUSY, ("%s: iq %p not BUSY", __func__, iq));
/*
* We always come back and check the descriptor ring for new indirect
* interrupts and other responses after running a single handler.
*/
for (;;) {
while (is_new_response(iq, &ctrl)) {
rmb();
m0 = NULL;
rsp_type = G_RSPD_TYPE(ctrl->u.type_gen);
lq = be32toh(ctrl->pldbuflen_qid);
rss = (const void *)iq->cdesc;
switch (rsp_type) {
case X_RSPD_TYPE_FLBUF:
KASSERT(iq->flags & IQ_HAS_FL,
("%s: data for an iq (%p) with no freelist",
__func__, iq));
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
m0 = get_fl_payload(sc, fl, lq, &fl_bufs_used);
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
if (__predict_false(m0 == NULL))
goto process_iql;
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
#ifdef T4_PKT_TIMESTAMP
/*
* 60 bit timestamp for the payload is
* *(uint64_t *)m0->m_pktdat. Note that it is
* in the leading free-space in the mbuf. The
* kernel can clobber it during a pullup,
* m_copymdata, etc. You need to make sure that
* the mbuf reaches you unmolested if you care
* about the timestamp.
*/
*(uint64_t *)m0->m_pktdat =
be64toh(ctrl->u.last_flit) &
0xfffffffffffffff;
#endif
/* fall through */
case X_RSPD_TYPE_CPL:
KASSERT(rss->opcode < NUM_CPL_CMDS,
("%s: bad opcode %02x.", __func__,
rss->opcode));
sc->cpl_handler[rss->opcode](iq, rss, m0);
break;
case X_RSPD_TYPE_INTR:
/*
* Interrupts should be forwarded only to queues
* that are not forwarding their interrupts.
* This means service_iq can recurse but only 1
* level deep.
*/
KASSERT(budget == 0,
("%s: budget %u, rsp_type %u", __func__,
budget, rsp_type));
Add hooks in base cxgbe(4) for the iWARP upper-layer driver. Update a couple of assertions in the TOE driver as well.
2013-08-28 20:45:45 +00:00
/*
* There are 1K interrupt-capable queues (qids 0
* through 1023). A response type indicating a
* forwarded interrupt with a qid >= 1K is an
* iWARP async notification.
*/
if (lq >= 1024) {
sc->an_handler(iq, ctrl);
break;
}
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
q = sc->sge.iqmap[lq - sc->sge.iq_start];
if (atomic_cmpset_int(&q->state, IQS_IDLE,
IQS_BUSY)) {
if (service_iq(q, q->qsize / 8) == 0) {
atomic_cmpset_int(&q->state,
IQS_BUSY, IQS_IDLE);
} else {
STAILQ_INSERT_TAIL(&iql, q,
link);
}
}
break;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
default:
Add hooks in base cxgbe(4) for the iWARP upper-layer driver. Update a couple of assertions in the TOE driver as well.
2013-08-28 20:45:45 +00:00
KASSERT(0,
("%s: illegal response type %d on iq %p",
__func__, rsp_type, iq));
log(LOG_ERR,
"%s: illegal response type %d on iq %p",
device_get_nameunit(sc->dev), rsp_type, iq);
- Updated TOE support in the kernel. - Stateful TCP offload drivers for Terminator 3 and 4 (T3 and T4) ASICs. These are available as t3_tom and t4_tom modules that augment cxgb(4) and cxgbe(4) respectively. The cxgb/cxgbe drivers continue to work as usual with or without these extra features. - iWARP driver for Terminator 3 ASIC (kernel verbs). T4 iWARP in the works and will follow soon. Build-tested with make universe. 30s overview ============ What interfaces support TCP offload? Look for TOE4 and/or TOE6 in the capabilities of an interface: # ifconfig -m | grep TOE Enable/disable TCP offload on an interface (just like any other ifnet capability): # ifconfig cxgbe0 toe # ifconfig cxgbe0 -toe Which connections are offloaded? Look for toe4 and/or toe6 in the output of netstat and sockstat: # netstat -np tcp | grep toe # sockstat -46c | grep toe Reviewed by: bz, gnn Sponsored by: Chelsio communications. MFC after: ~3 months (after 9.1, and after ensuring MFC is feasible)
2012-06-19 07:34:13 +00:00
break;
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
}
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
if (fl_bufs_used >= 16) {
FL_LOCK(fl);
fl->needed += fl_bufs_used;
refill_fl(sc, fl, 32);
FL_UNLOCK(fl);
fl_bufs_used = 0;
}
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
iq_next(iq);
if (++ndescs == limit) {
t4_write_reg(sc, MYPF_REG(A_SGE_PF_GTS),
V_CIDXINC(ndescs) |
V_INGRESSQID(iq->cntxt_id) |
V_SEINTARM(V_QINTR_TIMER_IDX(X_TIMERREG_UPDATE_CIDX)));
ndescs = 0;
Merge r254386 from user/np/cxl_tuning. Add an INET|INET6 check missing in said revision. r254386: Flush inactive LRO entries periodically.
2013-08-29 06:26:22 +00:00
#if defined(INET) || defined(INET6)
if (iq->flags & IQ_LRO_ENABLED &&
sc->lro_timeout != 0) {
tcp_lro_flush_inactive(&rxq->lro,
&lro_timeout);
}
#endif
cxgbe(4): Properly account for the freelist buffers used when returning early from service_iq due to a budget restriction. This fixes a potential rx hang when using INTx. MFC after: 3 days
2014-06-05 00:38:32 +00:00
if (budget) {
if (fl_bufs_used) {
FL_LOCK(fl);
fl->needed += fl_bufs_used;
refill_fl(sc, fl, 32);
FL_UNLOCK(fl);
}
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
return (EINPROGRESS);
cxgbe(4): Properly account for the freelist buffers used when returning early from service_iq due to a budget restriction. This fixes a potential rx hang when using INTx. MFC after: 3 days
2014-06-05 00:38:32 +00:00
}
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
}
}
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
process_iql:
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
if (STAILQ_EMPTY(&iql))
break;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
/*
* Process the head only, and send it to the back of the list if
* it's still not done.
*/
q = STAILQ_FIRST(&iql);
STAILQ_REMOVE_HEAD(&iql, link);
if (service_iq(q, q->qsize / 8) == 0)
atomic_cmpset_int(&q->state, IQS_BUSY, IQS_IDLE);
else
STAILQ_INSERT_TAIL(&iql, q, link);
}
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
cxgbe(4): support for IPv6 TSO and LRO. Submitted by: bz (this is a modified version of that patch)
2012-06-29 19:51:06 +00:00
#if defined(INET) || defined(INET6)
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
if (iq->flags & IQ_LRO_ENABLED) {
struct lro_ctrl *lro = &rxq->lro;
struct lro_entry *l;
while (!SLIST_EMPTY(&lro->lro_active)) {
l = SLIST_FIRST(&lro->lro_active);
SLIST_REMOVE_HEAD(&lro->lro_active, next);
tcp_lro_flush(lro, l);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
}
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
}
#endif
t4_write_reg(sc, MYPF_REG(A_SGE_PF_GTS), V_CIDXINC(ndescs) |
V_INGRESSQID((u32)iq->cntxt_id) | V_SEINTARM(iq->intr_params));
if (iq->flags & IQ_HAS_FL) {
int starved;
- Specialized ingress queues that take interrupts for other ingress queues. Try to have a set of these per port when possible, fall back to sharing a common pool between all ports otherwise. - One control queue per port (used to be one per hardware channel). - t4_eth_rx now handles Ethernet rx only. - sysctls to display pidx/cidx for some queues. MFC after: 1 week
2011-05-30 21:34:44 +00:00
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
FL_LOCK(fl);
fl->needed += fl_bufs_used;
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
starved = refill_fl(sc, fl, 64);
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
FL_UNLOCK(fl);
if (__predict_false(starved != 0))
add_fl_to_sfl(sc, fl);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
}
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
return (0);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
}
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
static inline int
cl_has_metadata(struct sge_fl *fl, struct cluster_layout *cll)
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
{
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
int rc = fl->flags & FL_BUF_PACKING || cll->region1 > 0;
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
if (rc)
MPASS(cll->region3 >= CL_METADATA_SIZE);
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
return (rc);
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
}
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
static inline struct cluster_metadata *
cl_metadata(struct adapter *sc, struct sge_fl *fl, struct cluster_layout *cll,
caddr_t cl)
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
{
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
if (cl_has_metadata(fl, cll)) {
struct sw_zone_info *swz = &sc->sge.sw_zone_info[cll->zidx];
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
return ((struct cluster_metadata *)(cl + swz->size) - 1);
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
}
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
return (NULL);
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
}
static int
rxb_free(struct mbuf *m, void *arg1, void *arg2)
{
uma_zone_t zone = arg1;
caddr_t cl = arg2;
uma_zfree(zone, cl);
return (EXT_FREE_OK);
}
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
/*
* The mbuf returned by this function could be allocated from zone_mbuf or
* constructed in spare room in the cluster.
*
* The mbuf carries the payload in one of these ways
* a) frame inside the mbuf (mbuf from zone_mbuf)
* b) m_cljset (for clusters without metadata) zone_mbuf
* c) m_extaddref (cluster with metadata) inline mbuf
* d) m_extaddref (cluster with metadata) zone_mbuf
*/
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
static struct mbuf *
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
get_scatter_segment(struct adapter *sc, struct sge_fl *fl, int total, int flags)
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
{
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
struct mbuf *m;
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
struct fl_sdesc *sd = &fl->sdesc[fl->cidx];
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
struct cluster_layout *cll = &sd->cll;
struct sw_zone_info *swz = &sc->sge.sw_zone_info[cll->zidx];
struct hw_buf_info *hwb = &sc->sge.hw_buf_info[cll->hwidx];
struct cluster_metadata *clm = cl_metadata(sc, fl, cll, sd->cl);
int len, padded_len;
caddr_t payload;
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
len = min(total, hwb->size - fl->rx_offset);
padded_len = roundup2(len, fl_pad);
payload = sd->cl + cll->region1 + fl->rx_offset;
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
if (sc->sc_do_rxcopy && len < RX_COPY_THRESHOLD) {
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
/*
* Copy payload into a freshly allocated mbuf.
*/
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
m = flags & M_PKTHDR ?
m_gethdr(M_NOWAIT, MT_DATA) : m_get(M_NOWAIT, MT_DATA);
if (m == NULL)
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
return (NULL);
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
fl->mbuf_allocated++;
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
#ifdef T4_PKT_TIMESTAMP
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
/* Leave room for a timestamp */
m->m_data += 8;
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
#endif
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
/* copy data to mbuf */
bcopy(payload, mtod(m, caddr_t), len);
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
cxgbe(4): Fix bug in the fast rx buffer recycle path. In some cases rx buffers were getting recycled when they should have been left alone. MFC after: 3 days
2014-06-18 00:16:35 +00:00
} else if (sd->nimbuf * MSIZE < cll->region1) {
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
/*
* There's spare room in the cluster for an mbuf. Create one
cxgbe(4): Fix bug in the fast rx buffer recycle path. In some cases rx buffers were getting recycled when they should have been left alone. MFC after: 3 days
2014-06-18 00:16:35 +00:00
* and associate it with the payload that's in the cluster.
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
*/
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
MPASS(clm != NULL);
cxgbe(4): Fix bug in the fast rx buffer recycle path. In some cases rx buffers were getting recycled when they should have been left alone. MFC after: 3 days
2014-06-18 00:16:35 +00:00
m = (struct mbuf *)(sd->cl + sd->nimbuf * MSIZE);
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
/* No bzero required */
if (m_init(m, NULL, 0, M_NOWAIT, MT_DATA, flags | M_NOFREE))
return (NULL);
fl->mbuf_inlined++;
m_extaddref(m, payload, padded_len, &clm->refcount, rxb_free,
swz->zone, sd->cl);
cxgbe(4): Fix bug in the fast rx buffer recycle path. In some cases rx buffers were getting recycled when they should have been left alone. MFC after: 3 days
2014-06-18 00:16:35 +00:00
sd->nimbuf++;
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
} else {
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
/*
* Grab an mbuf from zone_mbuf and associate it with the
* payload in the cluster.
*/
m = flags & M_PKTHDR ?
m_gethdr(M_NOWAIT, MT_DATA) : m_get(M_NOWAIT, MT_DATA);
if (m == NULL)
return (NULL);
fl->mbuf_allocated++;
cxgbe(4): Fix bug in the fast rx buffer recycle path. In some cases rx buffers were getting recycled when they should have been left alone. MFC after: 3 days
2014-06-18 00:16:35 +00:00
if (clm != NULL) {
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
m_extaddref(m, payload, padded_len, &clm->refcount,
rxb_free, swz->zone, sd->cl);
cxgbe(4): Fix bug in the fast rx buffer recycle path. In some cases rx buffers were getting recycled when they should have been left alone. MFC after: 3 days
2014-06-18 00:16:35 +00:00
sd->nembuf++;
} else {
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
m_cljset(m, sd->cl, swz->type);
sd->cl = NULL; /* consumed, not a recycle candidate */
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
}
}
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
if (flags & M_PKTHDR)
m->m_pkthdr.len = total;
m->m_len = len;
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
if (fl->flags & FL_BUF_PACKING) {
fl->rx_offset += roundup2(padded_len, sc->sge.pack_boundary);
MPASS(fl->rx_offset <= hwb->size);
if (fl->rx_offset < hwb->size)
return (m); /* without advancing the cidx */
}
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
if (__predict_false(++fl->cidx == fl->cap))
fl->cidx = 0;
fl->rx_offset = 0;
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
return (m);
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
}
static struct mbuf *
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
get_fl_payload(struct adapter *sc, struct sge_fl *fl, uint32_t len_newbuf,
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
int *fl_bufs_used)
{
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
struct mbuf *m0, *m, **pnext;
u_int nbuf, len;
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
/*
* No assertion for the fl lock because we don't need it. This routine
* is called only from the rx interrupt handler and it only updates
* fl->cidx. (Contrast that with fl->pidx/fl->needed which could be
* updated in the rx interrupt handler or the starvation helper routine.
* That's why code that manipulates fl->pidx/fl->needed needs the fl
* lock but this routine does not).
*/
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
nbuf = 0;
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
len = G_RSPD_LEN(len_newbuf);
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
if (__predict_false(fl->m0 != NULL)) {
cxgbe(4): Fix a NULL dereference when the very first call to get_scatter_segment() in get_fl_payload() fails. While here, fix the code to adjust fl_bufs_used when a failure occurs for any other scatter segment. MFC after: 3 days
2014-05-30 22:59:45 +00:00
M_ASSERTPKTHDR(fl->m0);
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
MPASS(len == fl->m0->m_pkthdr.len);
MPASS(fl->remaining < len);
m0 = fl->m0;
pnext = fl->pnext;
len = fl->remaining;
fl->m0 = NULL;
goto get_segment;
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
}
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
if (fl->rx_offset > 0 && len_newbuf & F_RSPD_NEWBUF) {
nbuf++;
fl->rx_offset = 0;
if (__predict_false(++fl->cidx == fl->cap))
fl->cidx = 0;
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
}
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
/*
* Payload starts at rx_offset in the current hw buffer. Its length is
* 'len' and it may span multiple hw buffers.
*/
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
m0 = get_scatter_segment(sc, fl, len, M_PKTHDR);
cxgbe(4): Fix a NULL dereference when the very first call to get_scatter_segment() in get_fl_payload() fails. While here, fix the code to adjust fl_bufs_used when a failure occurs for any other scatter segment. MFC after: 3 days
2014-05-30 22:59:45 +00:00
if (m0 == NULL)
goto done;
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
len -= m0->m_len;
pnext = &m0->m_next;
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
while (len > 0) {
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
nbuf++;
get_segment:
MPASS(fl->rx_offset == 0);
m = get_scatter_segment(sc, fl, len, 0);
if (m == NULL) {
fl->m0 = m0;
fl->pnext = pnext;
fl->remaining = len;
cxgbe(4): Fix a NULL dereference when the very first call to get_scatter_segment() in get_fl_payload() fails. While here, fix the code to adjust fl_bufs_used when a failure occurs for any other scatter segment. MFC after: 3 days
2014-05-30 22:59:45 +00:00
m0 = NULL;
goto done;
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
}
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
*pnext = m;
pnext = &m->m_next;
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
len -= m->m_len;
}
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
*pnext = NULL;
if (fl->rx_offset == 0)
nbuf++;
cxgbe(4): Fix a NULL dereference when the very first call to get_scatter_segment() in get_fl_payload() fails. While here, fix the code to adjust fl_bufs_used when a failure occurs for any other scatter segment. MFC after: 3 days
2014-05-30 22:59:45 +00:00
done:
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
(*fl_bufs_used) += nbuf;
return (m0);
}
static int
t4_eth_rx(struct sge_iq *iq, const struct rss_header *rss, struct mbuf *m0)
{
Do not allocate extra vectors when adapter is not TOE capable (or toecaps have been disallowed by the user). + one very minor unrelated cleanup in t4_sge.c
2012-06-22 22:59:42 +00:00
struct sge_rxq *rxq = iq_to_rxq(iq);
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
struct ifnet *ifp = rxq->ifp;
const struct cpl_rx_pkt *cpl = (const void *)(rss + 1);
cxgbe(4): support for IPv6 TSO and LRO. Submitted by: bz (this is a modified version of that patch)
2012-06-29 19:51:06 +00:00
#if defined(INET) || defined(INET6)
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
struct lro_ctrl *lro = &rxq->lro;
#endif
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
KASSERT(m0 != NULL, ("%s: no payload with opcode %02x", __func__,
rss->opcode));
Tweaks for rx: - everything related to LRO should be in #ifdef INET blocks - reorder sge_iq's fields so that the most frequently used are all together - pull all rx code into t4_intr_data directly - let go of the ingress queue lock when passing up data - refill the freelist only if it is short of at least 32 buffers
2011-03-05 03:42:03 +00:00
Convert some fixed parameters to tunables (with reasonable default values). - cong_drop specifies what to do on congestion: nothing, backpressure, or drop. - fl_pktshift specifies the padding before Ethernet payload. - fl_pad specifies the boundary upto which to pad Ethernet payload. - spg_len controls the length of the status page. MFC after: 2 weeks
2012-08-14 21:47:41 +00:00
m0->m_pkthdr.len -= fl_pktshift;
m0->m_len -= fl_pktshift;
m0->m_data += fl_pktshift;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
m0->m_pkthdr.rcvif = ifp;
m0->m_flags |= M_FLOWID;
cxgbe(4): save a copy of the RSS map for each port for the driver's use.
2013-12-08 17:47:37 +00:00
m0->m_pkthdr.flowid = be32toh(rss->hash_val);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
cxgbe(4): support for IPv6 hardware checksumming (rx and tx).
2012-06-29 16:50:52 +00:00
if (cpl->csum_calc && !cpl->err_vec) {
if (ifp->if_capenable & IFCAP_RXCSUM &&
cpl->l2info & htobe32(F_RXF_IP)) {
- Assign (don't OR) the CSUM_XXX bits to csum_flags in the rx checksum code. - Fix TSO/TSO4 mixup. - Add IFCAP_LINKSTATE to the available/enabled capabilities.
2012-06-30 02:05:09 +00:00
m0->m_pkthdr.csum_flags = (CSUM_IP_CHECKED |
cxgbe(4): support for IPv6 hardware checksumming (rx and tx).
2012-06-29 16:50:52 +00:00
CSUM_IP_VALID | CSUM_DATA_VALID | CSUM_PSEUDO_HDR);
rxq->rxcsum++;
} else if (ifp->if_capenable & IFCAP_RXCSUM_IPV6 &&
cpl->l2info & htobe32(F_RXF_IP6)) {
- Assign (don't OR) the CSUM_XXX bits to csum_flags in the rx checksum code. - Fix TSO/TSO4 mixup. - Add IFCAP_LINKSTATE to the available/enabled capabilities.
2012-06-30 02:05:09 +00:00
m0->m_pkthdr.csum_flags = (CSUM_DATA_VALID_IPV6 |
cxgbe(4): support for IPv6 hardware checksumming (rx and tx).
2012-06-29 16:50:52 +00:00
CSUM_PSEUDO_HDR);
rxq->rxcsum++;
}
if (__predict_false(cpl->ip_frag))
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
m0->m_pkthdr.csum_data = be16toh(cpl->csum);
else
m0->m_pkthdr.csum_data = 0xffff;
}
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
if (cpl->vlan_ex) {
m0->m_pkthdr.ether_vtag = be16toh(cpl->vlan);
m0->m_flags |= M_VLANTAG;
rxq->vlan_extraction++;
}
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
cxgbe(4): support for IPv6 TSO and LRO. Submitted by: bz (this is a modified version of that patch)
2012-06-29 19:51:06 +00:00
#if defined(INET) || defined(INET6)
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
if (cpl->l2info & htobe32(F_RXF_LRO) &&
iq->flags & IQ_LRO_ENABLED &&
tcp_lro_rx(lro, m0, 0) == 0) {
/* queued for LRO */
} else
#endif
ifp->if_input(ifp, m0);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
return (0);
}
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
/*
* Doesn't fail. Holds on to work requests it can't send right away.
*/
- Updated TOE support in the kernel. - Stateful TCP offload drivers for Terminator 3 and 4 (T3 and T4) ASICs. These are available as t3_tom and t4_tom modules that augment cxgb(4) and cxgbe(4) respectively. The cxgb/cxgbe drivers continue to work as usual with or without these extra features. - iWARP driver for Terminator 3 ASIC (kernel verbs). T4 iWARP in the works and will follow soon. Build-tested with make universe. 30s overview ============ What interfaces support TCP offload? Look for TOE4 and/or TOE6 in the capabilities of an interface: # ifconfig -m | grep TOE Enable/disable TCP offload on an interface (just like any other ifnet capability): # ifconfig cxgbe0 toe # ifconfig cxgbe0 -toe Which connections are offloaded? Look for toe4 and/or toe6 in the output of netstat and sockstat: # netstat -np tcp | grep toe # sockstat -46c | grep toe Reviewed by: bz, gnn Sponsored by: Chelsio communications. MFC after: ~3 months (after 9.1, and after ensuring MFC is feasible)
2012-06-19 07:34:13 +00:00
void
t4_wrq_tx_locked(struct adapter *sc, struct sge_wrq *wrq, struct wrqe *wr)
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
{
struct sge_eq *eq = &wrq->eq;
int can_reclaim;
caddr_t dst;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
TXQ_LOCK_ASSERT_OWNED(wrq);
- Updated TOE support in the kernel. - Stateful TCP offload drivers for Terminator 3 and 4 (T3 and T4) ASICs. These are available as t3_tom and t4_tom modules that augment cxgb(4) and cxgbe(4) respectively. The cxgb/cxgbe drivers continue to work as usual with or without these extra features. - iWARP driver for Terminator 3 ASIC (kernel verbs). T4 iWARP in the works and will follow soon. Build-tested with make universe. 30s overview ============ What interfaces support TCP offload? Look for TOE4 and/or TOE6 in the capabilities of an interface: # ifconfig -m | grep TOE Enable/disable TCP offload on an interface (just like any other ifnet capability): # ifconfig cxgbe0 toe # ifconfig cxgbe0 -toe Which connections are offloaded? Look for toe4 and/or toe6 in the output of netstat and sockstat: # netstat -np tcp | grep toe # sockstat -46c | grep toe Reviewed by: bz, gnn Sponsored by: Chelsio communications. MFC after: ~3 months (after 9.1, and after ensuring MFC is feasible)
2012-06-19 07:34:13 +00:00
#ifdef TCP_OFFLOAD
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
KASSERT((eq->flags & EQ_TYPEMASK) == EQ_OFLD ||
(eq->flags & EQ_TYPEMASK) == EQ_CTRL,
("%s: eq type %d", __func__, eq->flags & EQ_TYPEMASK));
- Updated TOE support in the kernel. - Stateful TCP offload drivers for Terminator 3 and 4 (T3 and T4) ASICs. These are available as t3_tom and t4_tom modules that augment cxgb(4) and cxgbe(4) respectively. The cxgb/cxgbe drivers continue to work as usual with or without these extra features. - iWARP driver for Terminator 3 ASIC (kernel verbs). T4 iWARP in the works and will follow soon. Build-tested with make universe. 30s overview ============ What interfaces support TCP offload? Look for TOE4 and/or TOE6 in the capabilities of an interface: # ifconfig -m | grep TOE Enable/disable TCP offload on an interface (just like any other ifnet capability): # ifconfig cxgbe0 toe # ifconfig cxgbe0 -toe Which connections are offloaded? Look for toe4 and/or toe6 in the output of netstat and sockstat: # netstat -np tcp | grep toe # sockstat -46c | grep toe Reviewed by: bz, gnn Sponsored by: Chelsio communications. MFC after: ~3 months (after 9.1, and after ensuring MFC is feasible)
2012-06-19 07:34:13 +00:00
#else
KASSERT((eq->flags & EQ_TYPEMASK) == EQ_CTRL,
("%s: eq type %d", __func__, eq->flags & EQ_TYPEMASK));
#endif
Tweaks for rx: - everything related to LRO should be in #ifdef INET blocks - reorder sge_iq's fields so that the most frequently used are all together - pull all rx code into t4_intr_data directly - let go of the ingress queue lock when passing up data - refill the freelist only if it is short of at least 32 buffers
2011-03-05 03:42:03 +00:00
- Updated TOE support in the kernel. - Stateful TCP offload drivers for Terminator 3 and 4 (T3 and T4) ASICs. These are available as t3_tom and t4_tom modules that augment cxgb(4) and cxgbe(4) respectively. The cxgb/cxgbe drivers continue to work as usual with or without these extra features. - iWARP driver for Terminator 3 ASIC (kernel verbs). T4 iWARP in the works and will follow soon. Build-tested with make universe. 30s overview ============ What interfaces support TCP offload? Look for TOE4 and/or TOE6 in the capabilities of an interface: # ifconfig -m | grep TOE Enable/disable TCP offload on an interface (just like any other ifnet capability): # ifconfig cxgbe0 toe # ifconfig cxgbe0 -toe Which connections are offloaded? Look for toe4 and/or toe6 in the output of netstat and sockstat: # netstat -np tcp | grep toe # sockstat -46c | grep toe Reviewed by: bz, gnn Sponsored by: Chelsio communications. MFC after: ~3 months (after 9.1, and after ensuring MFC is feasible)
2012-06-19 07:34:13 +00:00
if (__predict_true(wr != NULL))
STAILQ_INSERT_TAIL(&wrq->wr_list, wr, link);
Tweaks for rx: - everything related to LRO should be in #ifdef INET blocks - reorder sge_iq's fields so that the most frequently used are all together - pull all rx code into t4_intr_data directly - let go of the ingress queue lock when passing up data - refill the freelist only if it is short of at least 32 buffers
2011-03-05 03:42:03 +00:00
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
can_reclaim = reclaimable(eq);
if (__predict_false(eq->flags & EQ_STALLED)) {
Consider the total number of descriptors available (and not just those that are ready to be reclaimed) when deciding whether to resume tx after a stall. MFC after: 3 days
2014-06-20 20:28:46 +00:00
if (eq->avail + can_reclaim < tx_resume_threshold(eq))
- Updated TOE support in the kernel. - Stateful TCP offload drivers for Terminator 3 and 4 (T3 and T4) ASICs. These are available as t3_tom and t4_tom modules that augment cxgb(4) and cxgbe(4) respectively. The cxgb/cxgbe drivers continue to work as usual with or without these extra features. - iWARP driver for Terminator 3 ASIC (kernel verbs). T4 iWARP in the works and will follow soon. Build-tested with make universe. 30s overview ============ What interfaces support TCP offload? Look for TOE4 and/or TOE6 in the capabilities of an interface: # ifconfig -m | grep TOE Enable/disable TCP offload on an interface (just like any other ifnet capability): # ifconfig cxgbe0 toe # ifconfig cxgbe0 -toe Which connections are offloaded? Look for toe4 and/or toe6 in the output of netstat and sockstat: # netstat -np tcp | grep toe # sockstat -46c | grep toe Reviewed by: bz, gnn Sponsored by: Chelsio communications. MFC after: ~3 months (after 9.1, and after ensuring MFC is feasible)
2012-06-19 07:34:13 +00:00
return;
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
eq->flags &= ~EQ_STALLED;
eq->unstalled++;
}
eq->cidx += can_reclaim;
eq->avail += can_reclaim;
if (__predict_false(eq->cidx >= eq->cap))
eq->cidx -= eq->cap;
T4 packet timestamps. Reference code that shows how to get a packet's timestamp out of cxgbe(4). Disabled by default because we don't have a standard way today to pass this information up the stack. The timestamp is 60 bits wide and each increment represents 1 tick of the T4's core clock. As an example, the timestamp granularity is ~4.4ns for this card: # sysctl dev.t4nex.0.core_clock dev.t4nex.0.core_clock: 228125 MFC after: 1 week
2011-05-05 02:38:08 +00:00
- Updated TOE support in the kernel. - Stateful TCP offload drivers for Terminator 3 and 4 (T3 and T4) ASICs. These are available as t3_tom and t4_tom modules that augment cxgb(4) and cxgbe(4) respectively. The cxgb/cxgbe drivers continue to work as usual with or without these extra features. - iWARP driver for Terminator 3 ASIC (kernel verbs). T4 iWARP in the works and will follow soon. Build-tested with make universe. 30s overview ============ What interfaces support TCP offload? Look for TOE4 and/or TOE6 in the capabilities of an interface: # ifconfig -m | grep TOE Enable/disable TCP offload on an interface (just like any other ifnet capability): # ifconfig cxgbe0 toe # ifconfig cxgbe0 -toe Which connections are offloaded? Look for toe4 and/or toe6 in the output of netstat and sockstat: # netstat -np tcp | grep toe # sockstat -46c | grep toe Reviewed by: bz, gnn Sponsored by: Chelsio communications. MFC after: ~3 months (after 9.1, and after ensuring MFC is feasible)
2012-06-19 07:34:13 +00:00
while ((wr = STAILQ_FIRST(&wrq->wr_list)) != NULL) {
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
int ndesc;
- Updated TOE support in the kernel. - Stateful TCP offload drivers for Terminator 3 and 4 (T3 and T4) ASICs. These are available as t3_tom and t4_tom modules that augment cxgb(4) and cxgbe(4) respectively. The cxgb/cxgbe drivers continue to work as usual with or without these extra features. - iWARP driver for Terminator 3 ASIC (kernel verbs). T4 iWARP in the works and will follow soon. Build-tested with make universe. 30s overview ============ What interfaces support TCP offload? Look for TOE4 and/or TOE6 in the capabilities of an interface: # ifconfig -m | grep TOE Enable/disable TCP offload on an interface (just like any other ifnet capability): # ifconfig cxgbe0 toe # ifconfig cxgbe0 -toe Which connections are offloaded? Look for toe4 and/or toe6 in the output of netstat and sockstat: # netstat -np tcp | grep toe # sockstat -46c | grep toe Reviewed by: bz, gnn Sponsored by: Chelsio communications. MFC after: ~3 months (after 9.1, and after ensuring MFC is feasible)
2012-06-19 07:34:13 +00:00
if (__predict_false(wr->wr_len < 0 ||
wr->wr_len > SGE_MAX_WR_LEN || (wr->wr_len & 0x7))) {
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
#ifdef INVARIANTS
- Updated TOE support in the kernel. - Stateful TCP offload drivers for Terminator 3 and 4 (T3 and T4) ASICs. These are available as t3_tom and t4_tom modules that augment cxgb(4) and cxgbe(4) respectively. The cxgb/cxgbe drivers continue to work as usual with or without these extra features. - iWARP driver for Terminator 3 ASIC (kernel verbs). T4 iWARP in the works and will follow soon. Build-tested with make universe. 30s overview ============ What interfaces support TCP offload? Look for TOE4 and/or TOE6 in the capabilities of an interface: # ifconfig -m | grep TOE Enable/disable TCP offload on an interface (just like any other ifnet capability): # ifconfig cxgbe0 toe # ifconfig cxgbe0 -toe Which connections are offloaded? Look for toe4 and/or toe6 in the output of netstat and sockstat: # netstat -np tcp | grep toe # sockstat -46c | grep toe Reviewed by: bz, gnn Sponsored by: Chelsio communications. MFC after: ~3 months (after 9.1, and after ensuring MFC is feasible)
2012-06-19 07:34:13 +00:00
panic("%s: work request with length %d", __func__,
wr->wr_len);
T4 packet timestamps. Reference code that shows how to get a packet's timestamp out of cxgbe(4). Disabled by default because we don't have a standard way today to pass this information up the stack. The timestamp is 60 bits wide and each increment represents 1 tick of the T4's core clock. As an example, the timestamp granularity is ~4.4ns for this card: # sysctl dev.t4nex.0.core_clock dev.t4nex.0.core_clock: 228125 MFC after: 1 week
2011-05-05 02:38:08 +00:00
#endif
- Updated TOE support in the kernel. - Stateful TCP offload drivers for Terminator 3 and 4 (T3 and T4) ASICs. These are available as t3_tom and t4_tom modules that augment cxgb(4) and cxgbe(4) respectively. The cxgb/cxgbe drivers continue to work as usual with or without these extra features. - iWARP driver for Terminator 3 ASIC (kernel verbs). T4 iWARP in the works and will follow soon. Build-tested with make universe. 30s overview ============ What interfaces support TCP offload? Look for TOE4 and/or TOE6 in the capabilities of an interface: # ifconfig -m | grep TOE Enable/disable TCP offload on an interface (just like any other ifnet capability): # ifconfig cxgbe0 toe # ifconfig cxgbe0 -toe Which connections are offloaded? Look for toe4 and/or toe6 in the output of netstat and sockstat: # netstat -np tcp | grep toe # sockstat -46c | grep toe Reviewed by: bz, gnn Sponsored by: Chelsio communications. MFC after: ~3 months (after 9.1, and after ensuring MFC is feasible)
2012-06-19 07:34:13 +00:00
#ifdef KDB
kdb_backtrace();
#endif
log(LOG_ERR, "%s: %s work request with length %d",
device_get_nameunit(sc->dev), __func__, wr->wr_len);
STAILQ_REMOVE_HEAD(&wrq->wr_list, link);
free_wrqe(wr);
continue;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
}
- Updated TOE support in the kernel. - Stateful TCP offload drivers for Terminator 3 and 4 (T3 and T4) ASICs. These are available as t3_tom and t4_tom modules that augment cxgb(4) and cxgbe(4) respectively. The cxgb/cxgbe drivers continue to work as usual with or without these extra features. - iWARP driver for Terminator 3 ASIC (kernel verbs). T4 iWARP in the works and will follow soon. Build-tested with make universe. 30s overview ============ What interfaces support TCP offload? Look for TOE4 and/or TOE6 in the capabilities of an interface: # ifconfig -m | grep TOE Enable/disable TCP offload on an interface (just like any other ifnet capability): # ifconfig cxgbe0 toe # ifconfig cxgbe0 -toe Which connections are offloaded? Look for toe4 and/or toe6 in the output of netstat and sockstat: # netstat -np tcp | grep toe # sockstat -46c | grep toe Reviewed by: bz, gnn Sponsored by: Chelsio communications. MFC after: ~3 months (after 9.1, and after ensuring MFC is feasible)
2012-06-19 07:34:13 +00:00
ndesc = howmany(wr->wr_len, EQ_ESIZE);
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
if (eq->avail < ndesc) {
wrq->no_desc++;
break;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
}
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
dst = (void *)&eq->desc[eq->pidx];
- Updated TOE support in the kernel. - Stateful TCP offload drivers for Terminator 3 and 4 (T3 and T4) ASICs. These are available as t3_tom and t4_tom modules that augment cxgb(4) and cxgbe(4) respectively. The cxgb/cxgbe drivers continue to work as usual with or without these extra features. - iWARP driver for Terminator 3 ASIC (kernel verbs). T4 iWARP in the works and will follow soon. Build-tested with make universe. 30s overview ============ What interfaces support TCP offload? Look for TOE4 and/or TOE6 in the capabilities of an interface: # ifconfig -m | grep TOE Enable/disable TCP offload on an interface (just like any other ifnet capability): # ifconfig cxgbe0 toe # ifconfig cxgbe0 -toe Which connections are offloaded? Look for toe4 and/or toe6 in the output of netstat and sockstat: # netstat -np tcp | grep toe # sockstat -46c | grep toe Reviewed by: bz, gnn Sponsored by: Chelsio communications. MFC after: ~3 months (after 9.1, and after ensuring MFC is feasible)
2012-06-19 07:34:13 +00:00
copy_to_txd(eq, wrtod(wr), &dst, wr->wr_len);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
eq->pidx += ndesc;
eq->avail -= ndesc;
if (__predict_false(eq->pidx >= eq->cap))
eq->pidx -= eq->cap;
Tweaks for rx: - everything related to LRO should be in #ifdef INET blocks - reorder sge_iq's fields so that the most frequently used are all together - pull all rx code into t4_intr_data directly - let go of the ingress queue lock when passing up data - refill the freelist only if it is short of at least 32 buffers
2011-03-05 03:42:03 +00:00
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
eq->pending += ndesc;
Ring the egress queue's doorbell as soon as there are 8 or more descriptors ready to be processed. MFC after: 1 day
2013-07-04 19:15:41 +00:00
if (eq->pending >= 8)
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
ring_eq_db(sc, eq);
Tweaks for rx: - everything related to LRO should be in #ifdef INET blocks - reorder sge_iq's fields so that the most frequently used are all together - pull all rx code into t4_intr_data directly - let go of the ingress queue lock when passing up data - refill the freelist only if it is short of at least 32 buffers
2011-03-05 03:42:03 +00:00
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
wrq->tx_wrs++;
- Updated TOE support in the kernel. - Stateful TCP offload drivers for Terminator 3 and 4 (T3 and T4) ASICs. These are available as t3_tom and t4_tom modules that augment cxgb(4) and cxgbe(4) respectively. The cxgb/cxgbe drivers continue to work as usual with or without these extra features. - iWARP driver for Terminator 3 ASIC (kernel verbs). T4 iWARP in the works and will follow soon. Build-tested with make universe. 30s overview ============ What interfaces support TCP offload? Look for TOE4 and/or TOE6 in the capabilities of an interface: # ifconfig -m | grep TOE Enable/disable TCP offload on an interface (just like any other ifnet capability): # ifconfig cxgbe0 toe # ifconfig cxgbe0 -toe Which connections are offloaded? Look for toe4 and/or toe6 in the output of netstat and sockstat: # netstat -np tcp | grep toe # sockstat -46c | grep toe Reviewed by: bz, gnn Sponsored by: Chelsio communications. MFC after: ~3 months (after 9.1, and after ensuring MFC is feasible)
2012-06-19 07:34:13 +00:00
STAILQ_REMOVE_HEAD(&wrq->wr_list, link);
free_wrqe(wr);
Tweaks for rx: - everything related to LRO should be in #ifdef INET blocks - reorder sge_iq's fields so that the most frequently used are all together - pull all rx code into t4_intr_data directly - let go of the ingress queue lock when passing up data - refill the freelist only if it is short of at least 32 buffers
2011-03-05 03:42:03 +00:00
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
if (eq->avail < 8) {
can_reclaim = reclaimable(eq);
eq->cidx += can_reclaim;
eq->avail += can_reclaim;
if (__predict_false(eq->cidx >= eq->cap))
eq->cidx -= eq->cap;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
}
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
}
Tweaks for rx: - everything related to LRO should be in #ifdef INET blocks - reorder sge_iq's fields so that the most frequently used are all together - pull all rx code into t4_intr_data directly - let go of the ingress queue lock when passing up data - refill the freelist only if it is short of at least 32 buffers
2011-03-05 03:42:03 +00:00
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
if (eq->pending)
ring_eq_db(sc, eq);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
- Updated TOE support in the kernel. - Stateful TCP offload drivers for Terminator 3 and 4 (T3 and T4) ASICs. These are available as t3_tom and t4_tom modules that augment cxgb(4) and cxgbe(4) respectively. The cxgb/cxgbe drivers continue to work as usual with or without these extra features. - iWARP driver for Terminator 3 ASIC (kernel verbs). T4 iWARP in the works and will follow soon. Build-tested with make universe. 30s overview ============ What interfaces support TCP offload? Look for TOE4 and/or TOE6 in the capabilities of an interface: # ifconfig -m | grep TOE Enable/disable TCP offload on an interface (just like any other ifnet capability): # ifconfig cxgbe0 toe # ifconfig cxgbe0 -toe Which connections are offloaded? Look for toe4 and/or toe6 in the output of netstat and sockstat: # netstat -np tcp | grep toe # sockstat -46c | grep toe Reviewed by: bz, gnn Sponsored by: Chelsio communications. MFC after: ~3 months (after 9.1, and after ensuring MFC is feasible)
2012-06-19 07:34:13 +00:00
if (wr != NULL) {
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
eq->flags |= EQ_STALLED;
if (callout_pending(&eq->tx_callout) == 0)
callout_reset(&eq->tx_callout, 1, t4_tx_callout, eq);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
}
- Move all Ethernet specific items from sge_eq to sge_txq. sge_eq is now a suitable base for all kinds of egress queues. - Add control queues (sge_ctrlq) and allocate one of these per hardware channel. They can be used to program filters and steer traffic (and more). MFC after: 1 week
2011-04-19 22:08:28 +00:00
}
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
/* Per-packet header in a coalesced tx WR, before the SGL starts (in flits) */
#define TXPKTS_PKT_HDR ((\
sizeof(struct ulp_txpkt) + \
sizeof(struct ulptx_idata) + \
sizeof(struct cpl_tx_pkt_core) \
) / 8)
/* Header of a coalesced tx WR, before SGL of first packet (in flits) */
#define TXPKTS_WR_HDR (\
sizeof(struct fw_eth_tx_pkts_wr) / 8 + \
TXPKTS_PKT_HDR)
/* Header of a tx WR, before SGL of first packet (in flits) */
#define TXPKT_WR_HDR ((\
sizeof(struct fw_eth_tx_pkt_wr) + \
sizeof(struct cpl_tx_pkt_core) \
) / 8 )
/* Header of a tx LSO WR, before SGL of first packet (in flits) */
#define TXPKT_LSO_WR_HDR ((\
sizeof(struct fw_eth_tx_pkt_wr) + \
cxgbe(4): update to firmware interface 1.5.2.0; updates to shared code.
2012-06-22 07:51:15 +00:00
sizeof(struct cpl_tx_pkt_lso_core) + \
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
sizeof(struct cpl_tx_pkt_core) \
) / 8 )
int
t4_eth_tx(struct ifnet *ifp, struct sge_txq *txq, struct mbuf *m)
{
struct port_info *pi = (void *)ifp->if_softc;
struct adapter *sc = pi->adapter;
struct sge_eq *eq = &txq->eq;
- Move all Ethernet specific items from sge_eq to sge_txq. sge_eq is now a suitable base for all kinds of egress queues. - Add control queues (sge_ctrlq) and allocate one of these per hardware channel. They can be used to program filters and steer traffic (and more). MFC after: 1 week
2011-04-19 22:08:28 +00:00
struct buf_ring *br = txq->br;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
struct mbuf *next;
Calculate how many descriptors can be reclaimed before calling reclaim_tx_descs
2011-03-05 03:54:37 +00:00
int rc, coalescing, can_reclaim;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
struct txpkts txpkts;
struct sgl sgl;
TXQ_LOCK_ASSERT_OWNED(txq);
KASSERT(m, ("%s: called with nothing to do.", __func__));
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
KASSERT((eq->flags & EQ_TYPEMASK) == EQ_ETH,
("%s: eq type %d", __func__, eq->flags & EQ_TYPEMASK));
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
Calculate how many descriptors can be reclaimed before calling reclaim_tx_descs
2011-03-05 03:54:37 +00:00
prefetch(&eq->desc[eq->pidx]);
- Move all Ethernet specific items from sge_eq to sge_txq. sge_eq is now a suitable base for all kinds of egress queues. - Add control queues (sge_ctrlq) and allocate one of these per hardware channel. They can be used to program filters and steer traffic (and more). MFC after: 1 week
2011-04-19 22:08:28 +00:00
prefetch(&txq->sdesc[eq->pidx]);
Calculate how many descriptors can be reclaimed before calling reclaim_tx_descs
2011-03-05 03:54:37 +00:00
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
txpkts.npkt = 0;/* indicates there's nothing in txpkts */
coalescing = 0;
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
can_reclaim = reclaimable(eq);
if (__predict_false(eq->flags & EQ_STALLED)) {
Consider the total number of descriptors available (and not just those that are ready to be reclaimed) when deciding whether to resume tx after a stall. MFC after: 3 days
2014-06-20 20:28:46 +00:00
if (eq->avail + can_reclaim < tx_resume_threshold(eq)) {
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
txq->m = m;
return (0);
}
eq->flags &= ~EQ_STALLED;
eq->unstalled++;
}
if (__predict_false(eq->flags & EQ_DOOMED)) {
m_freem(m);
while ((m = buf_ring_dequeue_sc(txq->br)) != NULL)
m_freem(m);
return (ENETDOWN);
}
if (eq->avail < 8 && can_reclaim)
reclaim_tx_descs(txq, can_reclaim, 32);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
for (; m; m = next ? next : drbr_dequeue(ifp, br)) {
if (eq->avail < 8)
break;
next = m->m_nextpkt;
m->m_nextpkt = NULL;
if (next || buf_ring_peek(br))
coalescing = 1;
rc = get_pkt_sgl(txq, &m, &sgl, coalescing);
if (rc != 0) {
if (rc == ENOMEM) {
/* Short of resources, suspend tx */
m->m_nextpkt = next;
break;
}
/*
* Unrecoverable error for this packet, throw it away
* and move on to the next. get_pkt_sgl may already
* have freed m (it will be NULL in that case and the
* m_freem here is still safe).
*/
m_freem(m);
continue;
}
if (coalescing &&
add_to_txpkts(pi, txq, &txpkts, m, &sgl) == 0) {
/* Successfully absorbed into txpkts */
write_ulp_cpl_sgl(pi, txq, &txpkts, m, &sgl);
goto doorbell;
}
/*
* We weren't coalescing to begin with, or current frame could
* not be coalesced (add_to_txpkts flushes txpkts if a frame
* given to it can't be coalesced). Either way there should be
* nothing in txpkts.
*/
KASSERT(txpkts.npkt == 0,
("%s: txpkts not empty: %d", __func__, txpkts.npkt));
/* We're sending out individual packets now */
coalescing = 0;
if (eq->avail < 8)
- Move all Ethernet specific items from sge_eq to sge_txq. sge_eq is now a suitable base for all kinds of egress queues. - Add control queues (sge_ctrlq) and allocate one of these per hardware channel. They can be used to program filters and steer traffic (and more). MFC after: 1 week
2011-04-19 22:08:28 +00:00
reclaim_tx_descs(txq, 0, 8);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
rc = write_txpkt_wr(pi, txq, m, &sgl);
if (rc != 0) {
/* Short of hardware descriptors, suspend tx */
/*
* This is an unlikely but expensive failure. We've
* done all the hard work (DMA mappings etc.) and now we
* can't send out the packet. What's worse, we have to
* spend even more time freeing up everything in sgl.
*/
txq->no_desc++;
free_pkt_sgl(txq, &sgl);
m->m_nextpkt = next;
break;
}
ETHER_BPF_MTAP(ifp, m);
if (sgl.nsegs == 0)
m_freem(m);
doorbell:
Ring the egress queue's doorbell as soon as there are 8 or more descriptors ready to be processed. MFC after: 1 day
2013-07-04 19:15:41 +00:00
if (eq->pending >= 8)
ring_eq_db(sc, eq);
Calculate how many descriptors can be reclaimed before calling reclaim_tx_descs
2011-03-05 03:54:37 +00:00
can_reclaim = reclaimable(eq);
if (can_reclaim >= 32)
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
reclaim_tx_descs(txq, can_reclaim, 64);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
}
if (txpkts.npkt > 0)
write_txpkts_wr(txq, &txpkts);
/*
* m not NULL means there was an error but we haven't thrown it away.
* This can happen when we're short of tx descriptors (no_desc) or maybe
* even DMA maps (no_dmamap). Either way, a credit flush and reclaim
* will get things going again.
*/
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
if (m && !(eq->flags & EQ_CRFLUSHED)) {
- Move all Ethernet specific items from sge_eq to sge_txq. sge_eq is now a suitable base for all kinds of egress queues. - Add control queues (sge_ctrlq) and allocate one of these per hardware channel. They can be used to program filters and steer traffic (and more). MFC after: 1 week
2011-04-19 22:08:28 +00:00
struct tx_sdesc *txsd = &txq->sdesc[eq->pidx];
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
/*
* If EQ_CRFLUSHED is not set then we know we have at least one
* available descriptor because any WR that reduces eq->avail to
* 0 also sets EQ_CRFLUSHED.
*/
KASSERT(eq->avail > 0, ("%s: no space for eqflush.", __func__));
- Move all Ethernet specific items from sge_eq to sge_txq. sge_eq is now a suitable base for all kinds of egress queues. - Add control queues (sge_ctrlq) and allocate one of these per hardware channel. They can be used to program filters and steer traffic (and more). MFC after: 1 week
2011-04-19 22:08:28 +00:00
txsd->desc_used = 1;
txsd->credits = 0;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
write_eqflush_wr(eq);
- Move all Ethernet specific items from sge_eq to sge_txq. sge_eq is now a suitable base for all kinds of egress queues. - Add control queues (sge_ctrlq) and allocate one of these per hardware channel. They can be used to program filters and steer traffic (and more). MFC after: 1 week
2011-04-19 22:08:28 +00:00
}
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
txq->m = m;
if (eq->pending)
- Move all Ethernet specific items from sge_eq to sge_txq. sge_eq is now a suitable base for all kinds of egress queues. - Add control queues (sge_ctrlq) and allocate one of these per hardware channel. They can be used to program filters and steer traffic (and more). MFC after: 1 week
2011-04-19 22:08:28 +00:00
ring_eq_db(sc, eq);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
reclaim_tx_descs(txq, 0, 128);
if (eq->flags & EQ_STALLED && callout_pending(&eq->tx_callout) == 0)
callout_reset(&eq->tx_callout, 1, t4_tx_callout, eq);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
return (0);
}
void
t4_update_fl_bufsize(struct ifnet *ifp)
{
struct port_info *pi = ifp->if_softc;
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
struct adapter *sc = pi->adapter;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
struct sge_rxq *rxq;
- Make note of interface MTU change if the rx queues exist, and not just when the interface is up. - Add a tunable to control the TOE's rx coalesce feature (enabled by default as it always has been). Consider the interface MTU or the coalesce size when deciding which cluster zone to use to fill the offload rx queue's free list. The tunable is: dev.{t4nex,t5nex}.<N>.toe.rx_coalesce MFC after: 1 day
2013-07-04 21:19:01 +00:00
#ifdef TCP_OFFLOAD
struct sge_ofld_rxq *ofld_rxq;
#endif
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
struct sge_fl *fl;
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
int i, maxp, mtu = ifp->if_mtu;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
maxp = mtu_to_max_payload(sc, mtu, 0);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
for_each_rxq(pi, i, rxq) {
fl = &rxq->fl;
FL_LOCK(fl);
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
find_best_refill_source(sc, fl, maxp);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
FL_UNLOCK(fl);
}
- Make note of interface MTU change if the rx queues exist, and not just when the interface is up. - Add a tunable to control the TOE's rx coalesce feature (enabled by default as it always has been). Consider the interface MTU or the coalesce size when deciding which cluster zone to use to fill the offload rx queue's free list. The tunable is: dev.{t4nex,t5nex}.<N>.toe.rx_coalesce MFC after: 1 day
2013-07-04 21:19:01 +00:00
#ifdef TCP_OFFLOAD
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
maxp = mtu_to_max_payload(sc, mtu, 1);
- Make note of interface MTU change if the rx queues exist, and not just when the interface is up. - Add a tunable to control the TOE's rx coalesce feature (enabled by default as it always has been). Consider the interface MTU or the coalesce size when deciding which cluster zone to use to fill the offload rx queue's free list. The tunable is: dev.{t4nex,t5nex}.<N>.toe.rx_coalesce MFC after: 1 day
2013-07-04 21:19:01 +00:00
for_each_ofld_rxq(pi, i, ofld_rxq) {
fl = &ofld_rxq->fl;
FL_LOCK(fl);
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
find_best_refill_source(sc, fl, maxp);
- Make note of interface MTU change if the rx queues exist, and not just when the interface is up. - Add a tunable to control the TOE's rx coalesce feature (enabled by default as it always has been). Consider the interface MTU or the coalesce size when deciding which cluster zone to use to fill the offload rx queue's free list. The tunable is: dev.{t4nex,t5nex}.<N>.toe.rx_coalesce MFC after: 1 day
2013-07-04 21:19:01 +00:00
FL_UNLOCK(fl);
}
#endif
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
}
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
int
can_resume_tx(struct sge_eq *eq)
{
Consider the total number of descriptors available (and not just those that are ready to be reclaimed) when deciding whether to resume tx after a stall. MFC after: 3 days
2014-06-20 20:28:46 +00:00
return (eq->avail + reclaimable(eq) >= tx_resume_threshold(eq));
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
}
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
static inline void
init_iq(struct sge_iq *iq, struct adapter *sc, int tmr_idx, int pktc_idx,
Remove unused item. cxgbe's rx queue's lock was removed a long time ago. MFC after: 3 days
2012-10-10 16:52:39 +00:00
int qsize, int esize)
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
{
KASSERT(tmr_idx >= 0 && tmr_idx < SGE_NTIMERS,
("%s: bad tmr_idx %d", __func__, tmr_idx));
KASSERT(pktc_idx < SGE_NCOUNTERS, /* -ve is ok, means don't use */
("%s: bad pktc_idx %d", __func__, pktc_idx));
iq->flags = 0;
iq->adapter = sc;
Change the default to not use packet counters to generate rx interrupts. Rely solely on the timer based mechanism. Update man page to reflect this change. MFC after: 1 week
2012-04-30 09:46:05 +00:00
iq->intr_params = V_QINTR_TIMER_IDX(tmr_idx);
iq->intr_pktc_idx = SGE_NCOUNTERS - 1;
if (pktc_idx >= 0) {
iq->intr_params |= F_QINTR_CNT_EN;
iq->intr_pktc_idx = pktc_idx;
}
cxgbe(4): Add support for Chelsio's Terminator 5 (aka T5) ASIC. This includes support for the NIC and TOE features of the 40G, 10G, and 1G/100M cards based on the T5. The ASIC is mostly backward compatible with the Terminator 4 so cxgbe(4) has been updated instead of writing a brand new driver. T5 cards will show up as cxl (short for cxlgb) ports attached to the t5nex bus driver. Sponsored by: Chelsio
2013-03-30 02:26:20 +00:00
iq->qsize = roundup2(qsize, 16); /* See FW_IQ_CMD/iqsize */
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
iq->esize = max(esize, 16); /* See FW_IQ_CMD/iqesize */
}
static inline void
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
init_fl(struct adapter *sc, struct sge_fl *fl, int qsize, int maxp, int pack,
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
char *name)
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
{
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
fl->qsize = qsize;
strlcpy(fl->lockname, name, sizeof(fl->lockname));
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
if (pack)
fl->flags |= FL_BUF_PACKING;
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
find_best_refill_source(sc, fl, maxp);
find_safe_refill_source(sc, fl);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
}
static inline void
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
init_eq(struct sge_eq *eq, int eqtype, int qsize, uint8_t tx_chan,
uint16_t iqid, char *name)
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
{
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
KASSERT(tx_chan < NCHAN, ("%s: bad tx channel %d", __func__, tx_chan));
KASSERT(eqtype <= EQ_TYPEMASK, ("%s: bad qtype %d", __func__, eqtype));
eq->flags = eqtype & EQ_TYPEMASK;
eq->tx_chan = tx_chan;
eq->iqid = iqid;
- Move all Ethernet specific items from sge_eq to sge_txq. sge_eq is now a suitable base for all kinds of egress queues. - Add control queues (sge_ctrlq) and allocate one of these per hardware channel. They can be used to program filters and steer traffic (and more). MFC after: 1 week
2011-04-19 22:08:28 +00:00
eq->qsize = qsize;
strlcpy(eq->lockname, name, sizeof(eq->lockname));
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
TASK_INIT(&eq->tx_task, 0, t4_tx_task, eq);
callout_init(&eq->tx_callout, CALLOUT_MPSAFE);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
}
static int
alloc_ring(struct adapter *sc, size_t len, bus_dma_tag_t *tag,
bus_dmamap_t *map, bus_addr_t *pa, void **va)
{
int rc;
rc = bus_dma_tag_create(sc->dmat, 512, 0, BUS_SPACE_MAXADDR,
BUS_SPACE_MAXADDR, NULL, NULL, len, 1, len, 0, NULL, NULL, tag);
if (rc != 0) {
device_printf(sc->dev, "cannot allocate DMA tag: %d\n", rc);
goto done;
}
rc = bus_dmamem_alloc(*tag, va,
BUS_DMA_WAITOK | BUS_DMA_COHERENT | BUS_DMA_ZERO, map);
if (rc != 0) {
device_printf(sc->dev, "cannot allocate DMA memory: %d\n", rc);
goto done;
}
rc = bus_dmamap_load(*tag, *map, *va, len, oneseg_dma_callback, pa, 0);
if (rc != 0) {
device_printf(sc->dev, "cannot load DMA map: %d\n", rc);
goto done;
}
done:
if (rc)
free_ring(sc, *tag, *map, *pa, *va);
return (rc);
}
static int
free_ring(struct adapter *sc, bus_dma_tag_t tag, bus_dmamap_t map,
bus_addr_t pa, void *va)
{
if (pa)
bus_dmamap_unload(tag, map);
if (va)
bus_dmamem_free(tag, va, map);
if (tag)
bus_dma_tag_destroy(tag);
return (0);
}
/*
* Allocates the ring for an ingress queue and an optional freelist. If the
* freelist is specified it will be allocated and then associated with the
* ingress queue.
*
* Returns errno on failure. Resources allocated up to that point may still be
* allocated. Caller is responsible for cleanup in case this function fails.
*
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
* If the ingress queue will take interrupts directly (iq->flags & IQ_INTR) then
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
* the intr_idx specifies the vector, starting from 0. Otherwise it specifies
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
* the abs_id of the ingress queue to which its interrupts should be forwarded.
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
*/
static int
alloc_iq_fl(struct port_info *pi, struct sge_iq *iq, struct sge_fl *fl,
- Enable per-channel congestion notification. - Enable PCIe relaxed ordering for all egress queues and rx data buffers. MFC after: 3 days
2011-05-18 22:09:04 +00:00
int intr_idx, int cong)
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
{
int rc, i, cntxt_id;
size_t len;
struct fw_iq_cmd c;
struct adapter *sc = iq->adapter;
__be32 v = 0;
len = iq->qsize * iq->esize;
rc = alloc_ring(sc, len, &iq->desc_tag, &iq->desc_map, &iq->ba,
(void **)&iq->desc);
if (rc != 0)
return (rc);
bzero(&c, sizeof(c));
c.op_to_vfn = htobe32(V_FW_CMD_OP(FW_IQ_CMD) | F_FW_CMD_REQUEST |
F_FW_CMD_WRITE | F_FW_CMD_EXEC | V_FW_IQ_CMD_PFN(sc->pf) |
V_FW_IQ_CMD_VFN(0));
c.alloc_to_len16 = htobe32(F_FW_IQ_CMD_ALLOC | F_FW_IQ_CMD_IQSTART |
FW_LEN16(c));
/* Special handling for firmware event queue */
if (iq == &sc->sge.fwq)
v |= F_FW_IQ_CMD_IQASYNCH;
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
if (iq->flags & IQ_INTR) {
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
KASSERT(intr_idx < sc->intr_count,
("%s: invalid direct intr_idx %d", __func__, intr_idx));
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
} else
v |= F_FW_IQ_CMD_IQANDST;
v |= V_FW_IQ_CMD_IQANDSTINDEX(intr_idx);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
c.type_to_iqandstindex = htobe32(v |
V_FW_IQ_CMD_TYPE(FW_IQ_TYPE_FL_INT_CAP) |
V_FW_IQ_CMD_VIID(pi->viid) |
V_FW_IQ_CMD_IQANUD(X_UPDATEDELIVERY_INTERRUPT));
c.iqdroprss_to_iqesize = htobe16(V_FW_IQ_CMD_IQPCIECH(pi->tx_chan) |
F_FW_IQ_CMD_IQGTSMODE |
V_FW_IQ_CMD_IQINTCNTTHRESH(iq->intr_pktc_idx) |
V_FW_IQ_CMD_IQESIZE(ilog2(iq->esize) - 4));
c.iqsize = htobe16(iq->qsize);
c.iqaddr = htobe64(iq->ba);
- Enable per-channel congestion notification. - Enable PCIe relaxed ordering for all egress queues and rx data buffers. MFC after: 3 days
2011-05-18 22:09:04 +00:00
if (cong >= 0)
c.iqns_to_fl0congen = htobe32(F_FW_IQ_CMD_IQFLINTCONGEN);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
if (fl) {
mtx_init(&fl->fl_lock, fl->lockname, NULL, MTX_DEF);
len = fl->qsize * RX_FL_ESIZE;
rc = alloc_ring(sc, len, &fl->desc_tag, &fl->desc_map,
&fl->ba, (void **)&fl->desc);
if (rc)
return (rc);
/* Allocate space for one software descriptor per buffer. */
Better way to determine the status page length and rx pad boundary.
2012-06-23 22:12:27 +00:00
fl->cap = (fl->qsize - spg_len / RX_FL_ESIZE) * 8;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
rc = alloc_fl_sdesc(fl);
if (rc != 0) {
device_printf(sc->dev,
"failed to setup fl software descriptors: %d\n",
rc);
return (rc);
}
Ring the freelist doorbell from within refill_fl. While here, fix a bug that could have allowed the hardware pidx to reach the cidx even though the freelist isn't empty. (Haven't actually seen this but it was there waiting to happen..) MFC after: 1 week
2011-04-20 23:20:00 +00:00
fl->needed = fl->cap;
cxgbe(4): The T5 allows for a different freelist starvation threshold for queues with buffer packing. Use the correct value to calculate a freelist's low water mark. MFC after: 1 week
2014-02-06 03:21:43 +00:00
fl->lowat = fl->flags & FL_BUF_PACKING ?
roundup2(sc->sge.fl_starve_threshold2, 8) :
roundup2(sc->sge.fl_starve_threshold, 8);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
Do not clobber the ingress queue's congestion setting. MFC after: 1 month
2011-12-14 05:34:23 +00:00
c.iqns_to_fl0congen |=
- Enable per-channel congestion notification. - Enable PCIe relaxed ordering for all egress queues and rx data buffers. MFC after: 3 days
2011-05-18 22:09:04 +00:00
htobe32(V_FW_IQ_CMD_FL0HOSTFCMODE(X_HOSTFCMODE_NONE) |
F_FW_IQ_CMD_FL0FETCHRO | F_FW_IQ_CMD_FL0DATARO |
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
(fl_pad ? F_FW_IQ_CMD_FL0PADEN : 0) |
(fl->flags & FL_BUF_PACKING ? F_FW_IQ_CMD_FL0PACKEN :
0));
- Enable per-channel congestion notification. - Enable PCIe relaxed ordering for all egress queues and rx data buffers. MFC after: 3 days
2011-05-18 22:09:04 +00:00
if (cong >= 0) {
c.iqns_to_fl0congen |=
htobe32(V_FW_IQ_CMD_FL0CNGCHMAP(cong) |
F_FW_IQ_CMD_FL0CONGCIF |
F_FW_IQ_CMD_FL0CONGEN);
}
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
c.fl0dcaen_to_fl0cidxfthresh =
htobe16(V_FW_IQ_CMD_FL0FBMIN(X_FETCHBURSTMIN_64B) |
V_FW_IQ_CMD_FL0FBMAX(X_FETCHBURSTMAX_512B));
c.fl0size = htobe16(fl->qsize);
c.fl0addr = htobe64(fl->ba);
}
rc = -t4_wr_mbox(sc, sc->mbox, &c, sizeof(c), &c);
if (rc != 0) {
device_printf(sc->dev,
"failed to create ingress queue: %d\n", rc);
return (rc);
}
iq->cdesc = iq->desc;
iq->cidx = 0;
iq->gen = 1;
iq->intr_next = iq->intr_params;
iq->cntxt_id = be16toh(c.iqid);
iq->abs_id = be16toh(c.physiqid);
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
iq->flags |= IQ_ALLOCATED;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
cntxt_id = iq->cntxt_id - sc->sge.iq_start;
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
if (cntxt_id >= sc->sge.niq) {
panic ("%s: iq->cntxt_id (%d) more than the max (%d)", __func__,
cntxt_id, sc->sge.niq - 1);
}
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
sc->sge.iqmap[cntxt_id] = iq;
if (fl) {
fl->cntxt_id = be16toh(c.fl0id);
fl->pidx = fl->cidx = 0;
Fix an error while constructing the table that maps context id -> egress queue. MFC after: 1 day
2011-03-22 21:05:56 +00:00
cntxt_id = fl->cntxt_id - sc->sge.eq_start;
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
if (cntxt_id >= sc->sge.neq) {
panic("%s: fl->cntxt_id (%d) more than the max (%d)",
__func__, cntxt_id, sc->sge.neq - 1);
}
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
sc->sge.eqmap[cntxt_id] = (void *)fl;
FL_LOCK(fl);
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
/* Enough to make sure the SGE doesn't think it's starved */
refill_fl(sc, fl, fl->lowat);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
FL_UNLOCK(fl);
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
iq->flags |= IQ_HAS_FL;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
}
Set up congestion manager context properly for T5 based cards. MFC after: 3 days (will check with re@)
2013-08-01 23:38:30 +00:00
if (is_t5(sc) && cong >= 0) {
uint32_t param, val;
param = V_FW_PARAMS_MNEM(FW_PARAMS_MNEM_DMAQ) |
V_FW_PARAMS_PARAM_X(FW_PARAMS_PARAM_DMAQ_CONM_CTXT) |
V_FW_PARAMS_PARAM_YZ(iq->cntxt_id);
Fix previous commit (r253873). "cong" has one bit per channel but the congestion channel map has 1 nibble per channel. So bits wxyz need to be blown up into 000w000x000y000z.
2013-08-02 17:44:19 +00:00
if (cong == 0)
val = 1 << 19;
else {
val = 2 << 19;
for (i = 0; i < 4; i++) {
if (cong & (1 << i))
val |= 1 << (i << 2);
}
}
Set up congestion manager context properly for T5 based cards. MFC after: 3 days (will check with re@)
2013-08-01 23:38:30 +00:00
rc = -t4_set_params(sc, sc->mbox, sc->pf, 0, 1, &param, &val);
if (rc != 0) {
/* report error but carry on */
device_printf(sc->dev,
"failed to set congestion manager context for "
"ingress queue %d: %d\n", iq->cntxt_id, rc);
}
}
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
/* Enable IQ interrupts */
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
atomic_store_rel_int(&iq->state, IQS_IDLE);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
t4_write_reg(sc, MYPF_REG(A_SGE_PF_GTS), V_SEINTARM(iq->intr_params) |
V_INGRESSQID(iq->cntxt_id));
return (0);
}
static int
free_iq_fl(struct port_info *pi, struct sge_iq *iq, struct sge_fl *fl)
{
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
int rc;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
struct adapter *sc = iq->adapter;
device_t dev;
if (sc == NULL)
return (0); /* nothing to do */
dev = pi ? pi->dev : sc->dev;
if (iq->flags & IQ_ALLOCATED) {
rc = -t4_iq_free(sc, sc->mbox, sc->pf, 0,
FW_IQ_TYPE_FL_INT_CAP, iq->cntxt_id,
fl ? fl->cntxt_id : 0xffff, 0xffff);
if (rc != 0) {
device_printf(dev,
"failed to free queue %p: %d\n", iq, rc);
return (rc);
}
iq->flags &= ~IQ_ALLOCATED;
}
free_ring(sc, iq->desc_tag, iq->desc_map, iq->ba, iq->desc);
bzero(iq, sizeof(*iq));
if (fl) {
free_ring(sc, fl->desc_tag, fl->desc_map, fl->ba,
fl->desc);
There is no need to hold the freelist lock around alloc/free of software descriptors. This also silences WITNESS warnings when the software descriptors are allocated with M_WAITOK. MFC after: 1 week
2013-08-23 18:03:18 +00:00
if (fl->sdesc)
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
free_fl_sdesc(sc, fl);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
if (mtx_initialized(&fl->fl_lock))
mtx_destroy(&fl->fl_lock);
bzero(fl, sizeof(*fl));
}
return (0);
}
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
static void
add_fl_sysctls(struct sysctl_ctx_list *ctx, struct sysctl_oid *oid,
struct sge_fl *fl)
{
struct sysctl_oid_list *children = SYSCTL_CHILDREN(oid);
oid = SYSCTL_ADD_NODE(ctx, children, OID_AUTO, "fl", CTLFLAG_RD, NULL,
"freelist");
children = SYSCTL_CHILDREN(oid);
SYSCTL_ADD_PROC(ctx, children, OID_AUTO, "cntxt_id",
CTLTYPE_INT | CTLFLAG_RD, &fl->cntxt_id, 0, sysctl_uint16, "I",
"SGE context id of the freelist");
SYSCTL_ADD_UINT(ctx, children, OID_AUTO, "cidx", CTLFLAG_RD, &fl->cidx,
0, "consumer index");
if (fl->flags & FL_BUF_PACKING) {
SYSCTL_ADD_UINT(ctx, children, OID_AUTO, "rx_offset",
CTLFLAG_RD, &fl->rx_offset, 0, "packing rx offset");
}
SYSCTL_ADD_UINT(ctx, children, OID_AUTO, "pidx", CTLFLAG_RD, &fl->pidx,
0, "producer index");
SYSCTL_ADD_UQUAD(ctx, children, OID_AUTO, "mbuf_allocated",
CTLFLAG_RD, &fl->mbuf_allocated, "# of mbuf allocated");
SYSCTL_ADD_UQUAD(ctx, children, OID_AUTO, "mbuf_inlined",
CTLFLAG_RD, &fl->mbuf_inlined, "# of mbuf inlined in clusters");
SYSCTL_ADD_UQUAD(ctx, children, OID_AUTO, "cluster_allocated",
CTLFLAG_RD, &fl->cl_allocated, "# of clusters allocated");
SYSCTL_ADD_UQUAD(ctx, children, OID_AUTO, "cluster_recycled",
CTLFLAG_RD, &fl->cl_recycled, "# of clusters recycled");
SYSCTL_ADD_UQUAD(ctx, children, OID_AUTO, "cluster_fast_recycled",
CTLFLAG_RD, &fl->cl_fast_recycled, "# of clusters recycled (fast)");
}
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
static int
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
alloc_fwq(struct adapter *sc)
- Specialized ingress queues that take interrupts for other ingress queues. Try to have a set of these per port when possible, fall back to sharing a common pool between all ports otherwise. - One control queue per port (used to be one per hardware channel). - t4_eth_rx now handles Ethernet rx only. - sysctls to display pidx/cidx for some queues. MFC after: 1 week
2011-05-30 21:34:44 +00:00
{
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
int rc, intr_idx;
struct sge_iq *fwq = &sc->sge.fwq;
struct sysctl_oid *oid = device_get_sysctl_tree(sc->dev);
struct sysctl_oid_list *children = SYSCTL_CHILDREN(oid);
- Specialized ingress queues that take interrupts for other ingress queues. Try to have a set of these per port when possible, fall back to sharing a common pool between all ports otherwise. - One control queue per port (used to be one per hardware channel). - t4_eth_rx now handles Ethernet rx only. - sysctls to display pidx/cidx for some queues. MFC after: 1 week
2011-05-30 21:34:44 +00:00
Remove unused item. cxgbe's rx queue's lock was removed a long time ago. MFC after: 3 days
2012-10-10 16:52:39 +00:00
init_iq(fwq, sc, 0, 0, FW_IQ_QSIZE, FW_IQ_ESIZE);
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
fwq->flags |= IQ_INTR; /* always */
intr_idx = sc->intr_count > 1 ? 1 : 0;
rc = alloc_iq_fl(sc->port[0], fwq, NULL, intr_idx, -1);
if (rc != 0) {
device_printf(sc->dev,
"failed to create firmware event queue: %d\n", rc);
- Specialized ingress queues that take interrupts for other ingress queues. Try to have a set of these per port when possible, fall back to sharing a common pool between all ports otherwise. - One control queue per port (used to be one per hardware channel). - t4_eth_rx now handles Ethernet rx only. - sysctls to display pidx/cidx for some queues. MFC after: 1 week
2011-05-30 21:34:44 +00:00
return (rc);
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
}
- Specialized ingress queues that take interrupts for other ingress queues. Try to have a set of these per port when possible, fall back to sharing a common pool between all ports otherwise. - One control queue per port (used to be one per hardware channel). - t4_eth_rx now handles Ethernet rx only. - sysctls to display pidx/cidx for some queues. MFC after: 1 week
2011-05-30 21:34:44 +00:00
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
oid = SYSCTL_ADD_NODE(&sc->ctx, children, OID_AUTO, "fwq", CTLFLAG_RD,
NULL, "firmware event queue");
- Specialized ingress queues that take interrupts for other ingress queues. Try to have a set of these per port when possible, fall back to sharing a common pool between all ports otherwise. - One control queue per port (used to be one per hardware channel). - t4_eth_rx now handles Ethernet rx only. - sysctls to display pidx/cidx for some queues. MFC after: 1 week
2011-05-30 21:34:44 +00:00
children = SYSCTL_CHILDREN(oid);
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
SYSCTL_ADD_PROC(&sc->ctx, children, OID_AUTO, "abs_id",
CTLTYPE_INT | CTLFLAG_RD, &fwq->abs_id, 0, sysctl_uint16, "I",
"absolute id of the queue");
SYSCTL_ADD_PROC(&sc->ctx, children, OID_AUTO, "cntxt_id",
CTLTYPE_INT | CTLFLAG_RD, &fwq->cntxt_id, 0, sysctl_uint16, "I",
"SGE context id of the queue");
- Specialized ingress queues that take interrupts for other ingress queues. Try to have a set of these per port when possible, fall back to sharing a common pool between all ports otherwise. - One control queue per port (used to be one per hardware channel). - t4_eth_rx now handles Ethernet rx only. - sysctls to display pidx/cidx for some queues. MFC after: 1 week
2011-05-30 21:34:44 +00:00
SYSCTL_ADD_PROC(&sc->ctx, children, OID_AUTO, "cidx",
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
CTLTYPE_INT | CTLFLAG_RD, &fwq->cidx, 0, sysctl_uint16, "I",
- Specialized ingress queues that take interrupts for other ingress queues. Try to have a set of these per port when possible, fall back to sharing a common pool between all ports otherwise. - One control queue per port (used to be one per hardware channel). - t4_eth_rx now handles Ethernet rx only. - sysctls to display pidx/cidx for some queues. MFC after: 1 week
2011-05-30 21:34:44 +00:00
"consumer index");
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
return (0);
- Specialized ingress queues that take interrupts for other ingress queues. Try to have a set of these per port when possible, fall back to sharing a common pool between all ports otherwise. - One control queue per port (used to be one per hardware channel). - t4_eth_rx now handles Ethernet rx only. - sysctls to display pidx/cidx for some queues. MFC after: 1 week
2011-05-30 21:34:44 +00:00
}
static int
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
free_fwq(struct adapter *sc)
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
{
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
return free_iq_fl(NULL, &sc->sge.fwq, NULL);
- Specialized ingress queues that take interrupts for other ingress queues. Try to have a set of these per port when possible, fall back to sharing a common pool between all ports otherwise. - One control queue per port (used to be one per hardware channel). - t4_eth_rx now handles Ethernet rx only. - sysctls to display pidx/cidx for some queues. MFC after: 1 week
2011-05-30 21:34:44 +00:00
}
static int
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
alloc_mgmtq(struct adapter *sc)
- Specialized ingress queues that take interrupts for other ingress queues. Try to have a set of these per port when possible, fall back to sharing a common pool between all ports otherwise. - One control queue per port (used to be one per hardware channel). - t4_eth_rx now handles Ethernet rx only. - sysctls to display pidx/cidx for some queues. MFC after: 1 week
2011-05-30 21:34:44 +00:00
{
int rc;
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
struct sge_wrq *mgmtq = &sc->sge.mgmtq;
char name[16];
struct sysctl_oid *oid = device_get_sysctl_tree(sc->dev);
struct sysctl_oid_list *children = SYSCTL_CHILDREN(oid);
- Specialized ingress queues that take interrupts for other ingress queues. Try to have a set of these per port when possible, fall back to sharing a common pool between all ports otherwise. - One control queue per port (used to be one per hardware channel). - t4_eth_rx now handles Ethernet rx only. - sysctls to display pidx/cidx for some queues. MFC after: 1 week
2011-05-30 21:34:44 +00:00
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
oid = SYSCTL_ADD_NODE(&sc->ctx, children, OID_AUTO, "mgmtq", CTLFLAG_RD,
NULL, "management queue");
- Specialized ingress queues that take interrupts for other ingress queues. Try to have a set of these per port when possible, fall back to sharing a common pool between all ports otherwise. - One control queue per port (used to be one per hardware channel). - t4_eth_rx now handles Ethernet rx only. - sysctls to display pidx/cidx for some queues. MFC after: 1 week
2011-05-30 21:34:44 +00:00
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
snprintf(name, sizeof(name), "%s mgmtq", device_get_nameunit(sc->dev));
init_eq(&mgmtq->eq, EQ_CTRL, CTRL_EQ_QSIZE, sc->port[0]->tx_chan,
sc->sge.fwq.cntxt_id, name);
rc = alloc_wrq(sc, NULL, mgmtq, oid);
if (rc != 0) {
device_printf(sc->dev,
"failed to create management queue: %d\n", rc);
return (rc);
}
- Specialized ingress queues that take interrupts for other ingress queues. Try to have a set of these per port when possible, fall back to sharing a common pool between all ports otherwise. - One control queue per port (used to be one per hardware channel). - t4_eth_rx now handles Ethernet rx only. - sysctls to display pidx/cidx for some queues. MFC after: 1 week
2011-05-30 21:34:44 +00:00
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
return (0);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
}
static int
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
free_mgmtq(struct adapter *sc)
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
{
- Updated TOE support in the kernel. - Stateful TCP offload drivers for Terminator 3 and 4 (T3 and T4) ASICs. These are available as t3_tom and t4_tom modules that augment cxgb(4) and cxgbe(4) respectively. The cxgb/cxgbe drivers continue to work as usual with or without these extra features. - iWARP driver for Terminator 3 ASIC (kernel verbs). T4 iWARP in the works and will follow soon. Build-tested with make universe. 30s overview ============ What interfaces support TCP offload? Look for TOE4 and/or TOE6 in the capabilities of an interface: # ifconfig -m | grep TOE Enable/disable TCP offload on an interface (just like any other ifnet capability): # ifconfig cxgbe0 toe # ifconfig cxgbe0 -toe Which connections are offloaded? Look for toe4 and/or toe6 in the output of netstat and sockstat: # netstat -np tcp | grep toe # sockstat -46c | grep toe Reviewed by: bz, gnn Sponsored by: Chelsio communications. MFC after: ~3 months (after 9.1, and after ensuring MFC is feasible)
2012-06-19 07:34:13 +00:00
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
return free_wrq(sc, &sc->sge.mgmtq);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
}
Convert some fixed parameters to tunables (with reasonable default values). - cong_drop specifies what to do on congestion: nothing, backpressure, or drop. - fl_pktshift specifies the padding before Ethernet payload. - fl_pad specifies the boundary upto which to pad Ethernet payload. - spg_len controls the length of the status page. MFC after: 2 weeks
2012-08-14 21:47:41 +00:00
static inline int
tnl_cong(struct port_info *pi)
{
if (cong_drop == -1)
return (-1);
else if (cong_drop == 1)
return (0);
else
cxgbe(4): Use the rx channel map (instead of the tx channel map) as the congestion channel map. MFC after: 1 week
2014-02-06 03:30:12 +00:00
return (pi->rx_chan_map);
Convert some fixed parameters to tunables (with reasonable default values). - cong_drop specifies what to do on congestion: nothing, backpressure, or drop. - fl_pktshift specifies the padding before Ethernet payload. - fl_pad specifies the boundary upto which to pad Ethernet payload. - spg_len controls the length of the status page. MFC after: 2 weeks
2012-08-14 21:47:41 +00:00
}
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
static int
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
alloc_rxq(struct port_info *pi, struct sge_rxq *rxq, int intr_idx, int idx,
struct sysctl_oid *oid)
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
{
int rc;
struct sysctl_oid_list *children;
char name[16];
Convert some fixed parameters to tunables (with reasonable default values). - cong_drop specifies what to do on congestion: nothing, backpressure, or drop. - fl_pktshift specifies the padding before Ethernet payload. - fl_pad specifies the boundary upto which to pad Ethernet payload. - spg_len controls the length of the status page. MFC after: 2 weeks
2012-08-14 21:47:41 +00:00
rc = alloc_iq_fl(pi, &rxq->iq, &rxq->fl, intr_idx, tnl_cong(pi));
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
if (rc != 0)
return (rc);
Allow lazy fill up of freelists. MFC after: 3 days
2011-06-04 23:31:33 +00:00
FL_LOCK(&rxq->fl);
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
refill_fl(pi->adapter, &rxq->fl, rxq->fl.needed / 8);
Allow lazy fill up of freelists. MFC after: 3 days
2011-06-04 23:31:33 +00:00
FL_UNLOCK(&rxq->fl);
cxgbe(4): support for IPv6 TSO and LRO. Submitted by: bz (this is a modified version of that patch)
2012-06-29 19:51:06 +00:00
#if defined(INET) || defined(INET6)
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
rc = tcp_lro_init(&rxq->lro);
if (rc != 0)
return (rc);
rxq->lro.ifp = pi->ifp; /* also indicates LRO init'ed */
if (pi->ifp->if_capenable & IFCAP_LRO)
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
rxq->iq.flags |= IQ_LRO_ENABLED;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
#endif
Store the ifnet rather than the port_info in each txq and rxq struct. MFC after: 1 week
2011-03-05 03:27:14 +00:00
rxq->ifp = pi->ifp;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
children = SYSCTL_CHILDREN(oid);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
snprintf(name, sizeof(name), "%d", idx);
oid = SYSCTL_ADD_NODE(&pi->ctx, children, OID_AUTO, name, CTLFLAG_RD,
NULL, "rx queue");
children = SYSCTL_CHILDREN(oid);
sysctl that displays the absolute queue id of an rxq.
2011-05-14 19:27:15 +00:00
SYSCTL_ADD_PROC(&pi->ctx, children, OID_AUTO, "abs_id",
- Specialized ingress queues that take interrupts for other ingress queues. Try to have a set of these per port when possible, fall back to sharing a common pool between all ports otherwise. - One control queue per port (used to be one per hardware channel). - t4_eth_rx now handles Ethernet rx only. - sysctls to display pidx/cidx for some queues. MFC after: 1 week
2011-05-30 21:34:44 +00:00
CTLTYPE_INT | CTLFLAG_RD, &rxq->iq.abs_id, 0, sysctl_uint16, "I",
sysctl that displays the absolute queue id of an rxq.
2011-05-14 19:27:15 +00:00
"absolute id of the queue");
- driver ioctl to get SGE context for any given queue. - sysctls to display the context id, cidx, and pidx of all kinds of queues. MFC after: 3 days
2011-06-11 04:50:54 +00:00
SYSCTL_ADD_PROC(&pi->ctx, children, OID_AUTO, "cntxt_id",
CTLTYPE_INT | CTLFLAG_RD, &rxq->iq.cntxt_id, 0, sysctl_uint16, "I",
"SGE context id of the queue");
SYSCTL_ADD_PROC(&pi->ctx, children, OID_AUTO, "cidx",
CTLTYPE_INT | CTLFLAG_RD, &rxq->iq.cidx, 0, sysctl_uint16, "I",
"consumer index");
cxgbe(4): support for IPv6 TSO and LRO. Submitted by: bz (this is a modified version of that patch)
2012-06-29 19:51:06 +00:00
#if defined(INET) || defined(INET6)
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
SYSCTL_ADD_INT(&pi->ctx, children, OID_AUTO, "lro_queued", CTLFLAG_RD,
&rxq->lro.lro_queued, 0, NULL);
SYSCTL_ADD_INT(&pi->ctx, children, OID_AUTO, "lro_flushed", CTLFLAG_RD,
&rxq->lro.lro_flushed, 0, NULL);
Tweaks for rx: - everything related to LRO should be in #ifdef INET blocks - reorder sge_iq's fields so that the most frequently used are all together - pull all rx code into t4_intr_data directly - let go of the ingress queue lock when passing up data - refill the freelist only if it is short of at least 32 buffers
2011-03-05 03:42:03 +00:00
#endif
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
SYSCTL_ADD_UQUAD(&pi->ctx, children, OID_AUTO, "rxcsum", CTLFLAG_RD,
&rxq->rxcsum, "# of times hardware assisted with checksum");
SYSCTL_ADD_UQUAD(&pi->ctx, children, OID_AUTO, "vlan_extraction",
CTLFLAG_RD, &rxq->vlan_extraction,
"# of times hardware extracted 802.1Q tag");
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
add_fl_sysctls(&pi->ctx, oid, &rxq->fl);
- driver ioctl to get SGE context for any given queue. - sysctls to display the context id, cidx, and pidx of all kinds of queues. MFC after: 3 days
2011-06-11 04:50:54 +00:00
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
return (rc);
}
static int
free_rxq(struct port_info *pi, struct sge_rxq *rxq)
{
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
int rc;
cxgbe(4): support for IPv6 TSO and LRO. Submitted by: bz (this is a modified version of that patch)
2012-06-29 19:51:06 +00:00
#if defined(INET) || defined(INET6)
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
if (rxq->lro.ifp) {
tcp_lro_free(&rxq->lro);
rxq->lro.ifp = NULL;
}
#endif
rc = free_iq_fl(pi, &rxq->iq, &rxq->fl);
if (rc == 0)
bzero(rxq, sizeof(*rxq));
return (rc);
}
- Updated TOE support in the kernel. - Stateful TCP offload drivers for Terminator 3 and 4 (T3 and T4) ASICs. These are available as t3_tom and t4_tom modules that augment cxgb(4) and cxgbe(4) respectively. The cxgb/cxgbe drivers continue to work as usual with or without these extra features. - iWARP driver for Terminator 3 ASIC (kernel verbs). T4 iWARP in the works and will follow soon. Build-tested with make universe. 30s overview ============ What interfaces support TCP offload? Look for TOE4 and/or TOE6 in the capabilities of an interface: # ifconfig -m | grep TOE Enable/disable TCP offload on an interface (just like any other ifnet capability): # ifconfig cxgbe0 toe # ifconfig cxgbe0 -toe Which connections are offloaded? Look for toe4 and/or toe6 in the output of netstat and sockstat: # netstat -np tcp | grep toe # sockstat -46c | grep toe Reviewed by: bz, gnn Sponsored by: Chelsio communications. MFC after: ~3 months (after 9.1, and after ensuring MFC is feasible)
2012-06-19 07:34:13 +00:00
#ifdef TCP_OFFLOAD
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
static int
alloc_ofld_rxq(struct port_info *pi, struct sge_ofld_rxq *ofld_rxq,
int intr_idx, int idx, struct sysctl_oid *oid)
{
int rc;
struct sysctl_oid_list *children;
char name[16];
rc = alloc_iq_fl(pi, &ofld_rxq->iq, &ofld_rxq->fl, intr_idx,
cxgbe(4): Use the rx channel map (instead of the tx channel map) as the congestion channel map. MFC after: 1 week
2014-02-06 03:30:12 +00:00
pi->rx_chan_map);
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
if (rc != 0)
return (rc);
children = SYSCTL_CHILDREN(oid);
snprintf(name, sizeof(name), "%d", idx);
oid = SYSCTL_ADD_NODE(&pi->ctx, children, OID_AUTO, name, CTLFLAG_RD,
NULL, "rx queue");
children = SYSCTL_CHILDREN(oid);
SYSCTL_ADD_PROC(&pi->ctx, children, OID_AUTO, "abs_id",
CTLTYPE_INT | CTLFLAG_RD, &ofld_rxq->iq.abs_id, 0, sysctl_uint16,
"I", "absolute id of the queue");
SYSCTL_ADD_PROC(&pi->ctx, children, OID_AUTO, "cntxt_id",
CTLTYPE_INT | CTLFLAG_RD, &ofld_rxq->iq.cntxt_id, 0, sysctl_uint16,
"I", "SGE context id of the queue");
SYSCTL_ADD_PROC(&pi->ctx, children, OID_AUTO, "cidx",
CTLTYPE_INT | CTLFLAG_RD, &ofld_rxq->iq.cidx, 0, sysctl_uint16, "I",
"consumer index");
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
add_fl_sysctls(&pi->ctx, oid, &ofld_rxq->fl);
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
return (rc);
}
static int
free_ofld_rxq(struct port_info *pi, struct sge_ofld_rxq *ofld_rxq)
{
int rc;
rc = free_iq_fl(pi, &ofld_rxq->iq, &ofld_rxq->fl);
if (rc == 0)
bzero(ofld_rxq, sizeof(*ofld_rxq));
return (rc);
}
#endif
cxgbe(4): netmap support for Terminator 5 (T5) based 10G/40G cards. Netmap gets its own hardware-assisted virtual interface and won't take over or disrupt the "normal" interface in any way. You can use both simultaneously. For kernels with DEV_NETMAP, cxgbe(4) carves out an ncxl<N> interface (note the 'n' prefix) in the hardware to accompany each cxl<N> interface. These two ifnet's per port share the same wire but really are separate interfaces in the hardware and software. Each gets its own L2 MAC addresses (unicast and multicast), MTU, checksum caps, etc. You should run netmap on the 'n' interfaces only, that's what they are for. With this, pkt-gen is able to transmit > 45Mpps out of a single 40G port of a T580 card. 2 port tx is at ~56Mpps total (28M + 28M) as of now. Single port receive is at 33Mpps but this is very much a work in progress. I expect it to be closer to 40Mpps once done. In any case the current effort can already saturate multiple 10G ports of a T5 card at the smallest legal packet size. T4 gear is totally untested. trantor:~# ./pkt-gen -i ncxl0 -f tx -D 00:07:43:ab:cd:ef 881.952141 main [1621] interface is ncxl0 881.952250 extract_ip_range [275] range is 10.0.0.1:0 to 10.0.0.1:0 881.952253 extract_ip_range [275] range is 10.1.0.1:0 to 10.1.0.1:0 881.962540 main [1804] mapped 334980KB at 0x801dff000 Sending on netmap:ncxl0: 4 queues, 1 threads and 1 cpus. 10.0.0.1 -> 10.1.0.1 (00:00:00:00:00:00 -> 00:07:43:ab:cd:ef) 881.962562 main [1882] Sending 512 packets every 0.000000000 s 881.962563 main [1884] Wait 2 secs for phy reset 884.088516 main [1886] Ready... 884.088535 nm_open [457] overriding ifname ncxl0 ringid 0x0 flags 0x1 884.088607 sender_body [996] start 884.093246 sender_body [1064] drop copy 885.090435 main_thread [1418] 45206353 pps (45289533 pkts in 1001840 usec) 886.091600 main_thread [1418] 45322792 pps (45375593 pkts in 1001165 usec) 887.092435 main_thread [1418] 45313992 pps (45351784 pkts in 1000834 usec) 888.094434 main_thread [1418] 45315765 pps (45406397 pkts in 1002000 usec) 889.095434 main_thread [1418] 45333218 pps (45378551 pkts in 1001000 usec) 890.097434 main_thread [1418] 45315247 pps (45405877 pkts in 1002000 usec) 891.099434 main_thread [1418] 45326515 pps (45417168 pkts in 1002000 usec) 892.101434 main_thread [1418] 45333039 pps (45423705 pkts in 1002000 usec) 893.103434 main_thread [1418] 45324105 pps (45414708 pkts in 1001999 usec) 894.105434 main_thread [1418] 45318042 pps (45408723 pkts in 1002001 usec) 895.106434 main_thread [1418] 45332430 pps (45377762 pkts in 1001000 usec) 896.107434 main_thread [1418] 45338072 pps (45383410 pkts in 1001000 usec) ... Relnotes: Yes Sponsored by: Chelsio Communications.
2014-05-27 18:18:41 +00:00
#ifdef DEV_NETMAP
static int
alloc_nm_rxq(struct port_info *pi, struct sge_nm_rxq *nm_rxq, int intr_idx,
int idx, struct sysctl_oid *oid)
{
int rc;
struct sysctl_oid_list *children;
struct sysctl_ctx_list *ctx;
char name[16];
size_t len;
struct adapter *sc = pi->adapter;
struct netmap_adapter *na = NA(pi->nm_ifp);
MPASS(na != NULL);
len = pi->qsize_rxq * RX_IQ_ESIZE;
rc = alloc_ring(sc, len, &nm_rxq->iq_desc_tag, &nm_rxq->iq_desc_map,
&nm_rxq->iq_ba, (void **)&nm_rxq->iq_desc);
if (rc != 0)
return (rc);
len = na->num_rx_desc * RX_FL_ESIZE + spg_len;
rc = alloc_ring(sc, len, &nm_rxq->fl_desc_tag, &nm_rxq->fl_desc_map,
&nm_rxq->fl_ba, (void **)&nm_rxq->fl_desc);
if (rc != 0)
return (rc);
nm_rxq->pi = pi;
nm_rxq->nid = idx;
nm_rxq->iq_cidx = 0;
nm_rxq->iq_sidx = pi->qsize_rxq - spg_len / RX_IQ_ESIZE;
nm_rxq->iq_gen = F_RSPD_GEN;
nm_rxq->fl_pidx = nm_rxq->fl_cidx = 0;
nm_rxq->fl_sidx = na->num_rx_desc;
nm_rxq->intr_idx = intr_idx;
ctx = &pi->ctx;
children = SYSCTL_CHILDREN(oid);
snprintf(name, sizeof(name), "%d", idx);
oid = SYSCTL_ADD_NODE(ctx, children, OID_AUTO, name, CTLFLAG_RD, NULL,
"rx queue");
children = SYSCTL_CHILDREN(oid);
SYSCTL_ADD_PROC(ctx, children, OID_AUTO, "abs_id",
CTLTYPE_INT | CTLFLAG_RD, &nm_rxq->iq_abs_id, 0, sysctl_uint16,
"I", "absolute id of the queue");
SYSCTL_ADD_PROC(ctx, children, OID_AUTO, "cntxt_id",
CTLTYPE_INT | CTLFLAG_RD, &nm_rxq->iq_cntxt_id, 0, sysctl_uint16,
"I", "SGE context id of the queue");
SYSCTL_ADD_PROC(ctx, children, OID_AUTO, "cidx",
CTLTYPE_INT | CTLFLAG_RD, &nm_rxq->iq_cidx, 0, sysctl_uint16, "I",
"consumer index");
children = SYSCTL_CHILDREN(oid);
oid = SYSCTL_ADD_NODE(ctx, children, OID_AUTO, "fl", CTLFLAG_RD, NULL,
"freelist");
children = SYSCTL_CHILDREN(oid);
SYSCTL_ADD_PROC(ctx, children, OID_AUTO, "cntxt_id",
CTLTYPE_INT | CTLFLAG_RD, &nm_rxq->fl_cntxt_id, 0, sysctl_uint16,
"I", "SGE context id of the freelist");
SYSCTL_ADD_UINT(ctx, children, OID_AUTO, "cidx", CTLFLAG_RD,
&nm_rxq->fl_cidx, 0, "consumer index");
SYSCTL_ADD_UINT(ctx, children, OID_AUTO, "pidx", CTLFLAG_RD,
&nm_rxq->fl_pidx, 0, "producer index");
return (rc);
}
static int
free_nm_rxq(struct port_info *pi, struct sge_nm_rxq *nm_rxq)
{
struct adapter *sc = pi->adapter;
free_ring(sc, nm_rxq->iq_desc_tag, nm_rxq->iq_desc_map, nm_rxq->iq_ba,
nm_rxq->iq_desc);
free_ring(sc, nm_rxq->fl_desc_tag, nm_rxq->fl_desc_map, nm_rxq->fl_ba,
nm_rxq->fl_desc);
return (0);
}
static int
alloc_nm_txq(struct port_info *pi, struct sge_nm_txq *nm_txq, int iqidx, int idx,
struct sysctl_oid *oid)
{
int rc;
size_t len;
struct adapter *sc = pi->adapter;
struct netmap_adapter *na = NA(pi->nm_ifp);
char name[16];
struct sysctl_oid_list *children = SYSCTL_CHILDREN(oid);
len = na->num_tx_desc * EQ_ESIZE + spg_len;
rc = alloc_ring(sc, len, &nm_txq->desc_tag, &nm_txq->desc_map,
&nm_txq->ba, (void **)&nm_txq->desc);
if (rc)
return (rc);
nm_txq->pidx = nm_txq->cidx = 0;
nm_txq->sidx = na->num_tx_desc;
nm_txq->nid = idx;
nm_txq->iqidx = iqidx;
nm_txq->cpl_ctrl0 = htobe32(V_TXPKT_OPCODE(CPL_TX_PKT) |
V_TXPKT_INTF(pi->tx_chan) | V_TXPKT_PF(sc->pf));
snprintf(name, sizeof(name), "%d", idx);
oid = SYSCTL_ADD_NODE(&pi->ctx, children, OID_AUTO, name, CTLFLAG_RD,
NULL, "netmap tx queue");
children = SYSCTL_CHILDREN(oid);
SYSCTL_ADD_UINT(&pi->ctx, children, OID_AUTO, "cntxt_id", CTLFLAG_RD,
&nm_txq->cntxt_id, 0, "SGE context id of the queue");
SYSCTL_ADD_PROC(&pi->ctx, children, OID_AUTO, "cidx",
CTLTYPE_INT | CTLFLAG_RD, &nm_txq->cidx, 0, sysctl_uint16, "I",
"consumer index");
SYSCTL_ADD_PROC(&pi->ctx, children, OID_AUTO, "pidx",
CTLTYPE_INT | CTLFLAG_RD, &nm_txq->pidx, 0, sysctl_uint16, "I",
"producer index");
return (rc);
}
static int
free_nm_txq(struct port_info *pi, struct sge_nm_txq *nm_txq)
{
struct adapter *sc = pi->adapter;
free_ring(sc, nm_txq->desc_tag, nm_txq->desc_map, nm_txq->ba,
nm_txq->desc);
return (0);
}
#endif
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
static int
ctrl_eq_alloc(struct adapter *sc, struct sge_eq *eq)
{
int rc, cntxt_id;
struct fw_eq_ctrl_cmd c;
bzero(&c, sizeof(c));
c.op_to_vfn = htobe32(V_FW_CMD_OP(FW_EQ_CTRL_CMD) | F_FW_CMD_REQUEST |
F_FW_CMD_WRITE | F_FW_CMD_EXEC | V_FW_EQ_CTRL_CMD_PFN(sc->pf) |
V_FW_EQ_CTRL_CMD_VFN(0));
c.alloc_to_len16 = htobe32(F_FW_EQ_CTRL_CMD_ALLOC |
F_FW_EQ_CTRL_CMD_EQSTART | FW_LEN16(c));
c.cmpliqid_eqid = htonl(V_FW_EQ_CTRL_CMD_CMPLIQID(eq->iqid)); /* XXX */
c.physeqid_pkd = htobe32(0);
c.fetchszm_to_iqid =
htobe32(V_FW_EQ_CTRL_CMD_HOSTFCMODE(X_HOSTFCMODE_STATUS_PAGE) |
V_FW_EQ_CTRL_CMD_PCIECHN(eq->tx_chan) |
F_FW_EQ_CTRL_CMD_FETCHRO | V_FW_EQ_CTRL_CMD_IQID(eq->iqid));
c.dcaen_to_eqsize =
htobe32(V_FW_EQ_CTRL_CMD_FBMIN(X_FETCHBURSTMIN_64B) |
V_FW_EQ_CTRL_CMD_FBMAX(X_FETCHBURSTMAX_512B) |
V_FW_EQ_CTRL_CMD_CIDXFTHRESH(X_CIDXFLUSHTHRESH_32) |
V_FW_EQ_CTRL_CMD_EQSIZE(eq->qsize));
c.eqaddr = htobe64(eq->ba);
rc = -t4_wr_mbox(sc, sc->mbox, &c, sizeof(c), &c);
if (rc != 0) {
device_printf(sc->dev,
"failed to create control queue %d: %d\n", eq->tx_chan, rc);
return (rc);
}
eq->flags |= EQ_ALLOCATED;
eq->cntxt_id = G_FW_EQ_CTRL_CMD_EQID(be32toh(c.cmpliqid_eqid));
cntxt_id = eq->cntxt_id - sc->sge.eq_start;
if (cntxt_id >= sc->sge.neq)
panic("%s: eq->cntxt_id (%d) more than the max (%d)", __func__,
cntxt_id, sc->sge.neq - 1);
sc->sge.eqmap[cntxt_id] = eq;
return (rc);
}
static int
eth_eq_alloc(struct adapter *sc, struct port_info *pi, struct sge_eq *eq)
{
int rc, cntxt_id;
struct fw_eq_eth_cmd c;
bzero(&c, sizeof(c));
c.op_to_vfn = htobe32(V_FW_CMD_OP(FW_EQ_ETH_CMD) | F_FW_CMD_REQUEST |
F_FW_CMD_WRITE | F_FW_CMD_EXEC | V_FW_EQ_ETH_CMD_PFN(sc->pf) |
V_FW_EQ_ETH_CMD_VFN(0));
c.alloc_to_len16 = htobe32(F_FW_EQ_ETH_CMD_ALLOC |
F_FW_EQ_ETH_CMD_EQSTART | FW_LEN16(c));
cxgbe(4): Update the bundled T4 and T5 firmwares to versions 1.11.27.0. Obtained from: Chelsio MFC after: 3 days
2014-06-22 23:40:20 +00:00
c.autoequiqe_to_viid = htobe32(V_FW_EQ_ETH_CMD_VIID(pi->viid));
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
c.fetchszm_to_iqid =
htobe32(V_FW_EQ_ETH_CMD_HOSTFCMODE(X_HOSTFCMODE_STATUS_PAGE) |
V_FW_EQ_ETH_CMD_PCIECHN(eq->tx_chan) | F_FW_EQ_ETH_CMD_FETCHRO |
V_FW_EQ_ETH_CMD_IQID(eq->iqid));
c.dcaen_to_eqsize = htobe32(V_FW_EQ_ETH_CMD_FBMIN(X_FETCHBURSTMIN_64B) |
V_FW_EQ_ETH_CMD_FBMAX(X_FETCHBURSTMAX_512B) |
V_FW_EQ_ETH_CMD_CIDXFTHRESH(X_CIDXFLUSHTHRESH_32) |
V_FW_EQ_ETH_CMD_EQSIZE(eq->qsize));
c.eqaddr = htobe64(eq->ba);
rc = -t4_wr_mbox(sc, sc->mbox, &c, sizeof(c), &c);
if (rc != 0) {
device_printf(pi->dev,
"failed to create Ethernet egress queue: %d\n", rc);
return (rc);
}
eq->flags |= EQ_ALLOCATED;
eq->cntxt_id = G_FW_EQ_ETH_CMD_EQID(be32toh(c.eqid_pkd));
cntxt_id = eq->cntxt_id - sc->sge.eq_start;
if (cntxt_id >= sc->sge.neq)
panic("%s: eq->cntxt_id (%d) more than the max (%d)", __func__,
cntxt_id, sc->sge.neq - 1);
sc->sge.eqmap[cntxt_id] = eq;
return (rc);
}
- Updated TOE support in the kernel. - Stateful TCP offload drivers for Terminator 3 and 4 (T3 and T4) ASICs. These are available as t3_tom and t4_tom modules that augment cxgb(4) and cxgbe(4) respectively. The cxgb/cxgbe drivers continue to work as usual with or without these extra features. - iWARP driver for Terminator 3 ASIC (kernel verbs). T4 iWARP in the works and will follow soon. Build-tested with make universe. 30s overview ============ What interfaces support TCP offload? Look for TOE4 and/or TOE6 in the capabilities of an interface: # ifconfig -m | grep TOE Enable/disable TCP offload on an interface (just like any other ifnet capability): # ifconfig cxgbe0 toe # ifconfig cxgbe0 -toe Which connections are offloaded? Look for toe4 and/or toe6 in the output of netstat and sockstat: # netstat -np tcp | grep toe # sockstat -46c | grep toe Reviewed by: bz, gnn Sponsored by: Chelsio communications. MFC after: ~3 months (after 9.1, and after ensuring MFC is feasible)
2012-06-19 07:34:13 +00:00
#ifdef TCP_OFFLOAD
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
static int
ofld_eq_alloc(struct adapter *sc, struct port_info *pi, struct sge_eq *eq)
{
int rc, cntxt_id;
struct fw_eq_ofld_cmd c;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
bzero(&c, sizeof(c));
c.op_to_vfn = htonl(V_FW_CMD_OP(FW_EQ_OFLD_CMD) | F_FW_CMD_REQUEST |
F_FW_CMD_WRITE | F_FW_CMD_EXEC | V_FW_EQ_OFLD_CMD_PFN(sc->pf) |
V_FW_EQ_OFLD_CMD_VFN(0));
c.alloc_to_len16 = htonl(F_FW_EQ_OFLD_CMD_ALLOC |
F_FW_EQ_OFLD_CMD_EQSTART | FW_LEN16(c));
c.fetchszm_to_iqid =
htonl(V_FW_EQ_OFLD_CMD_HOSTFCMODE(X_HOSTFCMODE_STATUS_PAGE) |
V_FW_EQ_OFLD_CMD_PCIECHN(eq->tx_chan) |
F_FW_EQ_OFLD_CMD_FETCHRO | V_FW_EQ_OFLD_CMD_IQID(eq->iqid));
c.dcaen_to_eqsize =
htobe32(V_FW_EQ_OFLD_CMD_FBMIN(X_FETCHBURSTMIN_64B) |
V_FW_EQ_OFLD_CMD_FBMAX(X_FETCHBURSTMAX_512B) |
V_FW_EQ_OFLD_CMD_CIDXFTHRESH(X_CIDXFLUSHTHRESH_32) |
V_FW_EQ_OFLD_CMD_EQSIZE(eq->qsize));
c.eqaddr = htobe64(eq->ba);
rc = -t4_wr_mbox(sc, sc->mbox, &c, sizeof(c), &c);
if (rc != 0) {
device_printf(pi->dev,
"failed to create egress queue for TCP offload: %d\n", rc);
return (rc);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
}
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
eq->flags |= EQ_ALLOCATED;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
eq->cntxt_id = G_FW_EQ_OFLD_CMD_EQID(be32toh(c.eqid_pkd));
cntxt_id = eq->cntxt_id - sc->sge.eq_start;
if (cntxt_id >= sc->sge.neq)
panic("%s: eq->cntxt_id (%d) more than the max (%d)", __func__,
cntxt_id, sc->sge.neq - 1);
sc->sge.eqmap[cntxt_id] = eq;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
return (rc);
}
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
#endif
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
- Move all Ethernet specific items from sge_eq to sge_txq. sge_eq is now a suitable base for all kinds of egress queues. - Add control queues (sge_ctrlq) and allocate one of these per hardware channel. They can be used to program filters and steer traffic (and more). MFC after: 1 week
2011-04-19 22:08:28 +00:00
static int
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
alloc_eq(struct adapter *sc, struct port_info *pi, struct sge_eq *eq)
- Move all Ethernet specific items from sge_eq to sge_txq. sge_eq is now a suitable base for all kinds of egress queues. - Add control queues (sge_ctrlq) and allocate one of these per hardware channel. They can be used to program filters and steer traffic (and more). MFC after: 1 week
2011-04-19 22:08:28 +00:00
{
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
int rc;
- Move all Ethernet specific items from sge_eq to sge_txq. sge_eq is now a suitable base for all kinds of egress queues. - Add control queues (sge_ctrlq) and allocate one of these per hardware channel. They can be used to program filters and steer traffic (and more). MFC after: 1 week
2011-04-19 22:08:28 +00:00
size_t len;
mtx_init(&eq->eq_lock, eq->lockname, NULL, MTX_DEF);
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
len = eq->qsize * EQ_ESIZE;
- Move all Ethernet specific items from sge_eq to sge_txq. sge_eq is now a suitable base for all kinds of egress queues. - Add control queues (sge_ctrlq) and allocate one of these per hardware channel. They can be used to program filters and steer traffic (and more). MFC after: 1 week
2011-04-19 22:08:28 +00:00
rc = alloc_ring(sc, len, &eq->desc_tag, &eq->desc_map,
&eq->ba, (void **)&eq->desc);
if (rc)
return (rc);
Better way to determine the status page length and rx pad boundary.
2012-06-23 22:12:27 +00:00
eq->cap = eq->qsize - spg_len / EQ_ESIZE;
- Move all Ethernet specific items from sge_eq to sge_txq. sge_eq is now a suitable base for all kinds of egress queues. - Add control queues (sge_ctrlq) and allocate one of these per hardware channel. They can be used to program filters and steer traffic (and more). MFC after: 1 week
2011-04-19 22:08:28 +00:00
eq->spg = (void *)&eq->desc[eq->cap];
eq->avail = eq->cap - 1; /* one less to avoid cidx = pidx */
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
eq->pidx = eq->cidx = 0;
cxgbe(4): Add support for Chelsio's Terminator 5 (aka T5) ASIC. This includes support for the NIC and TOE features of the 40G, 10G, and 1G/100M cards based on the T5. The ASIC is mostly backward compatible with the Terminator 4 so cxgbe(4) has been updated instead of writing a brand new driver. T5 cards will show up as cxl (short for cxlgb) ports attached to the t5nex bus driver. Sponsored by: Chelsio
2013-03-30 02:26:20 +00:00
eq->doorbells = sc->doorbells;
- Move all Ethernet specific items from sge_eq to sge_txq. sge_eq is now a suitable base for all kinds of egress queues. - Add control queues (sge_ctrlq) and allocate one of these per hardware channel. They can be used to program filters and steer traffic (and more). MFC after: 1 week
2011-04-19 22:08:28 +00:00
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
switch (eq->flags & EQ_TYPEMASK) {
case EQ_CTRL:
rc = ctrl_eq_alloc(sc, eq);
break;
- Move all Ethernet specific items from sge_eq to sge_txq. sge_eq is now a suitable base for all kinds of egress queues. - Add control queues (sge_ctrlq) and allocate one of these per hardware channel. They can be used to program filters and steer traffic (and more). MFC after: 1 week
2011-04-19 22:08:28 +00:00
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
case EQ_ETH:
rc = eth_eq_alloc(sc, pi, eq);
break;
- Move all Ethernet specific items from sge_eq to sge_txq. sge_eq is now a suitable base for all kinds of egress queues. - Add control queues (sge_ctrlq) and allocate one of these per hardware channel. They can be used to program filters and steer traffic (and more). MFC after: 1 week
2011-04-19 22:08:28 +00:00
- Updated TOE support in the kernel. - Stateful TCP offload drivers for Terminator 3 and 4 (T3 and T4) ASICs. These are available as t3_tom and t4_tom modules that augment cxgb(4) and cxgbe(4) respectively. The cxgb/cxgbe drivers continue to work as usual with or without these extra features. - iWARP driver for Terminator 3 ASIC (kernel verbs). T4 iWARP in the works and will follow soon. Build-tested with make universe. 30s overview ============ What interfaces support TCP offload? Look for TOE4 and/or TOE6 in the capabilities of an interface: # ifconfig -m | grep TOE Enable/disable TCP offload on an interface (just like any other ifnet capability): # ifconfig cxgbe0 toe # ifconfig cxgbe0 -toe Which connections are offloaded? Look for toe4 and/or toe6 in the output of netstat and sockstat: # netstat -np tcp | grep toe # sockstat -46c | grep toe Reviewed by: bz, gnn Sponsored by: Chelsio communications. MFC after: ~3 months (after 9.1, and after ensuring MFC is feasible)
2012-06-19 07:34:13 +00:00
#ifdef TCP_OFFLOAD
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
case EQ_OFLD:
rc = ofld_eq_alloc(sc, pi, eq);
break;
#endif
default:
panic("%s: invalid eq type %d.", __func__,
eq->flags & EQ_TYPEMASK);
}
- Move all Ethernet specific items from sge_eq to sge_txq. sge_eq is now a suitable base for all kinds of egress queues. - Add control queues (sge_ctrlq) and allocate one of these per hardware channel. They can be used to program filters and steer traffic (and more). MFC after: 1 week
2011-04-19 22:08:28 +00:00
if (rc != 0) {
device_printf(sc->dev,
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
"failed to allocate egress queue(%d): %d",
eq->flags & EQ_TYPEMASK, rc);
- Move all Ethernet specific items from sge_eq to sge_txq. sge_eq is now a suitable base for all kinds of egress queues. - Add control queues (sge_ctrlq) and allocate one of these per hardware channel. They can be used to program filters and steer traffic (and more). MFC after: 1 week
2011-04-19 22:08:28 +00:00
}
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
eq->tx_callout.c_cpu = eq->cntxt_id % mp_ncpus;
- Move all Ethernet specific items from sge_eq to sge_txq. sge_eq is now a suitable base for all kinds of egress queues. - Add control queues (sge_ctrlq) and allocate one of these per hardware channel. They can be used to program filters and steer traffic (and more). MFC after: 1 week
2011-04-19 22:08:28 +00:00
cxgbe(4): Add support for Chelsio's Terminator 5 (aka T5) ASIC. This includes support for the NIC and TOE features of the 40G, 10G, and 1G/100M cards based on the T5. The ASIC is mostly backward compatible with the Terminator 4 so cxgbe(4) has been updated instead of writing a brand new driver. T5 cards will show up as cxl (short for cxlgb) ports attached to the t5nex bus driver. Sponsored by: Chelsio
2013-03-30 02:26:20 +00:00
if (isset(&eq->doorbells, DOORBELL_UDB) ||
isset(&eq->doorbells, DOORBELL_UDBWC) ||
Cosmetic change (s/wrwc/wcwr/;s/WRWC/WCWR/). MFC after: 3 days.
2013-04-11 22:49:29 +00:00
isset(&eq->doorbells, DOORBELL_WCWR)) {
cxgbe(4): Store the log2 of the # of doorbells per BAR2 page for both ingress and egress queues, and for both T4 and T5. These values are used by the T4/T5 iWARP driver.
2013-10-14 23:32:56 +00:00
uint32_t s_qpp = sc->sge.eq_s_qpp;
cxgbe(4): Add support for Chelsio's Terminator 5 (aka T5) ASIC. This includes support for the NIC and TOE features of the 40G, 10G, and 1G/100M cards based on the T5. The ASIC is mostly backward compatible with the Terminator 4 so cxgbe(4) has been updated instead of writing a brand new driver. T5 cards will show up as cxl (short for cxlgb) ports attached to the t5nex bus driver. Sponsored by: Chelsio
2013-03-30 02:26:20 +00:00
uint32_t mask = (1 << s_qpp) - 1;
volatile uint8_t *udb;
udb = sc->udbs_base + UDBS_DB_OFFSET;
udb += (eq->cntxt_id >> s_qpp) << PAGE_SHIFT; /* pg offset */
eq->udb_qid = eq->cntxt_id & mask; /* id in page */
if (eq->udb_qid > PAGE_SIZE / UDBS_SEG_SIZE)
Cosmetic change (s/wrwc/wcwr/;s/WRWC/WCWR/). MFC after: 3 days.
2013-04-11 22:49:29 +00:00
clrbit(&eq->doorbells, DOORBELL_WCWR);
cxgbe(4): Add support for Chelsio's Terminator 5 (aka T5) ASIC. This includes support for the NIC and TOE features of the 40G, 10G, and 1G/100M cards based on the T5. The ASIC is mostly backward compatible with the Terminator 4 so cxgbe(4) has been updated instead of writing a brand new driver. T5 cards will show up as cxl (short for cxlgb) ports attached to the t5nex bus driver. Sponsored by: Chelsio
2013-03-30 02:26:20 +00:00
else {
udb += eq->udb_qid << UDBS_SEG_SHIFT; /* seg offset */
eq->udb_qid = 0;
}
eq->udb = (volatile void *)udb;
}
- Move all Ethernet specific items from sge_eq to sge_txq. sge_eq is now a suitable base for all kinds of egress queues. - Add control queues (sge_ctrlq) and allocate one of these per hardware channel. They can be used to program filters and steer traffic (and more). MFC after: 1 week
2011-04-19 22:08:28 +00:00
return (rc);
}
static int
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
free_eq(struct adapter *sc, struct sge_eq *eq)
- Move all Ethernet specific items from sge_eq to sge_txq. sge_eq is now a suitable base for all kinds of egress queues. - Add control queues (sge_ctrlq) and allocate one of these per hardware channel. They can be used to program filters and steer traffic (and more). MFC after: 1 week
2011-04-19 22:08:28 +00:00
{
int rc;
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
if (eq->flags & EQ_ALLOCATED) {
switch (eq->flags & EQ_TYPEMASK) {
case EQ_CTRL:
rc = -t4_ctrl_eq_free(sc, sc->mbox, sc->pf, 0,
eq->cntxt_id);
break;
case EQ_ETH:
rc = -t4_eth_eq_free(sc, sc->mbox, sc->pf, 0,
eq->cntxt_id);
break;
- Updated TOE support in the kernel. - Stateful TCP offload drivers for Terminator 3 and 4 (T3 and T4) ASICs. These are available as t3_tom and t4_tom modules that augment cxgb(4) and cxgbe(4) respectively. The cxgb/cxgbe drivers continue to work as usual with or without these extra features. - iWARP driver for Terminator 3 ASIC (kernel verbs). T4 iWARP in the works and will follow soon. Build-tested with make universe. 30s overview ============ What interfaces support TCP offload? Look for TOE4 and/or TOE6 in the capabilities of an interface: # ifconfig -m | grep TOE Enable/disable TCP offload on an interface (just like any other ifnet capability): # ifconfig cxgbe0 toe # ifconfig cxgbe0 -toe Which connections are offloaded? Look for toe4 and/or toe6 in the output of netstat and sockstat: # netstat -np tcp | grep toe # sockstat -46c | grep toe Reviewed by: bz, gnn Sponsored by: Chelsio communications. MFC after: ~3 months (after 9.1, and after ensuring MFC is feasible)
2012-06-19 07:34:13 +00:00
#ifdef TCP_OFFLOAD
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
case EQ_OFLD:
rc = -t4_ofld_eq_free(sc, sc->mbox, sc->pf, 0,
eq->cntxt_id);
break;
#endif
default:
panic("%s: invalid eq type %d.", __func__,
eq->flags & EQ_TYPEMASK);
}
- Move all Ethernet specific items from sge_eq to sge_txq. sge_eq is now a suitable base for all kinds of egress queues. - Add control queues (sge_ctrlq) and allocate one of these per hardware channel. They can be used to program filters and steer traffic (and more). MFC after: 1 week
2011-04-19 22:08:28 +00:00
if (rc != 0) {
device_printf(sc->dev,
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
"failed to free egress queue (%d): %d\n",
eq->flags & EQ_TYPEMASK, rc);
- Move all Ethernet specific items from sge_eq to sge_txq. sge_eq is now a suitable base for all kinds of egress queues. - Add control queues (sge_ctrlq) and allocate one of these per hardware channel. They can be used to program filters and steer traffic (and more). MFC after: 1 week
2011-04-19 22:08:28 +00:00
return (rc);
}
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
eq->flags &= ~EQ_ALLOCATED;
- Move all Ethernet specific items from sge_eq to sge_txq. sge_eq is now a suitable base for all kinds of egress queues. - Add control queues (sge_ctrlq) and allocate one of these per hardware channel. They can be used to program filters and steer traffic (and more). MFC after: 1 week
2011-04-19 22:08:28 +00:00
}
free_ring(sc, eq->desc_tag, eq->desc_map, eq->ba, eq->desc);
if (mtx_initialized(&eq->eq_lock))
mtx_destroy(&eq->eq_lock);
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
bzero(eq, sizeof(*eq));
- Move all Ethernet specific items from sge_eq to sge_txq. sge_eq is now a suitable base for all kinds of egress queues. - Add control queues (sge_ctrlq) and allocate one of these per hardware channel. They can be used to program filters and steer traffic (and more). MFC after: 1 week
2011-04-19 22:08:28 +00:00
return (0);
}
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
static int
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
alloc_wrq(struct adapter *sc, struct port_info *pi, struct sge_wrq *wrq,
struct sysctl_oid *oid)
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
{
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
int rc;
struct sysctl_ctx_list *ctx = pi ? &pi->ctx : &sc->ctx;
struct sysctl_oid_list *children = SYSCTL_CHILDREN(oid);
rc = alloc_eq(sc, pi, &wrq->eq);
if (rc)
return (rc);
wrq->adapter = sc;
- Updated TOE support in the kernel. - Stateful TCP offload drivers for Terminator 3 and 4 (T3 and T4) ASICs. These are available as t3_tom and t4_tom modules that augment cxgb(4) and cxgbe(4) respectively. The cxgb/cxgbe drivers continue to work as usual with or without these extra features. - iWARP driver for Terminator 3 ASIC (kernel verbs). T4 iWARP in the works and will follow soon. Build-tested with make universe. 30s overview ============ What interfaces support TCP offload? Look for TOE4 and/or TOE6 in the capabilities of an interface: # ifconfig -m | grep TOE Enable/disable TCP offload on an interface (just like any other ifnet capability): # ifconfig cxgbe0 toe # ifconfig cxgbe0 -toe Which connections are offloaded? Look for toe4 and/or toe6 in the output of netstat and sockstat: # netstat -np tcp | grep toe # sockstat -46c | grep toe Reviewed by: bz, gnn Sponsored by: Chelsio communications. MFC after: ~3 months (after 9.1, and after ensuring MFC is feasible)
2012-06-19 07:34:13 +00:00
STAILQ_INIT(&wrq->wr_list);
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
SYSCTL_ADD_UINT(ctx, children, OID_AUTO, "cntxt_id", CTLFLAG_RD,
&wrq->eq.cntxt_id, 0, "SGE context id of the queue");
SYSCTL_ADD_PROC(ctx, children, OID_AUTO, "cidx",
CTLTYPE_INT | CTLFLAG_RD, &wrq->eq.cidx, 0, sysctl_uint16, "I",
"consumer index");
SYSCTL_ADD_PROC(ctx, children, OID_AUTO, "pidx",
CTLTYPE_INT | CTLFLAG_RD, &wrq->eq.pidx, 0, sysctl_uint16, "I",
"producer index");
SYSCTL_ADD_UQUAD(ctx, children, OID_AUTO, "tx_wrs", CTLFLAG_RD,
&wrq->tx_wrs, "# of work requests");
SYSCTL_ADD_UINT(ctx, children, OID_AUTO, "no_desc", CTLFLAG_RD,
&wrq->no_desc, 0,
"# of times queue ran out of hardware descriptors");
SYSCTL_ADD_UINT(ctx, children, OID_AUTO, "unstalled", CTLFLAG_RD,
&wrq->eq.unstalled, 0, "# of times queue recovered after stall");
return (rc);
}
static int
free_wrq(struct adapter *sc, struct sge_wrq *wrq)
{
int rc;
rc = free_eq(sc, &wrq->eq);
if (rc)
return (rc);
bzero(wrq, sizeof(*wrq));
return (0);
}
static int
alloc_txq(struct port_info *pi, struct sge_txq *txq, int idx,
struct sysctl_oid *oid)
{
int rc;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
struct adapter *sc = pi->adapter;
struct sge_eq *eq = &txq->eq;
char name[16];
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
struct sysctl_oid_list *children = SYSCTL_CHILDREN(oid);
Resume tx immediately in response to an SGE egress update from the hardware. MFC after: 1 week
2011-03-05 03:06:38 +00:00
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
rc = alloc_eq(sc, pi, eq);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
if (rc)
return (rc);
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
txq->ifp = pi->ifp;
- Move all Ethernet specific items from sge_eq to sge_txq. sge_eq is now a suitable base for all kinds of egress queues. - Add control queues (sge_ctrlq) and allocate one of these per hardware channel. They can be used to program filters and steer traffic (and more). MFC after: 1 week
2011-04-19 22:08:28 +00:00
txq->sdesc = malloc(eq->cap * sizeof(struct tx_sdesc), M_CXGBE,
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
M_ZERO | M_WAITOK);
- Move all Ethernet specific items from sge_eq to sge_txq. sge_eq is now a suitable base for all kinds of egress queues. - Add control queues (sge_ctrlq) and allocate one of these per hardware channel. They can be used to program filters and steer traffic (and more). MFC after: 1 week
2011-04-19 22:08:28 +00:00
txq->br = buf_ring_alloc(eq->qsize, M_CXGBE, M_WAITOK, &eq->eq_lock);
- Specialized ingress queues that take interrupts for other ingress queues. Try to have a set of these per port when possible, fall back to sharing a common pool between all ports otherwise. - One control queue per port (used to be one per hardware channel). - t4_eth_rx now handles Ethernet rx only. - sysctls to display pidx/cidx for some queues. MFC after: 1 week
2011-05-30 21:34:44 +00:00
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
rc = bus_dma_tag_create(sc->dmat, 1, 0, BUS_SPACE_MAXADDR,
BUS_SPACE_MAXADDR, NULL, NULL, 64 * 1024, TX_SGL_SEGS,
- Move all Ethernet specific items from sge_eq to sge_txq. sge_eq is now a suitable base for all kinds of egress queues. - Add control queues (sge_ctrlq) and allocate one of these per hardware channel. They can be used to program filters and steer traffic (and more). MFC after: 1 week
2011-04-19 22:08:28 +00:00
BUS_SPACE_MAXSIZE, BUS_DMA_ALLOCNOW, NULL, NULL, &txq->tx_tag);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
if (rc != 0) {
device_printf(sc->dev,
"failed to create tx DMA tag: %d\n", rc);
return (rc);
}
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
/*
* We can stuff ~10 frames in an 8-descriptor txpkts WR (8 is the SGE
* limit for any WR). txq->no_dmamap events shouldn't occur if maps is
* sized for the worst case.
*/
rc = t4_alloc_tx_maps(&txq->txmaps, txq->tx_tag, eq->qsize * 10 / 8,
M_WAITOK);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
if (rc != 0) {
device_printf(sc->dev, "failed to setup tx DMA maps: %d\n", rc);
return (rc);
}
snprintf(name, sizeof(name), "%d", idx);
oid = SYSCTL_ADD_NODE(&pi->ctx, children, OID_AUTO, name, CTLFLAG_RD,
NULL, "tx queue");
children = SYSCTL_CHILDREN(oid);
- driver ioctl to get SGE context for any given queue. - sysctls to display the context id, cidx, and pidx of all kinds of queues. MFC after: 3 days
2011-06-11 04:50:54 +00:00
SYSCTL_ADD_UINT(&pi->ctx, children, OID_AUTO, "cntxt_id", CTLFLAG_RD,
&eq->cntxt_id, 0, "SGE context id of the queue");
SYSCTL_ADD_PROC(&pi->ctx, children, OID_AUTO, "cidx",
CTLTYPE_INT | CTLFLAG_RD, &eq->cidx, 0, sysctl_uint16, "I",
"consumer index");
SYSCTL_ADD_PROC(&pi->ctx, children, OID_AUTO, "pidx",
CTLTYPE_INT | CTLFLAG_RD, &eq->pidx, 0, sysctl_uint16, "I",
"producer index");
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
SYSCTL_ADD_UQUAD(&pi->ctx, children, OID_AUTO, "txcsum", CTLFLAG_RD,
&txq->txcsum, "# of times hardware assisted with checksum");
SYSCTL_ADD_UQUAD(&pi->ctx, children, OID_AUTO, "vlan_insertion",
CTLFLAG_RD, &txq->vlan_insertion,
"# of times hardware inserted 802.1Q tag");
SYSCTL_ADD_UQUAD(&pi->ctx, children, OID_AUTO, "tso_wrs", CTLFLAG_RD,
cxgbe(4): support for IPv6 TSO and LRO. Submitted by: bz (this is a modified version of that patch)
2012-06-29 19:51:06 +00:00
&txq->tso_wrs, "# of TSO work requests");
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
SYSCTL_ADD_UQUAD(&pi->ctx, children, OID_AUTO, "imm_wrs", CTLFLAG_RD,
&txq->imm_wrs, "# of work requests with immediate data");
SYSCTL_ADD_UQUAD(&pi->ctx, children, OID_AUTO, "sgl_wrs", CTLFLAG_RD,
&txq->sgl_wrs, "# of work requests with direct SGL");
SYSCTL_ADD_UQUAD(&pi->ctx, children, OID_AUTO, "txpkt_wrs", CTLFLAG_RD,
&txq->txpkt_wrs, "# of txpkt work requests (one pkt/WR)");
SYSCTL_ADD_UQUAD(&pi->ctx, children, OID_AUTO, "txpkts_wrs", CTLFLAG_RD,
&txq->txpkts_wrs, "# of txpkts work requests (multiple pkts/WR)");
SYSCTL_ADD_UQUAD(&pi->ctx, children, OID_AUTO, "txpkts_pkts", CTLFLAG_RD,
&txq->txpkts_pkts, "# of frames tx'd using txpkts work requests");
Provide a statistic to track the number of drops in each of the port's txq's buf_ring. The aggregate for all the queues of a port is already provided in ifnet->if_snd.ifq_drops. MFC after: 3 days.
2013-01-29 20:59:22 +00:00
SYSCTL_ADD_UQUAD(&pi->ctx, children, OID_AUTO, "br_drops", CTLFLAG_RD,
&txq->br->br_drops, "# of drops in the buf_ring for this queue");
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
SYSCTL_ADD_UINT(&pi->ctx, children, OID_AUTO, "no_dmamap", CTLFLAG_RD,
&txq->no_dmamap, 0, "# of times txq ran out of DMA maps");
SYSCTL_ADD_UINT(&pi->ctx, children, OID_AUTO, "no_desc", CTLFLAG_RD,
&txq->no_desc, 0, "# of times txq ran out of hardware descriptors");
SYSCTL_ADD_UINT(&pi->ctx, children, OID_AUTO, "egr_update", CTLFLAG_RD,
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
&eq->egr_update, 0, "egress update notifications from the SGE");
SYSCTL_ADD_UINT(&pi->ctx, children, OID_AUTO, "unstalled", CTLFLAG_RD,
&eq->unstalled, 0, "# of times txq recovered after stall");
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
return (rc);
}
static int
free_txq(struct port_info *pi, struct sge_txq *txq)
{
int rc;
struct adapter *sc = pi->adapter;
struct sge_eq *eq = &txq->eq;
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
rc = free_eq(sc, eq);
if (rc)
return (rc);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
- Move all Ethernet specific items from sge_eq to sge_txq. sge_eq is now a suitable base for all kinds of egress queues. - Add control queues (sge_ctrlq) and allocate one of these per hardware channel. They can be used to program filters and steer traffic (and more). MFC after: 1 week
2011-04-19 22:08:28 +00:00
free(txq->sdesc, M_CXGBE);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
if (txq->txmaps.maps)
t4_free_tx_maps(&txq->txmaps, txq->tx_tag);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
- Move all Ethernet specific items from sge_eq to sge_txq. sge_eq is now a suitable base for all kinds of egress queues. - Add control queues (sge_ctrlq) and allocate one of these per hardware channel. They can be used to program filters and steer traffic (and more). MFC after: 1 week
2011-04-19 22:08:28 +00:00
buf_ring_free(txq->br, M_CXGBE);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
- Move all Ethernet specific items from sge_eq to sge_txq. sge_eq is now a suitable base for all kinds of egress queues. - Add control queues (sge_ctrlq) and allocate one of these per hardware channel. They can be used to program filters and steer traffic (and more). MFC after: 1 week
2011-04-19 22:08:28 +00:00
if (txq->tx_tag)
bus_dma_tag_destroy(txq->tx_tag);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
bzero(txq, sizeof(*txq));
return (0);
}
static void
oneseg_dma_callback(void *arg, bus_dma_segment_t *segs, int nseg, int error)
{
bus_addr_t *ba = arg;
KASSERT(nseg == 1,
("%s meant for single segment mappings only.", __func__));
*ba = error ? 0 : segs->ds_addr;
}
static inline bool
is_new_response(const struct sge_iq *iq, struct rsp_ctrl **ctrl)
{
*ctrl = (void *)((uintptr_t)iq->cdesc +
(iq->esize - sizeof(struct rsp_ctrl)));
return (((*ctrl)->u.type_gen >> S_RSPD_GEN) == iq->gen);
}
static inline void
iq_next(struct sge_iq *iq)
{
iq->cdesc = (void *) ((uintptr_t)iq->cdesc + iq->esize);
cxgbe(4): netmap support for Terminator 5 (T5) based 10G/40G cards. Netmap gets its own hardware-assisted virtual interface and won't take over or disrupt the "normal" interface in any way. You can use both simultaneously. For kernels with DEV_NETMAP, cxgbe(4) carves out an ncxl<N> interface (note the 'n' prefix) in the hardware to accompany each cxl<N> interface. These two ifnet's per port share the same wire but really are separate interfaces in the hardware and software. Each gets its own L2 MAC addresses (unicast and multicast), MTU, checksum caps, etc. You should run netmap on the 'n' interfaces only, that's what they are for. With this, pkt-gen is able to transmit > 45Mpps out of a single 40G port of a T580 card. 2 port tx is at ~56Mpps total (28M + 28M) as of now. Single port receive is at 33Mpps but this is very much a work in progress. I expect it to be closer to 40Mpps once done. In any case the current effort can already saturate multiple 10G ports of a T5 card at the smallest legal packet size. T4 gear is totally untested. trantor:~# ./pkt-gen -i ncxl0 -f tx -D 00:07:43:ab:cd:ef 881.952141 main [1621] interface is ncxl0 881.952250 extract_ip_range [275] range is 10.0.0.1:0 to 10.0.0.1:0 881.952253 extract_ip_range [275] range is 10.1.0.1:0 to 10.1.0.1:0 881.962540 main [1804] mapped 334980KB at 0x801dff000 Sending on netmap:ncxl0: 4 queues, 1 threads and 1 cpus. 10.0.0.1 -> 10.1.0.1 (00:00:00:00:00:00 -> 00:07:43:ab:cd:ef) 881.962562 main [1882] Sending 512 packets every 0.000000000 s 881.962563 main [1884] Wait 2 secs for phy reset 884.088516 main [1886] Ready... 884.088535 nm_open [457] overriding ifname ncxl0 ringid 0x0 flags 0x1 884.088607 sender_body [996] start 884.093246 sender_body [1064] drop copy 885.090435 main_thread [1418] 45206353 pps (45289533 pkts in 1001840 usec) 886.091600 main_thread [1418] 45322792 pps (45375593 pkts in 1001165 usec) 887.092435 main_thread [1418] 45313992 pps (45351784 pkts in 1000834 usec) 888.094434 main_thread [1418] 45315765 pps (45406397 pkts in 1002000 usec) 889.095434 main_thread [1418] 45333218 pps (45378551 pkts in 1001000 usec) 890.097434 main_thread [1418] 45315247 pps (45405877 pkts in 1002000 usec) 891.099434 main_thread [1418] 45326515 pps (45417168 pkts in 1002000 usec) 892.101434 main_thread [1418] 45333039 pps (45423705 pkts in 1002000 usec) 893.103434 main_thread [1418] 45324105 pps (45414708 pkts in 1001999 usec) 894.105434 main_thread [1418] 45318042 pps (45408723 pkts in 1002001 usec) 895.106434 main_thread [1418] 45332430 pps (45377762 pkts in 1001000 usec) 896.107434 main_thread [1418] 45338072 pps (45383410 pkts in 1001000 usec) ... Relnotes: Yes Sponsored by: Chelsio Communications.
2014-05-27 18:18:41 +00:00
if (__predict_false(++iq->cidx == iq->qsize - spg_len / iq->esize)) {
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
iq->cidx = 0;
iq->gen ^= 1;
iq->cdesc = iq->desc;
}
}
Ring the freelist doorbell from within refill_fl. While here, fix a bug that could have allowed the hardware pidx to reach the cidx even though the freelist isn't empty. (Haven't actually seen this but it was there waiting to happen..) MFC after: 1 week
2011-04-20 23:20:00 +00:00
#define FL_HW_IDX(x) ((x) >> 3)
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
static inline void
ring_fl_db(struct adapter *sc, struct sge_fl *fl)
{
int ndesc = fl->pending / 8;
cxgbe(4): Add support for Chelsio's Terminator 5 (aka T5) ASIC. This includes support for the NIC and TOE features of the 40G, 10G, and 1G/100M cards based on the T5. The ASIC is mostly backward compatible with the Terminator 4 so cxgbe(4) has been updated instead of writing a brand new driver. T5 cards will show up as cxl (short for cxlgb) ports attached to the t5nex bus driver. Sponsored by: Chelsio
2013-03-30 02:26:20 +00:00
uint32_t v;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
Ring the freelist doorbell from within refill_fl. While here, fix a bug that could have allowed the hardware pidx to reach the cidx even though the freelist isn't empty. (Haven't actually seen this but it was there waiting to happen..) MFC after: 1 week
2011-04-20 23:20:00 +00:00
if (FL_HW_IDX(fl->pidx) == FL_HW_IDX(fl->cidx))
ndesc--; /* hold back one credit */
if (ndesc <= 0)
return; /* nothing to do */
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
cxgbe(4): Add support for Chelsio's Terminator 5 (aka T5) ASIC. This includes support for the NIC and TOE features of the 40G, 10G, and 1G/100M cards based on the T5. The ASIC is mostly backward compatible with the Terminator 4 so cxgbe(4) has been updated instead of writing a brand new driver. T5 cards will show up as cxl (short for cxlgb) ports attached to the t5nex bus driver. Sponsored by: Chelsio
2013-03-30 02:26:20 +00:00
v = F_DBPRIO | V_QID(fl->cntxt_id) | V_PIDX(ndesc);
if (is_t5(sc))
v |= F_DBTYPE;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
wmb();
cxgbe(4): Add support for Chelsio's Terminator 5 (aka T5) ASIC. This includes support for the NIC and TOE features of the 40G, 10G, and 1G/100M cards based on the T5. The ASIC is mostly backward compatible with the Terminator 4 so cxgbe(4) has been updated instead of writing a brand new driver. T5 cards will show up as cxl (short for cxlgb) ports attached to the t5nex bus driver. Sponsored by: Chelsio
2013-03-30 02:26:20 +00:00
t4_write_reg(sc, MYPF_REG(A_SGE_PF_KDOORBELL), v);
Ring the freelist doorbell from within refill_fl. While here, fix a bug that could have allowed the hardware pidx to reach the cidx even though the freelist isn't empty. (Haven't actually seen this but it was there waiting to happen..) MFC after: 1 week
2011-04-20 23:20:00 +00:00
fl->pending -= ndesc * 8;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
}
Ring the freelist doorbell from within refill_fl. While here, fix a bug that could have allowed the hardware pidx to reach the cidx even though the freelist isn't empty. (Haven't actually seen this but it was there waiting to happen..) MFC after: 1 week
2011-04-20 23:20:00 +00:00
/*
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
* Fill up the freelist by upto nbufs and maybe ring its doorbell.
*
* Returns non-zero to indicate that it should be added to the list of starving
* freelists.
Ring the freelist doorbell from within refill_fl. While here, fix a bug that could have allowed the hardware pidx to reach the cidx even though the freelist isn't empty. (Haven't actually seen this but it was there waiting to happen..) MFC after: 1 week
2011-04-20 23:20:00 +00:00
*/
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
static int
refill_fl(struct adapter *sc, struct sge_fl *fl, int nbufs)
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
{
__be64 *d = &fl->desc[fl->pidx];
struct fl_sdesc *sd = &fl->sdesc[fl->pidx];
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
uintptr_t pa;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
caddr_t cl;
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
struct cluster_layout *cll = &fl->cll_def; /* default layout */
struct sw_zone_info *swz = &sc->sge.sw_zone_info[cll->zidx];
struct cluster_metadata *clm;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
FL_LOCK_ASSERT_OWNED(fl);
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
if (nbufs > fl->needed)
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
nbufs = fl->needed;
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
nbufs -= (fl->pidx + nbufs) % 8;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
while (nbufs--) {
if (sd->cl != NULL) {
cxgbe(4): Fix bug in the fast rx buffer recycle path. In some cases rx buffers were getting recycled when they should have been left alone. MFC after: 3 days
2014-06-18 00:16:35 +00:00
if (sd->nimbuf + sd->nembuf == 0) {
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
/*
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
* Fast recycle without involving any atomics on
* the cluster's metadata (if the cluster has
* metadata). This happens when all frames
* received in the cluster were small enough to
* fit within a single mbuf each.
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
*/
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
fl->cl_fast_recycled++;
cxgbe(4): Fix bug in the fast rx buffer recycle path. In some cases rx buffers were getting recycled when they should have been left alone. MFC after: 3 days
2014-06-18 00:16:35 +00:00
#ifdef INVARIANTS
clm = cl_metadata(sc, fl, &sd->cll, sd->cl);
if (clm != NULL)
MPASS(clm->refcount == 1);
#endif
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
goto recycled_fast;
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
}
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
/*
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
* Cluster is guaranteed to have metadata. Clusters
* without metadata always take the fast recycle path
* when they're recycled.
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
*/
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
clm = cl_metadata(sc, fl, &sd->cll, sd->cl);
MPASS(clm != NULL);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
if (atomic_fetchadd_int(&clm->refcount, -1) == 1) {
fl->cl_recycled++;
goto recycled;
}
sd->cl = NULL; /* gave up my reference */
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
}
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
MPASS(sd->cl == NULL);
alloc:
cl = uma_zalloc(swz->zone, M_NOWAIT);
if (__predict_false(cl == NULL)) {
if (cll == &fl->cll_alt || fl->cll_alt.zidx == -1 ||
fl->cll_def.zidx == fl->cll_alt.zidx)
break;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
/* fall back to the safe zone */
cll = &fl->cll_alt;
swz = &sc->sge.sw_zone_info[cll->zidx];
goto alloc;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
}
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
fl->cl_allocated++;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
pa = pmap_kextract((vm_offset_t)cl);
pa += cll->region1;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
sd->cl = cl;
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
sd->cll = *cll;
*d = htobe64(pa | cll->hwidx);
clm = cl_metadata(sc, fl, cll, cl);
if (clm != NULL) {
recycled:
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
#ifdef INVARIANTS
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
clm->sd = sd;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
#endif
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
clm->refcount = 1;
}
cxgbe(4): Fix bug in the fast rx buffer recycle path. In some cases rx buffers were getting recycled when they should have been left alone. MFC after: 3 days
2014-06-18 00:16:35 +00:00
sd->nimbuf = 0;
sd->nembuf = 0;
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
recycled_fast:
Tweaks for rx: - everything related to LRO should be in #ifdef INET blocks - reorder sge_iq's fields so that the most frequently used are all together - pull all rx code into t4_intr_data directly - let go of the ingress queue lock when passing up data - refill the freelist only if it is short of at least 32 buffers
2011-03-05 03:42:03 +00:00
fl->pending++;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
fl->needed--;
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
d++;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
sd++;
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
if (__predict_false(++fl->pidx == fl->cap)) {
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
fl->pidx = 0;
sd = fl->sdesc;
d = fl->desc;
}
}
Ring the freelist doorbell from within refill_fl. While here, fix a bug that could have allowed the hardware pidx to reach the cidx even though the freelist isn't empty. (Haven't actually seen this but it was there waiting to happen..) MFC after: 1 week
2011-04-20 23:20:00 +00:00
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
if (fl->pending >= 8)
Ring the freelist doorbell from within refill_fl. While here, fix a bug that could have allowed the hardware pidx to reach the cidx even though the freelist isn't empty. (Haven't actually seen this but it was there waiting to happen..) MFC after: 1 week
2011-04-20 23:20:00 +00:00
ring_fl_db(sc, fl);
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
return (FL_RUNNING_LOW(fl) && !(fl->flags & FL_STARVING));
}
/*
* Attempt to refill all starving freelists.
*/
static void
refill_sfl(void *arg)
{
struct adapter *sc = arg;
struct sge_fl *fl, *fl_temp;
mtx_lock(&sc->sfl_lock);
TAILQ_FOREACH_SAFE(fl, &sc->sfl, link, fl_temp) {
FL_LOCK(fl);
refill_fl(sc, fl, 64);
if (FL_NOT_RUNNING_LOW(fl) || fl->flags & FL_DOOMED) {
TAILQ_REMOVE(&sc->sfl, fl, link);
fl->flags &= ~FL_STARVING;
}
FL_UNLOCK(fl);
}
if (!TAILQ_EMPTY(&sc->sfl))
callout_schedule(&sc->sfl_callout, hz / 5);
mtx_unlock(&sc->sfl_lock);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
}
static int
alloc_fl_sdesc(struct sge_fl *fl)
{
fl->sdesc = malloc(fl->cap * sizeof(struct fl_sdesc), M_CXGBE,
M_ZERO | M_WAITOK);
return (0);
}
static void
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
free_fl_sdesc(struct adapter *sc, struct sge_fl *fl)
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
{
struct fl_sdesc *sd;
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
struct cluster_metadata *clm;
struct cluster_layout *cll;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
int i;
sd = fl->sdesc;
for (i = 0; i < fl->cap; i++, sd++) {
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
if (sd->cl == NULL)
continue;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
cll = &sd->cll;
clm = cl_metadata(sc, fl, cll, sd->cl);
cxgbe(4): Fix bug in the fast rx buffer recycle path. In some cases rx buffers were getting recycled when they should have been left alone. MFC after: 3 days
2014-06-18 00:16:35 +00:00
if (sd->nimbuf + sd->nembuf == 0 ||
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
(clm && atomic_fetchadd_int(&clm->refcount, -1) == 1)) {
uma_zfree(sc->sge.sw_zone_info[cll->zidx].zone, sd->cl);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
}
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
sd->cl = NULL;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
}
free(fl->sdesc, M_CXGBE);
fl->sdesc = NULL;
}
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
int
t4_alloc_tx_maps(struct tx_maps *txmaps, bus_dma_tag_t tx_tag, int count,
int flags)
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
{
struct tx_map *txm;
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
int i, rc;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
txmaps->map_total = txmaps->map_avail = count;
txmaps->map_cidx = txmaps->map_pidx = 0;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
txmaps->maps = malloc(count * sizeof(struct tx_map), M_CXGBE,
M_ZERO | flags);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
txm = txmaps->maps;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
for (i = 0; i < count; i++, txm++) {
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
rc = bus_dmamap_create(tx_tag, 0, &txm->map);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
if (rc != 0)
goto failed;
}
return (0);
failed:
while (--i >= 0) {
txm--;
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
bus_dmamap_destroy(tx_tag, txm->map);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
}
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
KASSERT(txm == txmaps->maps, ("%s: EDOOFUS", __func__));
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
free(txmaps->maps, M_CXGBE);
txmaps->maps = NULL;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
return (rc);
}
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
void
t4_free_tx_maps(struct tx_maps *txmaps, bus_dma_tag_t tx_tag)
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
{
struct tx_map *txm;
int i;
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
txm = txmaps->maps;
for (i = 0; i < txmaps->map_total; i++, txm++) {
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
if (txm->m) {
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
bus_dmamap_unload(tx_tag, txm->map);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
m_freem(txm->m);
txm->m = NULL;
}
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
bus_dmamap_destroy(tx_tag, txm->map);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
}
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
free(txmaps->maps, M_CXGBE);
txmaps->maps = NULL;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
}
/*
* We'll do immediate data tx for non-TSO, but only when not coalescing. We're
* willing to use upto 2 hardware descriptors which means a maximum of 96 bytes
* of immediate data.
*/
#define IMM_LEN ( \
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
2 * EQ_ESIZE \
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
- sizeof(struct fw_eth_tx_pkt_wr) \
- sizeof(struct cpl_tx_pkt_core))
/*
* Returns non-zero on failure, no need to cleanup anything in that case.
*
* Note 1: We always try to defrag the mbuf if required and return EFBIG only
* if the resulting chain still won't fit in a tx descriptor.
*
* Note 2: We'll pullup the mbuf chain if TSO is requested and the first mbuf
* does not have the TCP header in it.
*/
static int
get_pkt_sgl(struct sge_txq *txq, struct mbuf **fp, struct sgl *sgl,
int sgl_only)
{
struct mbuf *m = *fp;
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
struct tx_maps *txmaps;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
struct tx_map *txm;
int rc, defragged = 0, n;
TXQ_LOCK_ASSERT_OWNED(txq);
if (m->m_pkthdr.tso_segsz)
sgl_only = 1; /* Do not allow immediate data with LSO */
start: sgl->nsegs = 0;
if (m->m_pkthdr.len <= IMM_LEN && !sgl_only)
return (0); /* nsegs = 0 tells caller to use imm. tx */
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
txmaps = &txq->txmaps;
if (txmaps->map_avail == 0) {
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
txq->no_dmamap++;
return (ENOMEM);
}
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
txm = &txmaps->maps[txmaps->map_pidx];
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
if (m->m_pkthdr.tso_segsz && m->m_len < 50) {
*fp = m_pullup(m, 50);
m = *fp;
if (m == NULL)
return (ENOBUFS);
}
- Move all Ethernet specific items from sge_eq to sge_txq. sge_eq is now a suitable base for all kinds of egress queues. - Add control queues (sge_ctrlq) and allocate one of these per hardware channel. They can be used to program filters and steer traffic (and more). MFC after: 1 week
2011-04-19 22:08:28 +00:00
rc = bus_dmamap_load_mbuf_sg(txq->tx_tag, txm->map, m, sgl->seg,
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
&sgl->nsegs, BUS_DMA_NOWAIT);
if (rc == EFBIG && defragged == 0) {
Mechanically substitute flags from historic mbuf allocator with malloc(9) flags in sys/dev.
2012-12-04 09:32:43 +00:00
m = m_defrag(m, M_NOWAIT);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
if (m == NULL)
return (EFBIG);
defragged = 1;
*fp = m;
goto start;
}
if (rc != 0)
return (rc);
txm->m = m;
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
txmaps->map_avail--;
if (++txmaps->map_pidx == txmaps->map_total)
txmaps->map_pidx = 0;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
KASSERT(sgl->nsegs > 0 && sgl->nsegs <= TX_SGL_SEGS,
("%s: bad DMA mapping (%d segments)", __func__, sgl->nsegs));
/*
* Store the # of flits required to hold this frame's SGL in nflits. An
* SGL has a (ULPTX header + len0, addr0) tuple optionally followed by
* multiple (len0 + len1, addr0, addr1) tuples. If addr1 is not used
* then len1 must be set to 0.
*/
n = sgl->nsegs - 1;
sgl->nflits = (3 * n) / 2 + (n & 1) + 2;
return (0);
}
/*
* Releases all the txq resources used up in the specified sgl.
*/
static int
free_pkt_sgl(struct sge_txq *txq, struct sgl *sgl)
{
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
struct tx_maps *txmaps;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
struct tx_map *txm;
TXQ_LOCK_ASSERT_OWNED(txq);
if (sgl->nsegs == 0)
return (0); /* didn't use any map */
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
txmaps = &txq->txmaps;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
/* 1 pkt uses exactly 1 map, back it out */
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
txmaps->map_avail++;
if (txmaps->map_pidx > 0)
txmaps->map_pidx--;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
else
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
txmaps->map_pidx = txmaps->map_total - 1;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
txm = &txmaps->maps[txmaps->map_pidx];
- Move all Ethernet specific items from sge_eq to sge_txq. sge_eq is now a suitable base for all kinds of egress queues. - Add control queues (sge_ctrlq) and allocate one of these per hardware channel. They can be used to program filters and steer traffic (and more). MFC after: 1 week
2011-04-19 22:08:28 +00:00
bus_dmamap_unload(txq->tx_tag, txm->map);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
txm->m = NULL;
return (0);
}
static int
write_txpkt_wr(struct port_info *pi, struct sge_txq *txq, struct mbuf *m,
struct sgl *sgl)
{
struct sge_eq *eq = &txq->eq;
struct fw_eth_tx_pkt_wr *wr;
struct cpl_tx_pkt_core *cpl;
uint32_t ctrl; /* used in many unrelated places */
uint64_t ctrl1;
Resume tx immediately in response to an SGE egress update from the hardware. MFC after: 1 week
2011-03-05 03:06:38 +00:00
int nflits, ndesc, pktlen;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
struct tx_sdesc *txsd;
caddr_t dst;
TXQ_LOCK_ASSERT_OWNED(txq);
Resume tx immediately in response to an SGE egress update from the hardware. MFC after: 1 week
2011-03-05 03:06:38 +00:00
pktlen = m->m_pkthdr.len;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
/*
* Do we have enough flits to send this frame out?
*/
ctrl = sizeof(struct cpl_tx_pkt_core);
if (m->m_pkthdr.tso_segsz) {
nflits = TXPKT_LSO_WR_HDR;
cxgbe(4): update to firmware interface 1.5.2.0; updates to shared code.
2012-06-22 07:51:15 +00:00
ctrl += sizeof(struct cpl_tx_pkt_lso_core);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
} else
nflits = TXPKT_WR_HDR;
if (sgl->nsegs > 0)
nflits += sgl->nflits;
else {
Resume tx immediately in response to an SGE egress update from the hardware. MFC after: 1 week
2011-03-05 03:06:38 +00:00
nflits += howmany(pktlen, 8);
ctrl += pktlen;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
}
ndesc = howmany(nflits, 8);
if (ndesc > eq->avail)
return (ENOMEM);
/* Firmware work request header */
wr = (void *)&eq->desc[eq->pidx];
wr->op_immdlen = htobe32(V_FW_WR_OP(FW_ETH_TX_PKT_WR) |
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
V_FW_ETH_TX_PKT_WR_IMMDLEN(ctrl));
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
ctrl = V_FW_WR_LEN16(howmany(nflits, 2));
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
if (eq->avail == ndesc) {
if (!(eq->flags & EQ_CRFLUSHED)) {
ctrl |= F_FW_WR_EQUEQ | F_FW_WR_EQUIQ;
eq->flags |= EQ_CRFLUSHED;
}
eq->flags |= EQ_STALLED;
There is no need to request a tx credit flush if such a request is already pending. MFC after: 3 days
2011-04-14 20:06:23 +00:00
}
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
wr->equiq_to_len16 = htobe32(ctrl);
wr->r3 = 0;
if (m->m_pkthdr.tso_segsz) {
cxgbe(4): update to firmware interface 1.5.2.0; updates to shared code.
2012-06-22 07:51:15 +00:00
struct cpl_tx_pkt_lso_core *lso = (void *)(wr + 1);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
struct ether_header *eh;
cxgbe(4): support for IPv6 TSO and LRO. Submitted by: bz (this is a modified version of that patch)
2012-06-29 19:51:06 +00:00
void *l3hdr;
#if defined(INET) || defined(INET6)
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
struct tcphdr *tcp;
cxgbe(4): support for IPv6 TSO and LRO. Submitted by: bz (this is a modified version of that patch)
2012-06-29 19:51:06 +00:00
#endif
uint16_t eh_type;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
ctrl = V_LSO_OPCODE(CPL_TX_PKT_LSO) | F_LSO_FIRST_SLICE |
F_LSO_LAST_SLICE;
eh = mtod(m, struct ether_header *);
cxgbe(4): support for IPv6 TSO and LRO. Submitted by: bz (this is a modified version of that patch)
2012-06-29 19:51:06 +00:00
eh_type = ntohs(eh->ether_type);
if (eh_type == ETHERTYPE_VLAN) {
struct ether_vlan_header *evh = (void *)eh;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
ctrl |= V_LSO_ETHHDR_LEN(1);
cxgbe(4): support for IPv6 TSO and LRO. Submitted by: bz (this is a modified version of that patch)
2012-06-29 19:51:06 +00:00
l3hdr = evh + 1;
eh_type = ntohs(evh->evl_proto);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
} else
cxgbe(4): support for IPv6 TSO and LRO. Submitted by: bz (this is a modified version of that patch)
2012-06-29 19:51:06 +00:00
l3hdr = eh + 1;
switch (eh_type) {
#ifdef INET6
case ETHERTYPE_IPV6:
{
struct ip6_hdr *ip6 = l3hdr;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
cxgbe(4): support for IPv6 TSO and LRO. Submitted by: bz (this is a modified version of that patch)
2012-06-29 19:51:06 +00:00
/*
* XXX-BZ For now we do not pretend to support
* IPv6 extension headers.
*/
KASSERT(ip6->ip6_nxt == IPPROTO_TCP, ("%s: CSUM_TSO "
"with ip6_nxt != TCP: %u", __func__, ip6->ip6_nxt));
tcp = (struct tcphdr *)(ip6 + 1);
ctrl |= F_LSO_IPV6;
ctrl |= V_LSO_IPHDR_LEN(sizeof(*ip6) >> 2) |
V_LSO_TCPHDR_LEN(tcp->th_off);
break;
}
#endif
#ifdef INET
case ETHERTYPE_IP:
{
struct ip *ip = l3hdr;
tcp = (void *)((uintptr_t)ip + ip->ip_hl * 4);
ctrl |= V_LSO_IPHDR_LEN(ip->ip_hl) |
V_LSO_TCPHDR_LEN(tcp->th_off);
break;
}
#endif
default:
panic("%s: CSUM_TSO but no supported IP version "
"(0x%04x)", __func__, eh_type);
}
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
lso->lso_ctrl = htobe32(ctrl);
lso->ipid_ofst = htobe16(0);
lso->mss = htobe16(m->m_pkthdr.tso_segsz);
lso->seqno_offset = htobe32(0);
Resume tx immediately in response to an SGE egress update from the hardware. MFC after: 1 week
2011-03-05 03:06:38 +00:00
lso->len = htobe32(pktlen);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
cpl = (void *)(lso + 1);
txq->tso_wrs++;
} else
cpl = (void *)(wr + 1);
/* Checksum offload */
ctrl1 = 0;
cxgbe(4): Assume that CSUM_TSO in the transmit path implies CSUM_IP and CSUM_TCP too. They are all set explicitly by the kernel usually. While here, fix an unrelated bug where hardware L4 checksum calculation was accidentally disabled for some IPv6 packets. Reported by: alfred@ MFC after: 3 days
2013-02-20 23:15:40 +00:00
if (!(m->m_pkthdr.csum_flags & (CSUM_IP | CSUM_TSO)))
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
ctrl1 |= F_TXPKT_IPCSUM_DIS;
cxgbe(4): support for IPv6 hardware checksumming (rx and tx).
2012-06-29 16:50:52 +00:00
if (!(m->m_pkthdr.csum_flags & (CSUM_TCP | CSUM_UDP | CSUM_UDP_IPV6 |
cxgbe(4): Assume that CSUM_TSO in the transmit path implies CSUM_IP and CSUM_TCP too. They are all set explicitly by the kernel usually. While here, fix an unrelated bug where hardware L4 checksum calculation was accidentally disabled for some IPv6 packets. Reported by: alfred@ MFC after: 3 days
2013-02-20 23:15:40 +00:00
CSUM_TCP_IPV6 | CSUM_TSO)))
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
ctrl1 |= F_TXPKT_L4CSUM_DIS;
cxgbe(4): support for IPv6 hardware checksumming (rx and tx).
2012-06-29 16:50:52 +00:00
if (m->m_pkthdr.csum_flags & (CSUM_IP | CSUM_TCP | CSUM_UDP |
cxgbe(4): Assume that CSUM_TSO in the transmit path implies CSUM_IP and CSUM_TCP too. They are all set explicitly by the kernel usually. While here, fix an unrelated bug where hardware L4 checksum calculation was accidentally disabled for some IPv6 packets. Reported by: alfred@ MFC after: 3 days
2013-02-20 23:15:40 +00:00
CSUM_UDP_IPV6 | CSUM_TCP_IPV6 | CSUM_TSO))
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
txq->txcsum++; /* some hardware assistance provided */
/* VLAN tag insertion */
if (m->m_flags & M_VLANTAG) {
ctrl1 |= F_TXPKT_VLAN_VLD | V_TXPKT_VLAN(m->m_pkthdr.ether_vtag);
txq->vlan_insertion++;
}
/* CPL header */
cpl->ctrl0 = htobe32(V_TXPKT_OPCODE(CPL_TX_PKT) |
V_TXPKT_INTF(pi->tx_chan) | V_TXPKT_PF(pi->adapter->pf));
cpl->pack = 0;
Resume tx immediately in response to an SGE egress update from the hardware. MFC after: 1 week
2011-03-05 03:06:38 +00:00
cpl->len = htobe16(pktlen);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
cpl->ctrl1 = htobe64(ctrl1);
/* Software descriptor */
- Move all Ethernet specific items from sge_eq to sge_txq. sge_eq is now a suitable base for all kinds of egress queues. - Add control queues (sge_ctrlq) and allocate one of these per hardware channel. They can be used to program filters and steer traffic (and more). MFC after: 1 week
2011-04-19 22:08:28 +00:00
txsd = &txq->sdesc[eq->pidx];
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
txsd->desc_used = ndesc;
eq->pending += ndesc;
eq->avail -= ndesc;
eq->pidx += ndesc;
if (eq->pidx >= eq->cap)
eq->pidx -= eq->cap;
/* SGL */
dst = (void *)(cpl + 1);
if (sgl->nsegs > 0) {
- Move all Ethernet specific items from sge_eq to sge_txq. sge_eq is now a suitable base for all kinds of egress queues. - Add control queues (sge_ctrlq) and allocate one of these per hardware channel. They can be used to program filters and steer traffic (and more). MFC after: 1 week
2011-04-19 22:08:28 +00:00
txsd->credits = 1;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
txq->sgl_wrs++;
write_sgl_to_txd(eq, sgl, &dst);
} else {
- Move all Ethernet specific items from sge_eq to sge_txq. sge_eq is now a suitable base for all kinds of egress queues. - Add control queues (sge_ctrlq) and allocate one of these per hardware channel. They can be used to program filters and steer traffic (and more). MFC after: 1 week
2011-04-19 22:08:28 +00:00
txsd->credits = 0;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
txq->imm_wrs++;
for (; m; m = m->m_next) {
copy_to_txd(eq, mtod(m, caddr_t), &dst, m->m_len);
Resume tx immediately in response to an SGE egress update from the hardware. MFC after: 1 week
2011-03-05 03:06:38 +00:00
#ifdef INVARIANTS
pktlen -= m->m_len;
#endif
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
}
Resume tx immediately in response to an SGE egress update from the hardware. MFC after: 1 week
2011-03-05 03:06:38 +00:00
#ifdef INVARIANTS
KASSERT(pktlen == 0, ("%s: %d bytes left.", __func__, pktlen));
#endif
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
}
txq->txpkt_wrs++;
return (0);
}
/*
* Returns 0 to indicate that m has been accepted into a coalesced tx work
* request. It has either been folded into txpkts or txpkts was flushed and m
* has started a new coalesced work request (as the first frame in a fresh
* txpkts).
*
* Returns non-zero to indicate a failure - caller is responsible for
* transmitting m, if there was anything in txpkts it has been flushed.
*/
static int
add_to_txpkts(struct port_info *pi, struct sge_txq *txq, struct txpkts *txpkts,
struct mbuf *m, struct sgl *sgl)
{
struct sge_eq *eq = &txq->eq;
int can_coalesce;
struct tx_sdesc *txsd;
int flits;
TXQ_LOCK_ASSERT_OWNED(txq);
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
KASSERT(sgl->nsegs, ("%s: can't coalesce imm data", __func__));
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
if (txpkts->npkt > 0) {
flits = TXPKTS_PKT_HDR + sgl->nflits;
can_coalesce = m->m_pkthdr.tso_segsz == 0 &&
txpkts->nflits + flits <= TX_WR_FLITS &&
txpkts->nflits + flits <= eq->avail * 8 &&
txpkts->plen + m->m_pkthdr.len < 65536;
if (can_coalesce) {
txpkts->npkt++;
txpkts->nflits += flits;
txpkts->plen += m->m_pkthdr.len;
- Move all Ethernet specific items from sge_eq to sge_txq. sge_eq is now a suitable base for all kinds of egress queues. - Add control queues (sge_ctrlq) and allocate one of these per hardware channel. They can be used to program filters and steer traffic (and more). MFC after: 1 week
2011-04-19 22:08:28 +00:00
txsd = &txq->sdesc[eq->pidx];
txsd->credits++;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
return (0);
}
/*
* Couldn't coalesce m into txpkts. The first order of business
* is to send txpkts on its way. Then we'll revisit m.
*/
write_txpkts_wr(txq, txpkts);
}
/*
* Check if we can start a new coalesced tx work request with m as
* the first packet in it.
*/
KASSERT(txpkts->npkt == 0, ("%s: txpkts not empty", __func__));
flits = TXPKTS_WR_HDR + sgl->nflits;
can_coalesce = m->m_pkthdr.tso_segsz == 0 &&
flits <= eq->avail * 8 && flits <= TX_WR_FLITS;
if (can_coalesce == 0)
return (EINVAL);
/*
* Start a fresh coalesced tx WR with m as the first frame in it.
*/
txpkts->npkt = 1;
txpkts->nflits = flits;
txpkts->flitp = &eq->desc[eq->pidx].flit[2];
txpkts->plen = m->m_pkthdr.len;
- Move all Ethernet specific items from sge_eq to sge_txq. sge_eq is now a suitable base for all kinds of egress queues. - Add control queues (sge_ctrlq) and allocate one of these per hardware channel. They can be used to program filters and steer traffic (and more). MFC after: 1 week
2011-04-19 22:08:28 +00:00
txsd = &txq->sdesc[eq->pidx];
txsd->credits = 1;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
return (0);
}
/*
* Note that write_txpkts_wr can never run out of hardware descriptors (but
* write_txpkt_wr can). add_to_txpkts ensures that a frame is accepted for
* coalescing only if sufficient hardware descriptors are available.
*/
static void
write_txpkts_wr(struct sge_txq *txq, struct txpkts *txpkts)
{
struct sge_eq *eq = &txq->eq;
struct fw_eth_tx_pkts_wr *wr;
struct tx_sdesc *txsd;
uint32_t ctrl;
int ndesc;
TXQ_LOCK_ASSERT_OWNED(txq);
ndesc = howmany(txpkts->nflits, 8);
wr = (void *)&eq->desc[eq->pidx];
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
wr->op_pkd = htobe32(V_FW_WR_OP(FW_ETH_TX_PKTS_WR));
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
ctrl = V_FW_WR_LEN16(howmany(txpkts->nflits, 2));
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
if (eq->avail == ndesc) {
if (!(eq->flags & EQ_CRFLUSHED)) {
ctrl |= F_FW_WR_EQUEQ | F_FW_WR_EQUIQ;
eq->flags |= EQ_CRFLUSHED;
}
eq->flags |= EQ_STALLED;
There is no need to request a tx credit flush if such a request is already pending. MFC after: 3 days
2011-04-14 20:06:23 +00:00
}
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
wr->equiq_to_len16 = htobe32(ctrl);
wr->plen = htobe16(txpkts->plen);
wr->npkt = txpkts->npkt;
Update to firmware interface 1.3.10 MFC after: 1 week
2011-05-30 21:56:37 +00:00
wr->r3 = wr->type = 0;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
/* Everything else already written */
- Move all Ethernet specific items from sge_eq to sge_txq. sge_eq is now a suitable base for all kinds of egress queues. - Add control queues (sge_ctrlq) and allocate one of these per hardware channel. They can be used to program filters and steer traffic (and more). MFC after: 1 week
2011-04-19 22:08:28 +00:00
txsd = &txq->sdesc[eq->pidx];
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
txsd->desc_used = ndesc;
There is no need to request a tx credit flush if such a request is already pending. MFC after: 3 days
2011-04-14 20:06:23 +00:00
KASSERT(eq->avail >= ndesc, ("%s: out of descriptors", __func__));
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
eq->pending += ndesc;
eq->avail -= ndesc;
eq->pidx += ndesc;
if (eq->pidx >= eq->cap)
eq->pidx -= eq->cap;
txq->txpkts_pkts += txpkts->npkt;
txq->txpkts_wrs++;
txpkts->npkt = 0; /* emptied */
}
static inline void
write_ulp_cpl_sgl(struct port_info *pi, struct sge_txq *txq,
struct txpkts *txpkts, struct mbuf *m, struct sgl *sgl)
{
struct ulp_txpkt *ulpmc;
struct ulptx_idata *ulpsc;
struct cpl_tx_pkt_core *cpl;
struct sge_eq *eq = &txq->eq;
uintptr_t flitp, start, end;
uint64_t ctrl;
caddr_t dst;
KASSERT(txpkts->npkt > 0, ("%s: txpkts is empty", __func__));
start = (uintptr_t)eq->desc;
end = (uintptr_t)eq->spg;
/* Checksum offload */
ctrl = 0;
cxgbe(4): Assume that CSUM_TSO in the transmit path implies CSUM_IP and CSUM_TCP too. They are all set explicitly by the kernel usually. While here, fix an unrelated bug where hardware L4 checksum calculation was accidentally disabled for some IPv6 packets. Reported by: alfred@ MFC after: 3 days
2013-02-20 23:15:40 +00:00
if (!(m->m_pkthdr.csum_flags & (CSUM_IP | CSUM_TSO)))
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
ctrl |= F_TXPKT_IPCSUM_DIS;
cxgbe(4): Assume that CSUM_TSO in the transmit path implies CSUM_IP and CSUM_TCP too. They are all set explicitly by the kernel usually. While here, fix an unrelated bug where hardware L4 checksum calculation was accidentally disabled for some IPv6 packets. Reported by: alfred@ MFC after: 3 days
2013-02-20 23:15:40 +00:00
if (!(m->m_pkthdr.csum_flags & (CSUM_TCP | CSUM_UDP | CSUM_UDP_IPV6 |
CSUM_TCP_IPV6 | CSUM_TSO)))
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
ctrl |= F_TXPKT_L4CSUM_DIS;
cxgbe(4): Assume that CSUM_TSO in the transmit path implies CSUM_IP and CSUM_TCP too. They are all set explicitly by the kernel usually. While here, fix an unrelated bug where hardware L4 checksum calculation was accidentally disabled for some IPv6 packets. Reported by: alfred@ MFC after: 3 days
2013-02-20 23:15:40 +00:00
if (m->m_pkthdr.csum_flags & (CSUM_IP | CSUM_TCP | CSUM_UDP |
CSUM_UDP_IPV6 | CSUM_TCP_IPV6 | CSUM_TSO))
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
txq->txcsum++; /* some hardware assistance provided */
/* VLAN tag insertion */
if (m->m_flags & M_VLANTAG) {
ctrl |= F_TXPKT_VLAN_VLD | V_TXPKT_VLAN(m->m_pkthdr.ether_vtag);
txq->vlan_insertion++;
}
/*
* The previous packet's SGL must have ended at a 16 byte boundary (this
* is required by the firmware/hardware). It follows that flitp cannot
* wrap around between the ULPTX master command and ULPTX subcommand (8
* bytes each), and that it can not wrap around in the middle of the
* cpl_tx_pkt_core either.
*/
flitp = (uintptr_t)txpkts->flitp;
KASSERT((flitp & 0xf) == 0,
("%s: last SGL did not end at 16 byte boundary: %p",
__func__, txpkts->flitp));
/* ULP master command */
ulpmc = (void *)flitp;
A txpkts work request should have a valid FID. MFC after: 1 week
2011-03-05 03:18:56 +00:00
ulpmc->cmd_dest = htonl(V_ULPTX_CMD(ULP_TX_PKT) | V_ULP_TXPKT_DEST(0) |
V_ULP_TXPKT_FID(eq->iqid));
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
ulpmc->len = htonl(howmany(sizeof(*ulpmc) + sizeof(*ulpsc) +
sizeof(*cpl) + 8 * sgl->nflits, 16));
/* ULP subcommand */
ulpsc = (void *)(ulpmc + 1);
ulpsc->cmd_more = htobe32(V_ULPTX_CMD((u32)ULP_TX_SC_IMM) |
F_ULP_TX_SC_MORE);
ulpsc->len = htobe32(sizeof(struct cpl_tx_pkt_core));
flitp += sizeof(*ulpmc) + sizeof(*ulpsc);
if (flitp == end)
flitp = start;
/* CPL_TX_PKT */
cpl = (void *)flitp;
cpl->ctrl0 = htobe32(V_TXPKT_OPCODE(CPL_TX_PKT) |
V_TXPKT_INTF(pi->tx_chan) | V_TXPKT_PF(pi->adapter->pf));
cpl->pack = 0;
cpl->len = htobe16(m->m_pkthdr.len);
cpl->ctrl1 = htobe64(ctrl);
flitp += sizeof(*cpl);
if (flitp == end)
flitp = start;
/* SGL for this frame */
dst = (caddr_t)flitp;
txpkts->nflits += write_sgl_to_txd(eq, sgl, &dst);
txpkts->flitp = (void *)dst;
KASSERT(((uintptr_t)dst & 0xf) == 0,
("%s: SGL ends at %p (not a 16 byte boundary)", __func__, dst));
}
/*
* If the SGL ends on an address that is not 16 byte aligned, this function will
* add a 0 filled flit at the end. It returns 1 in that case.
*/
static int
write_sgl_to_txd(struct sge_eq *eq, struct sgl *sgl, caddr_t *to)
{
__be64 *flitp, *end;
struct ulptx_sgl *usgl;
bus_dma_segment_t *seg;
int i, padded;
KASSERT(sgl->nsegs > 0 && sgl->nflits > 0,
("%s: bad SGL - nsegs=%d, nflits=%d",
__func__, sgl->nsegs, sgl->nflits));
KASSERT(((uintptr_t)(*to) & 0xf) == 0,
("%s: SGL must start at a 16 byte boundary: %p", __func__, *to));
flitp = (__be64 *)(*to);
end = flitp + sgl->nflits;
seg = &sgl->seg[0];
usgl = (void *)flitp;
/*
* We start at a 16 byte boundary somewhere inside the tx descriptor
* ring, so we're at least 16 bytes away from the status page. There is
* no chance of a wrap around in the middle of usgl (which is 16 bytes).
*/
usgl->cmd_nsge = htobe32(V_ULPTX_CMD(ULP_TX_SC_DSGL) |
V_ULPTX_NSGE(sgl->nsegs));
usgl->len0 = htobe32(seg->ds_len);
usgl->addr0 = htobe64(seg->ds_addr);
seg++;
if ((uintptr_t)end <= (uintptr_t)eq->spg) {
/* Won't wrap around at all */
for (i = 0; i < sgl->nsegs - 1; i++, seg++) {
usgl->sge[i / 2].len[i & 1] = htobe32(seg->ds_len);
usgl->sge[i / 2].addr[i & 1] = htobe64(seg->ds_addr);
}
if (i & 1)
usgl->sge[i / 2].len[1] = htobe32(0);
} else {
/* Will wrap somewhere in the rest of the SGL */
/* 2 flits already written, write the rest flit by flit */
flitp = (void *)(usgl + 1);
for (i = 0; i < sgl->nflits - 2; i++) {
if ((uintptr_t)flitp == (uintptr_t)eq->spg)
flitp = (void *)eq->desc;
*flitp++ = get_flit(seg, sgl->nsegs - 1, i);
}
end = flitp;
}
if ((uintptr_t)end & 0xf) {
*(uint64_t *)end = 0;
end++;
padded = 1;
} else
padded = 0;
if ((uintptr_t)end == (uintptr_t)eq->spg)
*to = (void *)eq->desc;
else
*to = (void *)end;
return (padded);
}
static inline void
copy_to_txd(struct sge_eq *eq, caddr_t from, caddr_t *to, int len)
{
- Updated TOE support in the kernel. - Stateful TCP offload drivers for Terminator 3 and 4 (T3 and T4) ASICs. These are available as t3_tom and t4_tom modules that augment cxgb(4) and cxgbe(4) respectively. The cxgb/cxgbe drivers continue to work as usual with or without these extra features. - iWARP driver for Terminator 3 ASIC (kernel verbs). T4 iWARP in the works and will follow soon. Build-tested with make universe. 30s overview ============ What interfaces support TCP offload? Look for TOE4 and/or TOE6 in the capabilities of an interface: # ifconfig -m | grep TOE Enable/disable TCP offload on an interface (just like any other ifnet capability): # ifconfig cxgbe0 toe # ifconfig cxgbe0 -toe Which connections are offloaded? Look for toe4 and/or toe6 in the output of netstat and sockstat: # netstat -np tcp | grep toe # sockstat -46c | grep toe Reviewed by: bz, gnn Sponsored by: Chelsio communications. MFC after: ~3 months (after 9.1, and after ensuring MFC is feasible)
2012-06-19 07:34:13 +00:00
if (__predict_true((uintptr_t)(*to) + len <= (uintptr_t)eq->spg)) {
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
bcopy(from, *to, len);
(*to) += len;
} else {
int portion = (uintptr_t)eq->spg - (uintptr_t)(*to);
bcopy(from, *to, portion);
from += portion;
portion = len - portion; /* remaining */
bcopy(from, (void *)eq->desc, portion);
(*to) = (caddr_t)eq->desc + portion;
}
}
static inline void
- Move all Ethernet specific items from sge_eq to sge_txq. sge_eq is now a suitable base for all kinds of egress queues. - Add control queues (sge_ctrlq) and allocate one of these per hardware channel. They can be used to program filters and steer traffic (and more). MFC after: 1 week
2011-04-19 22:08:28 +00:00
ring_eq_db(struct adapter *sc, struct sge_eq *eq)
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
{
cxgbe(4): Add support for Chelsio's Terminator 5 (aka T5) ASIC. This includes support for the NIC and TOE features of the 40G, 10G, and 1G/100M cards based on the T5. The ASIC is mostly backward compatible with the Terminator 4 so cxgbe(4) has been updated instead of writing a brand new driver. T5 cards will show up as cxl (short for cxlgb) ports attached to the t5nex bus driver. Sponsored by: Chelsio
2013-03-30 02:26:20 +00:00
u_int db, pending;
db = eq->doorbells;
pending = eq->pending;
if (pending > 1)
Cosmetic change (s/wrwc/wcwr/;s/WRWC/WCWR/). MFC after: 3 days.
2013-04-11 22:49:29 +00:00
clrbit(&db, DOORBELL_WCWR);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
eq->pending = 0;
cxgbe(4): Add support for Chelsio's Terminator 5 (aka T5) ASIC. This includes support for the NIC and TOE features of the 40G, 10G, and 1G/100M cards based on the T5. The ASIC is mostly backward compatible with the Terminator 4 so cxgbe(4) has been updated instead of writing a brand new driver. T5 cards will show up as cxl (short for cxlgb) ports attached to the t5nex bus driver. Sponsored by: Chelsio
2013-03-30 02:26:20 +00:00
wmb();
switch (ffs(db) - 1) {
case DOORBELL_UDB:
*eq->udb = htole32(V_QID(eq->udb_qid) | V_PIDX(pending));
return;
Cosmetic change (s/wrwc/wcwr/;s/WRWC/WCWR/). MFC after: 3 days.
2013-04-11 22:49:29 +00:00
case DOORBELL_WCWR: {
cxgbe(4): Add support for Chelsio's Terminator 5 (aka T5) ASIC. This includes support for the NIC and TOE features of the 40G, 10G, and 1G/100M cards based on the T5. The ASIC is mostly backward compatible with the Terminator 4 so cxgbe(4) has been updated instead of writing a brand new driver. T5 cards will show up as cxl (short for cxlgb) ports attached to the t5nex bus driver. Sponsored by: Chelsio
2013-03-30 02:26:20 +00:00
volatile uint64_t *dst, *src;
int i;
/*
* Queues whose 128B doorbell segment fits in the page do not
* use relative qid (udb_qid is always 0). Only queues with
Cosmetic change (s/wrwc/wcwr/;s/WRWC/WCWR/). MFC after: 3 days.
2013-04-11 22:49:29 +00:00
* doorbell segments can do WCWR.
cxgbe(4): Add support for Chelsio's Terminator 5 (aka T5) ASIC. This includes support for the NIC and TOE features of the 40G, 10G, and 1G/100M cards based on the T5. The ASIC is mostly backward compatible with the Terminator 4 so cxgbe(4) has been updated instead of writing a brand new driver. T5 cards will show up as cxl (short for cxlgb) ports attached to the t5nex bus driver. Sponsored by: Chelsio
2013-03-30 02:26:20 +00:00
*/
KASSERT(eq->udb_qid == 0 && pending == 1,
("%s: inappropriate doorbell (0x%x, %d, %d) for eq %p",
__func__, eq->doorbells, pending, eq->pidx, eq));
dst = (volatile void *)((uintptr_t)eq->udb + UDBS_WR_OFFSET -
UDBS_DB_OFFSET);
i = eq->pidx ? eq->pidx - 1 : eq->cap - 1;
src = (void *)&eq->desc[i];
while (src != (void *)&eq->desc[i + 1])
*dst++ = *src++;
wmb();
return;
}
case DOORBELL_UDBWC:
*eq->udb = htole32(V_QID(eq->udb_qid) | V_PIDX(pending));
wmb();
return;
case DOORBELL_KDB:
t4_write_reg(sc, MYPF_REG(A_SGE_PF_KDOORBELL),
V_QID(eq->cntxt_id) | V_PIDX(pending));
return;
}
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
}
Calculate how many descriptors can be reclaimed before calling reclaim_tx_descs
2011-03-05 03:54:37 +00:00
static inline int
reclaimable(struct sge_eq *eq)
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
{
Calculate how many descriptors can be reclaimed before calling reclaim_tx_descs
2011-03-05 03:54:37 +00:00
unsigned int cidx;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
cidx = eq->spg->cidx; /* stable snapshot */
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
cidx = be16toh(cidx);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
if (cidx >= eq->cidx)
Calculate how many descriptors can be reclaimed before calling reclaim_tx_descs
2011-03-05 03:54:37 +00:00
return (cidx - eq->cidx);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
else
Calculate how many descriptors can be reclaimed before calling reclaim_tx_descs
2011-03-05 03:54:37 +00:00
return (cidx + eq->cap - eq->cidx);
}
/*
* There are "can_reclaim" tx descriptors ready to be reclaimed. Reclaim as
* many as possible but stop when there are around "n" mbufs to free.
*
* The actual number reclaimed is provided as the return value.
*/
static int
- Move all Ethernet specific items from sge_eq to sge_txq. sge_eq is now a suitable base for all kinds of egress queues. - Add control queues (sge_ctrlq) and allocate one of these per hardware channel. They can be used to program filters and steer traffic (and more). MFC after: 1 week
2011-04-19 22:08:28 +00:00
reclaim_tx_descs(struct sge_txq *txq, int can_reclaim, int n)
Calculate how many descriptors can be reclaimed before calling reclaim_tx_descs
2011-03-05 03:54:37 +00:00
{
struct tx_sdesc *txsd;
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
struct tx_maps *txmaps;
Calculate how many descriptors can be reclaimed before calling reclaim_tx_descs
2011-03-05 03:54:37 +00:00
struct tx_map *txm;
unsigned int reclaimed, maps;
- Move all Ethernet specific items from sge_eq to sge_txq. sge_eq is now a suitable base for all kinds of egress queues. - Add control queues (sge_ctrlq) and allocate one of these per hardware channel. They can be used to program filters and steer traffic (and more). MFC after: 1 week
2011-04-19 22:08:28 +00:00
struct sge_eq *eq = &txq->eq;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
TXQ_LOCK_ASSERT_OWNED(txq);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
Calculate how many descriptors can be reclaimed before calling reclaim_tx_descs
2011-03-05 03:54:37 +00:00
if (can_reclaim == 0)
can_reclaim = reclaimable(eq);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
maps = reclaimed = 0;
Calculate how many descriptors can be reclaimed before calling reclaim_tx_descs
2011-03-05 03:54:37 +00:00
while (can_reclaim && maps < n) {
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
int ndesc;
- Move all Ethernet specific items from sge_eq to sge_txq. sge_eq is now a suitable base for all kinds of egress queues. - Add control queues (sge_ctrlq) and allocate one of these per hardware channel. They can be used to program filters and steer traffic (and more). MFC after: 1 week
2011-04-19 22:08:28 +00:00
txsd = &txq->sdesc[eq->cidx];
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
ndesc = txsd->desc_used;
/* Firmware doesn't return "partial" credits. */
KASSERT(can_reclaim >= ndesc,
("%s: unexpected number of credits: %d, %d",
__func__, can_reclaim, ndesc));
- Move all Ethernet specific items from sge_eq to sge_txq. sge_eq is now a suitable base for all kinds of egress queues. - Add control queues (sge_ctrlq) and allocate one of these per hardware channel. They can be used to program filters and steer traffic (and more). MFC after: 1 week
2011-04-19 22:08:28 +00:00
maps += txsd->credits;
Calculate how many descriptors can be reclaimed before calling reclaim_tx_descs
2011-03-05 03:54:37 +00:00
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
reclaimed += ndesc;
Calculate how many descriptors can be reclaimed before calling reclaim_tx_descs
2011-03-05 03:54:37 +00:00
can_reclaim -= ndesc;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
eq->cidx += ndesc;
Calculate how many descriptors can be reclaimed before calling reclaim_tx_descs
2011-03-05 03:54:37 +00:00
if (__predict_false(eq->cidx >= eq->cap))
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
eq->cidx -= eq->cap;
Calculate how many descriptors can be reclaimed before calling reclaim_tx_descs
2011-03-05 03:54:37 +00:00
}
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
txmaps = &txq->txmaps;
txm = &txmaps->maps[txmaps->map_cidx];
Calculate how many descriptors can be reclaimed before calling reclaim_tx_descs
2011-03-05 03:54:37 +00:00
if (maps)
prefetch(txm->m);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
eq->avail += reclaimed;
KASSERT(eq->avail < eq->cap, /* avail tops out at (cap - 1) */
("%s: too many descriptors available", __func__));
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
txmaps->map_avail += maps;
KASSERT(txmaps->map_avail <= txmaps->map_total,
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
("%s: too many maps available", __func__));
while (maps--) {
Calculate how many descriptors can be reclaimed before calling reclaim_tx_descs
2011-03-05 03:54:37 +00:00
struct tx_map *next;
next = txm + 1;
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
if (__predict_false(txmaps->map_cidx + 1 == txmaps->map_total))
next = txmaps->maps;
Calculate how many descriptors can be reclaimed before calling reclaim_tx_descs
2011-03-05 03:54:37 +00:00
prefetch(next->m);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
- Move all Ethernet specific items from sge_eq to sge_txq. sge_eq is now a suitable base for all kinds of egress queues. - Add control queues (sge_ctrlq) and allocate one of these per hardware channel. They can be used to program filters and steer traffic (and more). MFC after: 1 week
2011-04-19 22:08:28 +00:00
bus_dmamap_unload(txq->tx_tag, txm->map);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
m_freem(txm->m);
txm->m = NULL;
Calculate how many descriptors can be reclaimed before calling reclaim_tx_descs
2011-03-05 03:54:37 +00:00
txm = next;
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
if (__predict_false(++txmaps->map_cidx == txmaps->map_total))
txmaps->map_cidx = 0;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
}
return (reclaimed);
}
static void
write_eqflush_wr(struct sge_eq *eq)
{
struct fw_eq_flush_wr *wr;
EQ_LOCK_ASSERT_OWNED(eq);
KASSERT(eq->avail > 0, ("%s: no descriptors left.", __func__));
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
KASSERT(!(eq->flags & EQ_CRFLUSHED), ("%s: flushed already", __func__));
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
wr = (void *)&eq->desc[eq->pidx];
bzero(wr, sizeof(*wr));
wr->opcode = FW_EQ_FLUSH_WR;
wr->equiq_to_len16 = htobe32(V_FW_WR_LEN16(sizeof(*wr) / 16) |
F_FW_WR_EQUEQ | F_FW_WR_EQUIQ);
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
eq->flags |= (EQ_CRFLUSHED | EQ_STALLED);
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
eq->pending++;
eq->avail--;
if (++eq->pidx == eq->cap)
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
eq->pidx = 0;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
}
static __be64
get_flit(bus_dma_segment_t *sgl, int nsegs, int idx)
{
int i = (idx / 3) * 2;
switch (idx % 3) {
case 0: {
__be64 rc;
rc = htobe32(sgl[i].ds_len);
if (i + 1 < nsegs)
rc |= (uint64_t)htobe32(sgl[i + 1].ds_len) << 32;
return (rc);
}
case 1:
return htobe64(sgl[i].ds_addr);
case 2:
return htobe64(sgl[i + 1].ds_addr);
}
return (0);
}
static void
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
find_best_refill_source(struct adapter *sc, struct sge_fl *fl, int maxp)
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
{
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
int8_t zidx, hwidx, idx;
uint16_t region1, region3;
int spare, spare_needed, n;
struct sw_zone_info *swz;
struct hw_buf_info *hwb, *hwb_list = &sc->sge.hw_buf_info[0];
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
/*
* Buffer Packing: Look for PAGE_SIZE or larger zone which has a bufsize
* large enough for the max payload and cluster metadata. Otherwise
* settle for the largest bufsize that leaves enough room in the cluster
* for metadata.
*
* Without buffer packing: Look for the smallest zone which has a
* bufsize large enough for the max payload. Settle for the largest
* bufsize available if there's nothing big enough for max payload.
*/
spare_needed = fl->flags & FL_BUF_PACKING ? CL_METADATA_SIZE : 0;
swz = &sc->sge.sw_zone_info[0];
hwidx = -1;
for (zidx = 0; zidx < SW_ZONE_SIZES; zidx++, swz++) {
if (swz->size > largest_rx_cluster) {
if (__predict_true(hwidx != -1))
break;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
/*
* This is a misconfiguration. largest_rx_cluster is
* preventing us from finding a refill source. See
* dev.t5nex.<n>.buffer_sizes to figure out why.
*/
device_printf(sc->dev, "largest_rx_cluster=%u leaves no"
" refill source for fl %p (dma %u). Ignored.\n",
largest_rx_cluster, fl, maxp);
}
for (idx = swz->head_hwidx; idx != -1; idx = hwb->next) {
hwb = &hwb_list[idx];
spare = swz->size - hwb->size;
if (spare < spare_needed)
continue;
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
hwidx = idx; /* best option so far */
if (hwb->size >= maxp) {
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
if ((fl->flags & FL_BUF_PACKING) == 0)
goto done; /* stop looking (not packing) */
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
if (swz->size >= safest_rx_cluster)
goto done; /* stop looking (packing) */
}
break; /* keep looking, next zone */
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
}
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
}
done:
/* A usable hwidx has been located. */
MPASS(hwidx != -1);
hwb = &hwb_list[hwidx];
zidx = hwb->zidx;
swz = &sc->sge.sw_zone_info[zidx];
region1 = 0;
region3 = swz->size - hwb->size;
/*
* Stay within this zone and see if there is a better match when mbuf
* inlining is allowed. Remember that the hwidx's are sorted in
* decreasing order of size (so in increasing order of spare area).
*/
for (idx = hwidx; idx != -1; idx = hwb->next) {
hwb = &hwb_list[idx];
spare = swz->size - hwb->size;
if (allow_mbufs_in_cluster == 0 || hwb->size < maxp)
break;
if (spare < CL_METADATA_SIZE + MSIZE)
continue;
n = (spare - CL_METADATA_SIZE) / MSIZE;
if (n > howmany(hwb->size, maxp))
break;
hwidx = idx;
if (fl->flags & FL_BUF_PACKING) {
region1 = n * MSIZE;
region3 = spare - region1;
} else {
region1 = MSIZE;
region3 = spare - region1;
break;
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
}
}
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
KASSERT(zidx >= 0 && zidx < SW_ZONE_SIZES,
("%s: bad zone %d for fl %p, maxp %d", __func__, zidx, fl, maxp));
KASSERT(hwidx >= 0 && hwidx <= SGE_FLBUF_SIZES,
("%s: bad hwidx %d for fl %p, maxp %d", __func__, hwidx, fl, maxp));
KASSERT(region1 + sc->sge.hw_buf_info[hwidx].size + region3 ==
sc->sge.sw_zone_info[zidx].size,
("%s: bad buffer layout for fl %p, maxp %d. "
"cl %d; r1 %d, payload %d, r3 %d", __func__, fl, maxp,
sc->sge.sw_zone_info[zidx].size, region1,
sc->sge.hw_buf_info[hwidx].size, region3));
if (fl->flags & FL_BUF_PACKING || region1 > 0) {
KASSERT(region3 >= CL_METADATA_SIZE,
("%s: no room for metadata. fl %p, maxp %d; "
"cl %d; r1 %d, payload %d, r3 %d", __func__, fl, maxp,
sc->sge.sw_zone_info[zidx].size, region1,
sc->sge.hw_buf_info[hwidx].size, region3));
KASSERT(region1 % MSIZE == 0,
("%s: bad mbuf region for fl %p, maxp %d. "
"cl %d; r1 %d, payload %d, r3 %d", __func__, fl, maxp,
sc->sge.sw_zone_info[zidx].size, region1,
sc->sge.hw_buf_info[hwidx].size, region3));
}
fl->cll_def.zidx = zidx;
fl->cll_def.hwidx = hwidx;
fl->cll_def.region1 = region1;
fl->cll_def.region3 = region3;
}
static void
find_safe_refill_source(struct adapter *sc, struct sge_fl *fl)
{
struct sge *s = &sc->sge;
struct hw_buf_info *hwb;
struct sw_zone_info *swz;
int spare;
int8_t hwidx;
if (fl->flags & FL_BUF_PACKING)
hwidx = s->safe_hwidx2; /* with room for metadata */
else if (allow_mbufs_in_cluster && s->safe_hwidx2 != -1) {
hwidx = s->safe_hwidx2;
hwb = &s->hw_buf_info[hwidx];
swz = &s->sw_zone_info[hwb->zidx];
spare = swz->size - hwb->size;
/* no good if there isn't room for an mbuf as well */
if (spare < CL_METADATA_SIZE + MSIZE)
hwidx = s->safe_hwidx1;
} else
hwidx = s->safe_hwidx1;
if (hwidx == -1) {
/* No fallback source */
fl->cll_alt.hwidx = -1;
fl->cll_alt.zidx = -1;
return;
}
hwb = &s->hw_buf_info[hwidx];
swz = &s->sw_zone_info[hwb->zidx];
spare = swz->size - hwb->size;
fl->cll_alt.hwidx = hwidx;
fl->cll_alt.zidx = hwb->zidx;
if (allow_mbufs_in_cluster)
fl->cll_alt.region1 = ((spare - CL_METADATA_SIZE) / MSIZE) * MSIZE;
Implement support for rx buffer packing. Enable it by default for T5 cards. This is a T4 and T5 chip feature which lets the chip deliver multiple Ethernet frames in a single buffer. This is more efficient within the chip, in the driver, and reduces wastage of space in rx buffers. - Always allocate rx buffers from the jumbop zone, no matter what the MTU is. Do not use the normal cluster refcounting mechanism. - Reserve space for an mbuf and a refcount in the cluster itself and let the chip DMA multiple frames in the rest. - Use the embedded mbuf for the first frame and allocate mbufs on the fly for any additional frames delivered in the cluster. Each of these mbufs has a reference on the underlying cluster.
2013-08-30 01:45:36 +00:00
else
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
fl->cll_alt.region1 = 0;
fl->cll_alt.region3 = spare - fl->cll_alt.region1;
cxgbe(4) - NIC driver for Chelsio T4 (Terminator 4) based 10Gb/1Gb adapters. MFC after: 3 weeks
2011-02-18 08:00:26 +00:00
}
Resume tx immediately in response to an SGE egress update from the hardware. MFC after: 1 week
2011-03-05 03:06:38 +00:00
- Specialized ingress queues that take interrupts for other ingress queues. Try to have a set of these per port when possible, fall back to sharing a common pool between all ports otherwise. - One control queue per port (used to be one per hardware channel). - t4_eth_rx now handles Ethernet rx only. - sysctls to display pidx/cidx for some queues. MFC after: 1 week
2011-05-30 21:34:44 +00:00
static void
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
add_fl_to_sfl(struct adapter *sc, struct sge_fl *fl)
- Specialized ingress queues that take interrupts for other ingress queues. Try to have a set of these per port when possible, fall back to sharing a common pool between all ports otherwise. - One control queue per port (used to be one per hardware channel). - t4_eth_rx now handles Ethernet rx only. - sysctls to display pidx/cidx for some queues. MFC after: 1 week
2011-05-30 21:34:44 +00:00
{
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
mtx_lock(&sc->sfl_lock);
FL_LOCK(fl);
if ((fl->flags & FL_DOOMED) == 0) {
fl->flags |= FL_STARVING;
TAILQ_INSERT_TAIL(&sc->sfl, fl, link);
callout_reset(&sc->sfl_callout, hz / 5, refill_sfl, sc);
- Specialized ingress queues that take interrupts for other ingress queues. Try to have a set of these per port when possible, fall back to sharing a common pool between all ports otherwise. - One control queue per port (used to be one per hardware channel). - t4_eth_rx now handles Ethernet rx only. - sysctls to display pidx/cidx for some queues. MFC after: 1 week
2011-05-30 21:34:44 +00:00
}
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
FL_UNLOCK(fl);
mtx_unlock(&sc->sfl_lock);
- Specialized ingress queues that take interrupts for other ingress queues. Try to have a set of these per port when possible, fall back to sharing a common pool between all ports otherwise. - One control queue per port (used to be one per hardware channel). - t4_eth_rx now handles Ethernet rx only. - sysctls to display pidx/cidx for some queues. MFC after: 1 week
2011-05-30 21:34:44 +00:00
}
- Move all Ethernet specific items from sge_eq to sge_txq. sge_eq is now a suitable base for all kinds of egress queues. - Add control queues (sge_ctrlq) and allocate one of these per hardware channel. They can be used to program filters and steer traffic (and more). MFC after: 1 week
2011-04-19 22:08:28 +00:00
static int
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
handle_sge_egr_update(struct sge_iq *iq, const struct rss_header *rss,
struct mbuf *m)
- Move all Ethernet specific items from sge_eq to sge_txq. sge_eq is now a suitable base for all kinds of egress queues. - Add control queues (sge_ctrlq) and allocate one of these per hardware channel. They can be used to program filters and steer traffic (and more). MFC after: 1 week
2011-04-19 22:08:28 +00:00
{
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
const struct cpl_sge_egr_update *cpl = (const void *)(rss + 1);
unsigned int qid = G_EGR_QID(ntohl(cpl->opcode_qid));
struct adapter *sc = iq->adapter;
struct sge *s = &sc->sge;
struct sge_eq *eq;
- Move all Ethernet specific items from sge_eq to sge_txq. sge_eq is now a suitable base for all kinds of egress queues. - Add control queues (sge_ctrlq) and allocate one of these per hardware channel. They can be used to program filters and steer traffic (and more). MFC after: 1 week
2011-04-19 22:08:28 +00:00
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
KASSERT(m == NULL, ("%s: payload with opcode %02x", __func__,
rss->opcode));
- Move all Ethernet specific items from sge_eq to sge_txq. sge_eq is now a suitable base for all kinds of egress queues. - Add control queues (sge_ctrlq) and allocate one of these per hardware channel. They can be used to program filters and steer traffic (and more). MFC after: 1 week
2011-04-19 22:08:28 +00:00
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
eq = s->eqmap[qid - s->eq_start];
- Move all Ethernet specific items from sge_eq to sge_txq. sge_eq is now a suitable base for all kinds of egress queues. - Add control queues (sge_ctrlq) and allocate one of these per hardware channel. They can be used to program filters and steer traffic (and more). MFC after: 1 week
2011-04-19 22:08:28 +00:00
EQ_LOCK(eq);
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
KASSERT(eq->flags & EQ_CRFLUSHED,
("%s: unsolicited egress update", __func__));
eq->flags &= ~EQ_CRFLUSHED;
eq->egr_update++;
if (__predict_false(eq->flags & EQ_DOOMED))
wakeup_one(eq);
else if (eq->flags & EQ_STALLED && can_resume_tx(eq))
taskqueue_enqueue(sc->tq[eq->tx_chan], &eq->tx_task);
EQ_UNLOCK(eq);
- Move all Ethernet specific items from sge_eq to sge_txq. sge_eq is now a suitable base for all kinds of egress queues. - Add control queues (sge_ctrlq) and allocate one of these per hardware channel. They can be used to program filters and steer traffic (and more). MFC after: 1 week
2011-04-19 22:08:28 +00:00
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
return (0);
}
- Move all Ethernet specific items from sge_eq to sge_txq. sge_eq is now a suitable base for all kinds of egress queues. - Add control queues (sge_ctrlq) and allocate one of these per hardware channel. They can be used to program filters and steer traffic (and more). MFC after: 1 week
2011-04-19 22:08:28 +00:00
cxgbe(4): Ask the card's firmware to pad up tiny CPLs by encapsulating them in a firmware message if it is able to do so. This works out better for one of the FIFOs in the chip. MFC after: 5 days
2013-02-26 00:27:27 +00:00
/* handle_fw_msg works for both fw4_msg and fw6_msg because this is valid */
CTASSERT(offsetof(struct cpl_fw4_msg, data) == \
offsetof(struct cpl_fw6_msg, data));
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
static int
Allow for a different handler for each type of firmware message. MFC after: 2 weeks
2012-08-16 18:31:50 +00:00
handle_fw_msg(struct sge_iq *iq, const struct rss_header *rss, struct mbuf *m)
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
{
Allow for a different handler for each type of firmware message. MFC after: 2 weeks
2012-08-16 18:31:50 +00:00
struct adapter *sc = iq->adapter;
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
const struct cpl_fw6_msg *cpl = (const void *)(rss + 1);
- Move all Ethernet specific items from sge_eq to sge_txq. sge_eq is now a suitable base for all kinds of egress queues. - Add control queues (sge_ctrlq) and allocate one of these per hardware channel. They can be used to program filters and steer traffic (and more). MFC after: 1 week
2011-04-19 22:08:28 +00:00
Many updates to cxgbe(4) - Device configuration via plain text config file. Also able to operate when not attached to the chip as the master driver. - Generic "work request" queue that serves as the base for both ctrl and ofld tx queues. - Generic interrupt handler routine that can process any event on any kind of ingress queue (via a dispatch table). - A couple of new driver ioctls. cxgbetool can now install a firmware to the card ("loadfw" command) and can read the card's memory ("memdump" and "tcb" commands). - Lots of assorted information within dev.t4nex.X.misc.* This is primarily for debugging and won't show up in sysctl -a. - Code to manage the L2 tables on the chip. - Updates to cxgbe(4) man page to go with the tunables that have changed. - Updates to the shared code in common/ - Updates to the driver-firmware interface (now at fw 1.4.16.0) MFC after: 1 month
2011-12-16 02:09:51 +00:00
KASSERT(m == NULL, ("%s: payload with opcode %02x", __func__,
rss->opcode));
- Move all Ethernet specific items from sge_eq to sge_txq. sge_eq is now a suitable base for all kinds of egress queues. - Add control queues (sge_ctrlq) and allocate one of these per hardware channel. They can be used to program filters and steer traffic (and more). MFC after: 1 week
2011-04-19 22:08:28 +00:00
cxgbe(4): Ask the card's firmware to pad up tiny CPLs by encapsulating them in a firmware message if it is able to do so. This works out better for one of the FIFOs in the chip. MFC after: 5 days
2013-02-26 00:27:27 +00:00
if (cpl->type == FW_TYPE_RSSCPL || cpl->type == FW6_TYPE_RSSCPL) {
const struct rss_header *rss2;
rss2 = (const struct rss_header *)&cpl->data[0];
return (sc->cpl_handler[rss2->opcode](iq, rss2, m));
}
Allow for a different handler for each type of firmware message. MFC after: 2 weeks
2012-08-16 18:31:50 +00:00
return (sc->fw_msg_handler[cpl->type](sc, &cpl->data[0]));
- Move all Ethernet specific items from sge_eq to sge_txq. sge_eq is now a suitable base for all kinds of egress queues. - Add control queues (sge_ctrlq) and allocate one of these per hardware channel. They can be used to program filters and steer traffic (and more). MFC after: 1 week
2011-04-19 22:08:28 +00:00
}
sysctl that displays the absolute queue id of an rxq.
2011-05-14 19:27:15 +00:00
static int
- Specialized ingress queues that take interrupts for other ingress queues. Try to have a set of these per port when possible, fall back to sharing a common pool between all ports otherwise. - One control queue per port (used to be one per hardware channel). - t4_eth_rx now handles Ethernet rx only. - sysctls to display pidx/cidx for some queues. MFC after: 1 week
2011-05-30 21:34:44 +00:00
sysctl_uint16(SYSCTL_HANDLER_ARGS)
sysctl that displays the absolute queue id of an rxq.
2011-05-14 19:27:15 +00:00
{
uint16_t *id = arg1;
int i = *id;
return sysctl_handle_int(oidp, &i, 0, req);
}
cxgbe(4): significant rx rework. - More flexible cluster size selection, including the ability to fall back to a safe cluster size (PAGE_SIZE from zone_jumbop by default) in case an allocation of a larger size fails. - A single get_fl_payload() function that assembles the payload into an mbuf chain for any kind of freelist. This replaces two variants: one for freelists with buffer packing enabled and another for those without. - Buffer packing with any sized cluster. It was limited to 4K clusters only before this change. - Enable buffer packing for TOE rx queues as well. - Statistics and tunables to go with all these changes. The driver's man page will be updated separately. MFC after: 5 weeks
2014-03-18 20:14:13 +00:00
static int
sysctl_bufsizes(SYSCTL_HANDLER_ARGS)
{
struct sge *s = arg1;
struct hw_buf_info *hwb = &s->hw_buf_info[0];
struct sw_zone_info *swz = &s->sw_zone_info[0];
int i, rc;
struct sbuf sb;
char c;
sbuf_new(&sb, NULL, 32, SBUF_AUTOEXTEND);
for (i = 0; i < SGE_FLBUF_SIZES; i++, hwb++) {
if (hwb->zidx >= 0 && swz[hwb->zidx].size <= largest_rx_cluster)
c = '*';
else
c = '\0';
sbuf_printf(&sb, "%u%c ", hwb->size, c);
}
sbuf_trim(&sb);
sbuf_finish(&sb);
rc = sysctl_handle_string(oidp, sbuf_data(&sb), sbuf_len(&sb), req);
sbuf_delete(&sb);
return (rc);
}
Reference in New Issue Copy Permalink