Commit Graph

88 Commits

Author SHA1 Message Date
eadler
a312b7ef4e Define missing DEBUGOUT# macros. DEBUGOUT[45] are not yet used but are
being defined pre-emptively to avoid future build breakage

PR:		kern/168967
Submitted by:	fuzhli <fuzl@arraynetworks.com.cn>
Approved by:	cperciva
MFC after:	1 week
2012-09-13 14:40:24 +00:00
scottl
076656eb8b Remove a prefetch() directive that, after careful testing, does more harm
than good.

Submitted by:	Fabien Thomas
Reviewed by:	jfv
2012-09-11 16:59:04 +00:00
kevlo
f9b0e29ecf Add missing braces
Obtained from:	DragonFly
2012-09-06 02:07:58 +00:00
scottl
693652fc56 Heavily optimize the case of small RX packets of 160 bytes or less. For
this case, allocate a plain mbuf and copy the frame into it, then send the
copy up the stack, leaving the original mbuf+cluster in place in the
receive ring for immediate re-use.  This saves a trip through 2 of the
3 zones of the compound mbuf allocator, a trip through busdma, and a trip
through the 1 of the 3 mbuf destructors.  For our load at Netflix, this can
lower CPU consumption by as much as 20%.  The copy algorithm is based on
investigative work from Luigi Rizzo earlier in the year.

Reviewed by:	jfv
Obtained from:	Netflix
2012-08-31 10:07:38 +00:00
jfv
8f97f54d9e Update to the ixgbe driver:
- Add a couple of new devices
  - Flow control changes in shared and core code
  - Bug fix to Flow Director for 82598
  - Shared code sync to internal with required core change

Thanks to those helping in the testing and improvements to this driver!

MFC after:5 days
2012-07-05 20:51:44 +00:00
emax
3479b2c8d6 Correct typo(?) and actually set PTHRESH to 32 and not 16 as per Intel
Linux driver 3.8.21.

MFC after:	1 week
2012-06-07 22:57:26 +00:00
emax
ce863bea70 Before it gets lost in the noise.
Put a bandaid to prevent ixgbe(4) from completely locking up the system
under high load. Our platform has a few CPU cores and a single active
ixgbe(4) port with 4 queues. Under high enough traffic load, at about
7.5GBs and 700,000 packets/sec (outbound), the entire system would
deadlock. What we found was that each CPU was in an endless loop on a
different ix taskqueue thread. The OACTIVE flag had gotten set on each
queue, and the ixgbe_handle_queue() function was continuously rescheduling
itself via the taskqueue_enqueue. Since all CPUs were busy with their
taskqueue threads, the ixgbe_local_timer() function couldn't run to clear
the OACTIVE flag.

Submitted by:	scottl
MFC after:	1 week
2012-06-05 18:48:02 +00:00
bz
1a89a35532 MFp4 bz_ipv6_fast:
Add TSO6 and LRO/IPv6 support.
  Fix the module Makefile to at least properly inlcude opt_inet6.h
  and allow builds without INET or INET6.

  Sponsored by:	The FreeBSD Foundation
  Sponsored by:	iXsystems

Reviewed by:	gnn (as part of the whole)
MFC After:	3 days
2012-05-25 03:02:56 +00:00
luigi
36ef5534f7 fix a typo in a comment 2012-05-17 14:36:19 +00:00
bz
22fae261d4 If we pass down 64k - L2 hdr size + 1 to 64K L3+ data adding an ether
header will make the data go over the 64k limits announced to busdma as
maxsize and the transaction will fail.

With TSO this can result in a TCP regression due to the lost packet.

According to the data sheets ixgbe(4) 82598 and 82599 can handle up to
256k so increase the maximum.

Reported by:	Jon Kåre Hellan, UNINETT (jon.kare.hellan uninett.no)
Tested by:	Jon Kåre Hellan, UNINETT (jon.kare.hellan uninett.no)
MFC after:	1 week
2012-04-23 22:05:09 +00:00
luigi
3cef61f727 Properly disable crc stripping when operating in netmap mode.
Contrarily to what i wrote in my previous commit, the 82599
does include the CRC in the length. The operating mode is
reset in ixgbe_init_locked() and so we need to hook into
the places where the two registers (HLREG0 and RDRXCTL) are
modified.
2012-04-13 16:42:54 +00:00
luigi
bef4cb5b9b Enable prefetching of descriptors on the TX ring, using the same
values as in the Intel driver 3.8.21 for linux.  The fact that it
is standard in the above driver suggests that it has no bad side
effects.

But of course there must be a reason for enabling features, not
just "it does not harm", so here it is a good one:

Prefetching enables full line rate even using a single queue (14.88
Mpps, compared to ~12 Mpps without prefetch).  This in turn is
terribly useful when one wants to schedule traffic.

For obvious reasons the difference is only visible with netmap
or other high speed solutions, but presumably the advantage
should be in the order of a fraction of a microsecond when
starting transmission on an empty queue.

Discussed with Jack Vogel.

MFC after:	1 week
2012-04-11 15:02:14 +00:00
scottl
2e7ae86807 More conversions of drivers to use the PCI parent DMA tag. 2012-03-12 18:15:08 +00:00
luigi
3ac0fcfb97 A bunch of netmap fixes:
USERSPACE:
1. add support for devices with different number of rx and tx queues;

2. add better support for zero-copy operation, adding an extra field
   to the netmap ring to indicate how many buffers we have already processed
   but not yet released (with help from Eddie Kohler);

3. The two changes above unfortunately require an API change, so while
   at it add a version field and some spares to the ioctl() argument
   to help detect mismatches.

4. update the manual page for the two changes above;

5. update sample applications in tools/tools/netmap

KERNEL:

1. simplify the internal structures moving the global wait queues
   to the 'struct netmap_adapter';

2. simplify the functions that map kring<->nic ring indexes

3. normalize device-specific code, helps mainteinance;

4. start exploring the impact of micro-optimizations (prefetch etc.)
   in the ixgbe driver.
   Use 'legacy' descriptors on the tx ring and prefetch slots gives
   about 20% speedup at 900 MHz. Another 7-10% would come from removing
   the explict calls to bus_dmamap* in the core (they are effectively
   NOPs in this case, but it takes expensive load of the per-buffer
   dma maps to figure out that they are all NULL.

   Rx performance not investigated.

I am postponing the MFC so i can import a few more improvements
before merging.
2012-02-27 19:05:01 +00:00
luigi
626117dcad (This commit only touches code within the DEV_NETMAP blocks)
Introduce some functions to map NIC ring indexes into netmap ring
indexes and vice versa. This way we can implement the bound
checks only in one place (and hopefully in a correct way).

On passing, make the code and comments more uniform across the
various drivers.
2012-02-15 23:13:29 +00:00
jfv
ae08051fd8 Wrap the bool typedef 2012-01-30 23:03:21 +00:00
jfv
f19302c07e New hardware support: Intel X540 adapter support added.
Some shared code reorganization along with the new adapter.
Sync changes to OACTIVE in igb into this driver.
Misc small fixes.
2012-01-30 16:42:02 +00:00
luigi
ef0f580b11 ixgbe changes:
- remove experimental code for disabling CRC
- use the correct constant for conversion between interrupt rate
  and EITR values (the previous values were off by a factor of 2)
- make dev.ix.N.queueM.interrupt_rate a RW sysctl variable.
  Changing individual values affects the queue immediately,
  and propagates to all interfaces at the next reinit.
- add dev.ix.N.queueM.irqs rdonly sysctl, to export the actual
  interrupt counts

Netmap-related changes for ixgbe:
- use the "new" format for TX descriptors in netmap mode.
- pass interrupt mitigation delays to the user process doing poll()
  on a netmap file descriptor.
  On the RX side this means we will not check the ring more than once
  per interrupt. This gives the process a chance to sleep and process
  packets in larger batches, thus reducing CPU usage.
  On the TX side we take this even further: completed transmissions are
  reclaimed every half ring even if the NIC interrupts more often.
  This saves even more CPU without any additional tx delays.

Generic Netmap-related changes:
- align the netmap_kring to cache lines so that there is no false sharing
  (possibly useful for multiqueue NICs and MSIX interrupts, which are
  handled by different cores). It's a minor improvement but it does not
  cost anything.

Reviewed by:	Jack Vogel
Approved by:	Jack Vogel
2012-01-26 09:55:16 +00:00
luigi
c20448129b netmap-related changes:
1. correct the initialization of RDT when there is an ixgbe_init()
   while a netmap client is active. This code was previously
   in ixgbe_initialize_receive_units() but RDT is overwritten
   shortly afterwards in ixgbe_init_locked()

2. add code (not active yet) to disable CRCSTRIP while in netmap mode.
   From all evidence i could gather, it seems that when the 82599 has to
   write a data block that is not a full cache line, it first reads
   the line (64 bytes) and then writes back the updated version.
   This hurts reception of min-sized frames, which are only 60 bytes
   if the CRC is stripped: i could never get above 11Mpps
   (received from one queue) with CRCSTRIP enabled, whyle 64+4-byte
   packets reach 14.2 Mpps (the theoretical maximum).
   Leaving the CRC in gets us 14.88Mpps for 60+4 byte frames,
   (and penalizes 64+4). The min-size case is important not just because
   it looks good in benchmarks, but also because this is the size
   of pure acks.
   Note we cannot leave CRCSTRIP on by default because it is
   incompatible with some other features (LRO etc.)
2012-01-19 09:36:19 +00:00
luigi
9b3b9221ef small code cleanup in preparation for future modifications in
the memory allocator used by netmap. No functional change,
two small bug fixes:
- in if_re.c add a missing bus_dmamap_sync()
- in netmap.c comment out a spurious free() in an error handling block
2012-01-10 19:57:23 +00:00
kevlo
41b065b8d4 ether_ifattach() sets if_mtu to ETHERMTU, don't bother set it again
Reviewed by:	yongari
2012-01-07 09:41:57 +00:00
mdf
d81aa8478d Consistently use types in ixgbe driver code:
- {ixgbe,ixv}_header_split is passed to TUNABLE_INT, so delcare it
   int, not bool.
 - {ixgbe,ixv}_tx_ctx_setup() returns a boolean value, so declare it
   bool, not int.
 - {ixgbe,ixv}_tso_setup() returns a bool, so declare it bool, not boolean_t.
 - {ixgbe,ixv}_txeof() returns a bool, so declare it bool, not boolean_t.
 - Do not re-define bool if the symbol already exists.

MFC after:	2 weeks
Sponsored by:	Isilon Systems, LLC
2011-12-12 18:27:28 +00:00
luigi
298ffde665 1. Fix the handling of link reset while in netmap more.
A link reset now is completely transparent for the netmap client:
   even if the NIC resets its own ring (e.g. restarting from 0),
   the client will not see any change in the current rx/tx positions,
   because the driver will keep track of the offset between the two.

2. make the device-specific code more uniform across different drivers
   There were some inconsistencies in the implementation of the netmap
   support routines, now drivers have been aligned to a common
   code structure.

3. import netmap support for ixgbe . This is implemented as a very
   small patch for ixgbe.c (233 lines, 11 chunks, mostly comments:
   in total the patch has only 54 lines of new code) , as most of
   the code is in an external file sys/dev/netmap/ixgbe_netmap.h ,
   following some initial comments from Jack Vogel about making
   changes less intrusive.
   (Note, i have emailed Jack multiple times asking if he had
   comments on this structure of the code; i got no reply so
   i assume he is fine with it).

Support for other drivers (em, lem, re, igb) will come later.

"ixgbe" is now the reference driver for netmap support. Both the
external file (sys/dev/netmap/ixgbe_netmap.h) and the device-specific
patches (in sys/dev/ixgbe/ixgbe.c) are heavily commented and should
serve as a reference for other device drivers.

Tested on i386 and amd64 with the pkt-gen program in tools/tools/netmap,
the sender does 14.88 Mpps at 1050 Mhz and 14.2 Mpps at 900 MHz
on an i7-860 with 4 cores and 82599 card. Haven't tried yet more
aggressive optimizations such as adding 'prefetch' instructions
in the time-critical parts of the code.
2011-12-05 12:06:53 +00:00
qingli
a3cb964a30 The maximum read size of incoming packets is done in 1024-byte increments.
The current code was rounding down the maximum frame size instead of
routing up, resulting in a read size of 1024 bytes, in the non-jumbo
frame case, and splitting the packets across multiple mbufs.

Consequently the above problem exposed another issue, which is when
packets were splitted across multiple mbufs, and all of the mbufs in the
chain have the M_PKTHDR flag set.

Submitted by:	original patch by Ray Ruvinskiy at BlueCoat dot com
Reviewed by:	jfv, kmacy, rwatson
Approved by:	re (rwatson)
MFC after:	5 days
2011-09-05 17:54:19 +00:00
jfv
a420ce0fad Cut and paste mistake corrected. 2011-06-02 05:31:54 +00:00
jfv
64fa4da5ee First off: update the driver README, the old one was horribly
crusty, and this still isn't perfect, but its at least a bit
more recent.

Secondly, a few improvements to the driver from Andrew Boyer,
support hint to allow devices to not attach, add VLAN_HWTSO
capability so vlans can use TSO, fix in the interrupt handler
to make sure the stack TX queue is processed. Oh, and also
make sure IPv6 does not cause a re-init in the ioctl routine.
Thanks for your efforts Andrew!

Thanks to Claudio Jeker for noticing the ixgbe_xmit() routine
was not correctly swapping the dma map from the first to the
last descriptor in a multi-descriptor transmission, corrected
this.
2011-06-02 00:34:57 +00:00
jfv
6ec3bafb99 Add a #define for driver portability to older OS 2011-04-28 23:21:40 +00:00
jfv
ebe803c491 - Add the RX refresh changes from igb to ixgbe
- Also a couple minor tweaks to the TX code from the same source.
- Add the INET ioctl code which has been missing from this driver,
  and which caused IP aliases to reset the interface.
- Last, some minor logic changes that just reflect upcoming
  hardware support, but have no other functional effect now.

MFC after a week
2011-04-25 23:34:21 +00:00
jhb
00c3c01f4f Do a sweep of the tree replacing calls to pci_find_extcap() with calls to
pci_find_cap() instead.
2011-03-23 13:10:15 +00:00
jfv
809122d5c7 Don't bother to run the flowcontrol code if there
is no change. Thanks to Andrew for the tweak.
2011-01-22 00:19:15 +00:00
jfv
2d67362e24 Missing case for 82598DA type adapter, thanks Andrew. 2011-01-22 00:08:06 +00:00
jfv
50008afcfe Leftover bogus TX UNLOCK removed. Thanks to
Andrew Boyer.
2011-01-21 23:55:28 +00:00
jfv
fbad453baa Update driver to version 2.3.8:
CRITICAL FIX - with stats changes the older 82598 will panic
	and trash the stack on driver load, FCOE registers ONLY exist
	in 82599 and must not be read otherwise.

	kern/153951 - to correct incorrect media type on adapters
	with pluggable modules I have eliminated the old static
	table in favor of a new dynamic shared code routine. This
	also has the benefit of detecting changes when a different
	module is inserted.

	Performance/enhancement to the Flow Director code from my
	linux coworker (the developer of the code).

	Fixes from Michael Tuexen - a data corruption problem on the
	82599 (CRITICAL), fix so the buf size correctly adjusts as
	the cluster changes, and max descriptors are set properly.
	Also added 16K clusters for those REALLY big jumbos :)

	In the RX path, the RX LOCK was not being released, and this
	causes LOR problems. Add the code that igb already has.

	Sync with in house shared code, this was necessary for the
	Flow Director fix.

MFC in 2 days
2011-01-19 19:36:27 +00:00
mdf
a21f29df61 Specify a CTLTYPE_FOO so that a future sysctl(8) change does not need
to rely on the format string.
2011-01-18 21:14:23 +00:00
mdf
0898b4847c sysctl(9) cleanup checkpoint: amd64 GENERIC builds cleanly.
Commit the Intel drivers.
2011-01-12 19:53:23 +00:00
jfv
16de4d95f5 CSUM flags need to be OS version sensitive in ixv code
MFC in 3 days
2011-01-07 23:39:41 +00:00
jfv
cc26d4fc8d kern/150247 - virtualization code also needs fix for 7.X to be buildable...
MFC in 3 days
2011-01-07 23:19:13 +00:00
jfv
9e1a6b2f78 Fix to kern/150247 - make ixgbe buildable for 7.x 2011-01-07 22:58:12 +00:00
jfv
b7f688f9ee kern/153772 fix variable names.
Thank you Andrew Boyer for catching these

MFC in 3 days
2011-01-07 22:34:56 +00:00
jfv
0aa625fd3e This small little change is a bug that drove me nuts
finding. The test to compare the mbuf m_len against
a fixed value and then returning needs to be removed.

When using VLANS and doing HW_TAGGING, and IPV6, the
ICMP6 packets actually fail this condition, the constant
assumes that the tag is IN the frame, and its not, so
the length is actually tiny. Furthermore, I'm not sure
what the point was to just return??

MFC after: 3 days
2010-12-04 01:43:38 +00:00
jfv
27ac42d29b Interrupt handler, and stats changes from Michael Tuexen,
thanks Michael!
2010-11-27 01:34:09 +00:00
jfv
a79f9a9e7d and the header... 2010-11-27 00:00:33 +00:00
jfv
649f983bb9 A couple fixes got clobbered, putting them back. 2010-11-26 23:57:13 +00:00
jfv
48bbd73f9a Update ixgbe driver to verion 2.3.6
- This adds a VM SRIOV interface, ixv, it is however
	  transparent to the user, it links with the ixgbe.ko,
	  but when ixgbe is loaded in a virtualized guest with
	  SRIOV configured this will be detected.
	- Sync shared code to latest
	- Many bug fixes and improvements, thanks to everyone
	  who has been using the driver and reporting issues.
2010-11-26 22:46:32 +00:00
brucec
696c4e1f9b Fix typos.
PR:	bin/148894
Submitted by:	olgeni
2010-11-09 10:59:09 +00:00
yongari
8203781f0e Do not allocate multicast array memory in multicast filter
configuration function. For failed memory allocations, em(4)/lem(4)
called panic(9) which is not acceptable on production box.
igb(4)/ixgb(4)/ix(4) allocated the required memory in stack which
consumed 768 bytes of stack memory which looks too big.

To address these issues, allocate multicast array memory in device
attach time and make multicast configuration success under any
conditions. This change also removes the excessive use of memory in
stack.

Reviewed by:	jfv
2010-08-28 00:34:22 +00:00
yongari
56b7a9338b Do not call voluntary panic(9) in case of if_alloc() failure.
Reviewed by:	jfv
2010-08-28 00:09:19 +00:00
kevlo
08695e1982 Fix build 2010-07-01 05:03:24 +00:00
jfv
af5d2bea56 Left out header change in last delta - new member
in adapter so that advertise changes can be done
to one port without the other changing.
2010-06-30 16:28:28 +00:00
glebius
9d2ae47a68 Fix build. 2010-06-30 11:17:55 +00:00