Commit Graph

285 Commits

Author SHA1 Message Date
Gleb Smirnoff
063efed28c The drbr(9) API appeared to be so unclear, that most drivers in
tree used it incorrectly, which lead to inaccurate overrated
if_obytes accounting. The drbr(9) used to update ifnet stats on
drbr_enqueue(), which is not accurate since enqueuing doesn't
imply successful processing by driver. Dequeuing neither mean
that. Most drivers also called drbr_stats_update() which did
accounting again, leading to doubled if_obytes statistics. And
in case of severe transmitting, when a packet could be several
times enqueued and dequeued it could have been accounted several
times.

o Thus, make drbr(9) API thinner. Now drbr(9) merely chooses between
  ALTQ queueing or buf_ring(9) queueing.
  - It doesn't touch the buf_ring stats any more.
  - It doesn't touch ifnet stats anymore.
  - drbr_stats_update() no longer exists.

o buf_ring(9) handles its stats itself:
  - It handles br_drops itself.
  - br_prod_bytes stats are dropped. Rationale: no one ever
    reads them but update of a common counter on every packet
    negatively affects performance due to excessive cache
    invalidation.
  - buf_ring_enqueue_bytes() reduced to buf_ring_enqueue(), since
    we no longer account bytes.

o Drivers handle their stats theirselves: if_obytes, if_omcasts.

o mlx4(4), igb(4), em(4), vxge(4), oce(4) and  ixv(4) no longer
  use drbr_stats_update(), and update ifnet stats theirselves.

o bxe(4) was the most correct driver, it didn't call
  drbr_stats_update(), thus it was the only driver accurate under
  moderate load. Now it also maintains stats itself.

o ixgbe(4) had already taken stats from hardware, so just
  - drop software stats updating.
  - take multicast packet count from hardware as well.

o mxge(4) just no longer needs NO_SLOW_STATS define.

o cxgb(4), cxgbe(4) need no change, since they obtain stats
  from hardware.

Reviewed by:	jfv, gnn
2012-09-28 18:28:27 +00:00
John Baldwin
aceb040376 Merge similar fixes from 223198 from igb to ixgbe:
- Use a dedicated task to handle deferred transmits from the if_transmit
  method instead of reusing the existing per-queue interrupt task.
  Reusing the per-queue interrupt task could result in both an interrupt
  thread and the taskqueue thread trying to handle received packets on a
  single queue resulting in out-of-order packet processing and lock
  contention.
- Don't define ixgbe_start() at all where if_transmit is used.

Tested by:	Vijay Singh
Reviewed by:	jfv
MFC after:	2 weeks
2012-09-26 18:11:43 +00:00
Sean Bruno
126a39ce60 This patch fixes a nit in the em, lem, and igb driver statistics. Increment
adapter->dropped_pkts instead of if_ierrors because if_ierrors is
overwritten by hw stats collection.

Submitted by:	Andrew Boyer <aboyer@averesystems.com>
Reviewed by:	Jack F Vogel <jfv@freebsd.org>
MFC after:	2 weeks
2012-09-23 22:53:39 +00:00
Gavin Atkinson
e935190a33 Switch some PCI register reads from using magic numbers to using the names
defined in pcireg.h

MFC after:	1 week
2012-09-19 12:27:23 +00:00
Gavin Atkinson
389c8bd51e Align the PCI Express #defines with the style used for the PCI-X
#defines.  This also has the advantage that it makes the names more
compact, iand also allows us to correct the non-uniform naming of
the PCIM_LINK_* defines, making them all consistent amongst themselves.

This is a mostly mechanical rename:
  s/PCIR_EXPRESS_/PCIER_/g
  s/PCIM_EXP_/PCIEM_/g
  s/PCIM_LINK_/PCIEM_LINK_/g

When this is MFC'd, #defines will be added for the old names to assist
out-of-tree drivers.

Discussed with:	jhb
MFC after:	1 week
2012-09-18 22:04:59 +00:00
Eitan Adler
96240c89f0 Correct double "the the"
Approved by:	cperciva
MFC after:	3 days
2012-09-14 21:28:56 +00:00
Jack F Vogel
252781f47d Customer report of a panic on boot due to the old
"m_getjcl:invalid cluster type" that occurred some
time back with the igb driver. This happens often when
booting over the net. I believe the NIC hardware is left
in a warm state when handed over to the driver, and a stray
RX interrupt happens earlier than the code is prepared for
it to happen. This change was verified to fix the problem,
its kind of a bandaid... but it is similar to what was done
in the igb code.
2012-08-15 17:12:40 +00:00
Jack F Vogel
724f79462b Make the polling interface in igb able to handle
multiqueue, and correct the rxdone handling. Update
the polling man page to include igb as well.

Thanks to Mark Johnston for these changes.
2012-08-06 22:43:49 +00:00
Jack F Vogel
6aa4d618ca Correct the mq_start routine to avoid out-of-order
packet delivery, always enqueue when possible. Also
correct the DEPLETED test as multiple bits might be
set.  Thanks to Randall Stewart for the changes!
2012-08-06 20:44:05 +00:00
Sean Bruno
8844c80848 CPU_NEXT() already handles wrapping around to the beginning. Also, in a
system with sparse CPU IDs, you can have a valid CPU ID > mp_ncpus (e.g. if
you have two CPUs 0 and 4, with mp_maxid == 4 and mp_ncpus == 2).

Introduced at svn r235210

Submitted by:	jhb@
Reviewed by:	jfv@
2012-08-02 00:00:34 +00:00
Jack F Vogel
b4750260cd Clean up some unused leftover code from em
Make IRQ style a tuneable
Fix lock handling in the interrupt handler

MFC after:3 days
2012-07-31 18:44:10 +00:00
Luigi Rizzo
fc1fa1f2fe remove some extra testing code that slipped into the previous commit
Reported-by: Alexander Motin
2012-07-25 12:51:33 +00:00
Luigi Rizzo
8d1717963b Use legacy interrupts as a default. This gives up to 10% speedup
when used in qemu (and this driver is for non-PCIe cards,
so probably its largest use is in virtualized environments).

Approved by:	Jack Vogel
MFC after:	3 days
2012-07-25 11:28:15 +00:00
Jack F Vogel
fcc144ad4e Change the interface to the Energy Efficient Ethernet (EEE)
setting in the igb and em driver. This was necessitated by
a shared code change that I was given late in the game, a data
type changed from bool to int, in the last update I dealt with
it by a cast, but it was pointed out (thanks jhb) that there
was a potential problem with this. John suggested this safer
approach, and it is fine with me...

MFC after:2 days (to catch the 9.1 update)
2012-07-07 20:21:05 +00:00
Jack F Vogel
996922aeee Correct small regressions pointed out by jhb, thanks John.
MFC after:5 days
2012-07-05 23:36:17 +00:00
Jack F Vogel
ab5d036272 Sync with Intel internal source:
shared code update and small changes in core required
Add support for new i210/i211 devices
Improve queue calculation based on mac type

MFC after:5 days
2012-07-05 20:26:57 +00:00
John Baldwin
03b0ca8b28 Commit a portion of 233708 I missed earlier and don't include the
definition of igb_start() and igb_start_locked() (nor set if_start in
the ifnet) when igb(4) uses if_transmit.
2012-06-01 15:52:41 +00:00
Kevin Lo
4d8b94d278 Initialize "error" to zero when it's declared in em_setup_receive_ring() 2012-05-11 03:15:22 +00:00
Sean Bruno
daf8162d1f Modify the binding of queues to attach to as many CPUs
as possible when using more than one igb(4) adapter.  This
means that queues will not be bound to the same CPUs if
there are more CPUs availble.

This is only applicable to a system that has multiple interfaces.

Obtained from:	Yahoo! Inc.
MFC after:	3 days
2012-05-10 00:00:28 +00:00
Ed Maste
a3d2552747 Fix cut-and-paste comment error
Submitted by:	sbruno
2012-04-25 02:05:14 +00:00
John Baldwin
8546e82467 Reapply r223198 which was reverted in the previous vendor import. Some
portions were already reapplied in r233708:
- Use a dedicated task to handle deferred transmits from the if_transmit
  method instead of reusing the existing per-queue interrupt task.
  Reusing the per-queue interrupt task could result in both an interrupt
  thread and the taskqueue thread trying to handle received packets on a
  single queue resulting in out-of-order packet processing.
- Call ether_ifdetach() earlier in igb_detach().
- Drain tasks and free taskqueues during igb_detach().

MFC after:	1 week
2012-04-11 21:33:45 +00:00
John Baldwin
d8a8648379 Fix a few issues with transmit handling in em(4) and igb(4):
- Do not define the foo_start() methods or set if_start in the ifnet if
  multiq transmit is enabled.  Also, set if_transmit and if_qflush before
  ether_ifattach rather than after when multiq transmit is enabled.  This
  helps to ensure that the drivers never try to mix different transmit
  methods.
- Properly restart transmit during resume.  igb(4) was not restarting it
  at all, and em(4) was restarting even if the link was down and was
  calling the wrong method if multiq transmit was enabled.
- Remove all the 'more' handling for transmit completions.  Transmit
  completion processing does not have a processing limit, so it always
  runs to completion and never has more work to do when it returns.
  Instead, the previous code was returning 'true' anytime there were
  packets in the queue that weren't still in the process of being
  transmitted.  The effect was that the driver would continuously
  reschedule a task to process TX completions in effect running at 100%
  CPU polling the hardware until it finished transmitting all of the
  packets in the ring.  Now it will just wait for the next TX completion
  interrupt.
- Restart packet transmission when the link becomes active.
- Fix the MSI-X queue interrupt handlers to restart packet transmission if
  there are pending packets in the relevant software queue (IFQ or buf_ring)
  after processing TX completions.  This is the root cause for the OACTIVE
  hangs as if the MSI-X queue handler drained all the pending packets from
  the TX ring, nothing would ever restart it.  As such, remove some
  previously-added workarounds to reschedule a task to poll the TX ring
  anytime OACTIVE was set.

Tested by:	sbruno
Reviewed by:	jfv
MFC after:	1 week
2012-03-30 19:54:48 +00:00
Marius Strobl
b3f46df007 Initialize the mutexes used for the NVM and the swflag as MTX_DUPOK in
order to avoid otherwise harmless witness warnings when these are acquired
at the same time and due to both using MTX_NETWORK_LOCK as their type.
The right fix actually would be to use different, descriptive types for
these. However, the latter would require undesirable changes to the shared
code base. Another approach would be to just supply NULL as the type, which
was deemed as less desirable though as it would cause the unique but cryptic
name also to be used for the type and to diverge from the type used by other
network device drivers.

MFC after:	1 week
2012-03-24 15:15:34 +00:00
John Baldwin
c3173381be Properly handle failures in igb_setup_msix() by returning 0 if MSI or MSI-X
allocation fails.

Reviewed by:	jfv
MFC after:	2 weeks
2012-03-01 22:13:10 +00:00
Luigi Rizzo
64ae02c365 A bunch of netmap fixes:
USERSPACE:
1. add support for devices with different number of rx and tx queues;

2. add better support for zero-copy operation, adding an extra field
   to the netmap ring to indicate how many buffers we have already processed
   but not yet released (with help from Eddie Kohler);

3. The two changes above unfortunately require an API change, so while
   at it add a version field and some spares to the ioctl() argument
   to help detect mismatches.

4. update the manual page for the two changes above;

5. update sample applications in tools/tools/netmap

KERNEL:

1. simplify the internal structures moving the global wait queues
   to the 'struct netmap_adapter';

2. simplify the functions that map kring<->nic ring indexes

3. normalize device-specific code, helps mainteinance;

4. start exploring the impact of micro-optimizations (prefetch etc.)
   in the ixgbe driver.
   Use 'legacy' descriptors on the tx ring and prefetch slots gives
   about 20% speedup at 900 MHz. Another 7-10% would come from removing
   the explict calls to bus_dmamap* in the core (they are effectively
   NOPs in this case, but it takes expensive load of the per-buffer
   dma maps to figure out that they are all NULL.

   Rx performance not investigated.

I am postponing the MFC so i can import a few more improvements
before merging.
2012-02-27 19:05:01 +00:00
Luigi Rizzo
5644ccec61 (This commit only touches code within the DEV_NETMAP blocks)
Introduce some functions to map NIC ring indexes into netmap ring
indexes and vice versa. This way we can implement the bound
checks only in one place (and hopefully in a correct way).

On passing, make the code and comments more uniform across the
various drivers.
2012-02-15 23:13:29 +00:00
Eitan Adler
c6618b7f79 GS105v3 exhibit the same behavior
PR:			docs/135999
Submitted by:	Boris Kochergin <spawky@acm.poly.edu>
No objection from:	jfv
Approved by:	cperciva
MFC after:		3 days
2012-01-29 14:52:42 +00:00
Luigi Rizzo
ce9f43b467 clear the pointer after freeing the mbuf. Without that, we
risk a double free if the subsequent mbuf allocation fails.
This bug is not netmap-related and was introduced in  rev. 228387
2012-01-12 17:30:44 +00:00
Luigi Rizzo
467bd5c2cb fix the initialization of the rings when netmap is used,
to adapt it to the changes in  228387 .
Now the code is similar to the one used in other drivers.
Not applicable to stable/9 and stable/8
2012-01-12 17:28:00 +00:00
Luigi Rizzo
6e10c8b8c5 small code cleanup in preparation for future modifications in
the memory allocator used by netmap. No functional change,
two small bug fixes:
- in if_re.c add a missing bus_dmamap_sync()
- in netmap.c comment out a spurious free() in an error handling block
2012-01-10 19:57:23 +00:00
Kevin Lo
5bbe0c5357 ether_ifattach() sets if_mtu to ETHERMTU, don't bother set it again
Reviewed by:	yongari
2012-01-07 09:41:57 +00:00
Robert Watson
19d52de5f4 When extracting the VLAN tag from if_em and if_lem receive descriptor
rings, copy the whole VLAN tag, not just the VLAN ID.  This fixes a
problem in which VLAN priority information was dropped when using
offloaded VLAN processing with these drivers.

Discussed with:	jfv, rrs
Sponsored by:	ADARA Networks, Inc.
MFC after:	3 days
2012-01-05 17:30:15 +00:00
Luigi Rizzo
2d0d326d91 put back netmap support, deleted by mistake in a previous commit 2011-12-22 15:33:41 +00:00
John Baldwin
ef93f57495 Restore the sysctl changes from 223676 and 227309 lost in the previous
import:
- Add read-only sysctls for all of the tunables supported by the igb and
  em drivers.
- Make the per-instance 'enable_aim' sysctl truly per-instance by having it
  change a per-instance variable (which is used to control AIM) rather
  than having all of the per-instance sysctls operate on a single global
  variable.

While here, restore the previously existing hw.igb.rx_processing_limit
tunable as it is very useful to be able to set a default tunable that
applies to all adapters in the system.
2011-12-21 20:10:11 +00:00
Matthew D Fleming
30a497c860 Consistently use types in e1000 driver code:
- Two struct members eee_disable are used in a function that expects
   an int *, so declare them int, not bool.
 - igb_tx_ctx_setup() returns a boolean value, so declare it bool, not int.
 - igb_header_split is passed to TUNABLE_INT, so delcare it int, not bool.
 - igb_tso_setup() returns a bool, so declare it bool, not boolean_t.
 - Do not re-define bool/true/false if the symbols already exist.

MFC after:	2 weeks
Sponsored by:	Isilon Systems, LLC
2011-12-12 18:27:34 +00:00
Jack F Vogel
62aca36544 Last change still had an issue, one more time... 2011-12-11 18:46:14 +00:00
Jack F Vogel
133f283b45 Correct LINT build issues in the ioctl code. 2011-12-11 09:37:25 +00:00
Jack F Vogel
96b38ade36 Fix NETMAP code problem in the build. 2011-12-10 18:00:53 +00:00
Jack F Vogel
fd33ce416e Part 2 of 2 New deltas for the 1G drivers.
There have still been intermittent problems with apparent TX
hangs for some customers. These have been problematic to reproduce
but I believe these changes will address them. Testing on a number
of fronts have been positive.

EM: there is an important 'chicken bit' fix for 82574 in the shared
code this is supported in the core here.
    - The TX path has been tightened up to improve performance. In
      particular UDP with jumbo frames was having problems, and the
      changes here have improved that.
    - OACTIVE has been used more carefully on the theory that some
      hangs may be due to a problem in this interaction
    - Problems with the RX init code, the "lazy" allocation and
      ring initialization has been found to cause problems in some
      newer client systems, and as it really is not that big a win
      (its not in a hot path) it seems best to remove it.
    - HWTSO was broken when VLAN HWTAGGING or HWFILTER is used, I
      found this was due to an error in setting up the descriptors
      in em_xmit.

IGB:
    - TX is also improved here. With multiqueue I realized its very
      important to handle OACTIVE only under the CORE lock so there
      are no races between the queues.
    - Flow Control handling was broken in a couple ways, I have changed
      and I hope improved that in this delta.
    - UDP also had a problem in the TX path here, it was change to
      improve that.
    - On some hardware, with the driver static, a weird stray interrupt
      seems to sometimes fire and cause a panic in the RX mbuf refresh
      code. This is addressed by setting interrupts late in the init
      path, and also to set all interrupts bits off at the start of that.
2011-12-10 07:08:52 +00:00
Jack F Vogel
4dab5c3769 Part 1 of two parts, this is the shared code changes in
support of new deltas for both em and igb drivers.

Note that I am not able to track all the bugs fixed in
this code, I am a consumer of it as a component of my
core drivers. It is important to keep the FreeBSD drivers
up to date with it however.

One important note is there is a key fix for 82574 in this
update. Also, there are lots of white space changes, I am
not happy about them but have no control over it :)
2011-12-10 06:55:02 +00:00
Luigi Rizzo
579a6e3c4e add netmap support for "em", "lem", "igb" and "re".
On my hardware, "em" in netmap mode does about 1.388 Mpps
on one card (on an Asus motherboard), and 1.1 Mpps on another
card (PCIe bus). Both seem to be NIC-limited, because
i have the same rate even with the CPU running at 150 MHz.

On the "re" driver the tx throughput is around 420-450 Kpps
on various (8111C and the like) chipsets. On the Rx side
performance seems much better, and i can receive the full
load generated by the "em" cards.

"igb" is untested as i don't have the hardware.
2011-12-05 15:33:13 +00:00
Ed Schouten
6472ac3d8a Mark all SYSCTL_NODEs static that have no corresponding SYSCTL_DECLs.
The SYSCTL_NODE macro defines a list that stores all child-elements of
that node. If there's no SYSCTL_DECL macro anywhere else, there's no
reason why it shouldn't be static.
2011-11-07 15:43:11 +00:00
Eitan Adler
36daf0495a - change "is is" to "is" or "it is"
- change "the the" to "the"

Approved by:	lstewart
Approved by:	sahil (mentor)
MFC after:	3 days
2011-10-16 14:30:28 +00:00
Ryan Stone
11dc48dfde Clear transmit checksum offload context state upon lem(4) interface
initialization.  Prior to this change packets may be transmitted with an
incorrect checksum.

Em(4) already has an equivalent change in r213234.

Obtained From:  Sandvine
MFC After:      1 week
Approved by:    re (bz)
2011-09-17 13:48:09 +00:00
Jack F Vogel
980729b244 A fix to make the LINT-NOINET build happy, if this
works out the ixgbe driver should be changed as well.
2011-07-07 00:46:50 +00:00
John Baldwin
b37e0f6e9a - Add read-only sysctls for all of the tunables supported by the igb and
em drivers.
- Make the per-instance 'enable_aim' sysctl truly per-instance by having it
  change a per-instance variable (which is used to control AIM) rather
  than having all of the per-instance sysctls operate on a single global
  variable.

Reviewed by:	jfv (earlier version)
MFC after:	1 week
2011-06-29 16:20:52 +00:00
Jack F Vogel
61cf9896b6 Put back the global for rx processing due to popular demand. 2011-06-23 17:42:27 +00:00
Jack F Vogel
1c1010187a Eliminate some global tuneables in favor of adapter-specific,
particular flow control and dma coalesce. Also improve the
sysctl operation on those too.

Add IPv6 detection in the ioctl code, this was done for
ixgbe first, carrying that over.

Add resource ability to disable particular adapter.

Add HW TSO capability so vlans can make use of TSO
2011-06-20 22:59:29 +00:00
John Baldwin
1c2ed38455 - Use a dedicated task to handle deferred transmits from the if_transmit
method instead of reusing the existing per-queue interrupt task.
  Reusing the per-queue interrupt task could result in both an interrupt
  thread and the taskqueue thread trying to handle received packets on a
  single queue resulting in out-of-order packet processing.
- Don't define igb_start() at all on 8.0 and where if_transmit is used.
  Replace last remaining call to igb_start() with a loop to kick off
  transmit on each queue instead.
- Call ether_ifdetach() earlier in igb_detach().
- Drain tasks and free taskqueues during igb_detach().

Reviewed by:	jfv
MFC after:	1 week
2011-06-17 20:06:52 +00:00
Jack F Vogel
3cec53b8a7 Add an initialization to the error variable, without
this there is a rare return path that bogusly appears
to fail when it should not.  Also white space correction.

Thanks to Arnaud Lacombe for noticing the problem.
2011-05-05 17:28:45 +00:00
Jack F Vogel
efa439f740 Small change to make backporting to stable/7,
thanks to Arnaud Lacombe for suggesting it.
2011-04-28 22:21:53 +00:00
Jack F Vogel
cf696f26c9 Important update for the igb driver:
- Add the change made in em to the actual unrefreshed number
    of descriptors is used as a basis in rxeof on the way out
    to determine if more refresh is needed. NOTE: there is a
    difference in the ring setup in igb, this is not accidental,
    it is necessitated by hardware behavior, when you reset the
    newer adapters it will not let you write RDH, it ALWAYS sets
    it to 0. Thus the way em does it is not possible.
  - Change the sysctl handling of flow control, it will now make
    the change dynamically when the variable setting changes rather
    than requiring a reset.
  - Change the eee sysctl naming, validation found the old unintuitive :)
  - Last but not least, some important performance tweaks in the TX
    path, I found that UDP behavior could be drastically hindered or
    improved with just small changes in the start loop. What I have
    here is what testing has shown to be the best overall. Its interesting
    to note that changing the clean threshold to start at a full half of
    the ring, made a BIG difference in performance.  I hope that this
    will prove to be advantageous for most workloads.

MFC in a week.
2011-04-05 21:55:43 +00:00
Jack F Vogel
62d8da8c3a Fix to an error condition case, when an mbuf chain
get's defragged due to a mapping failure the header
pointers will be invalidated and can result in a
TSO or other failure down the line. So, when the
remapping occurs force a retry thru the offload
calculation code. Thanks to Andrew Boyer for discovering
this and cooking up the fix!!
2011-04-01 20:24:51 +00:00
Jack F Vogel
e61e0b91af Change the refresh_mbuf logic slightly, add an inline
to calculate the outstanding descriptors that need to be
refreshed at any time, and use THAT in rxeof to determine
if refreshing needs to be done. Also change the local_timer
to simply fire off the appropriate interrupt rather than
schedule a tasklet, its simpler.

MFC in two weeks
2011-04-01 18:48:31 +00:00
John Baldwin
3b0a4aef96 Do a sweep of the tree replacing calls to pci_find_extcap() with calls to
pci_find_cap() instead.
2011-03-23 13:10:15 +00:00
Jack F Vogel
d69437b146 A cut and paste here was wrong also. 2011-03-19 00:31:35 +00:00
Jack F Vogel
1c88efb221 Correct broken define 2011-03-19 00:19:18 +00:00
Jack F Vogel
1fd3c44f77 This delta updates the em driver to version 7.2.2 which has
been undergoing test for some weeks. This improves the RX
mbuf handling to avoid system hang due to depletion. Thanks
to all those who have been testing the code, and to Beezar
Liu for the design changes.

Next the igb driver is updated for similar RX changes, but
also to add new features support for our upcoming i350 family
of adapters.

MFC after a week
2011-03-18 18:54:00 +00:00
Rebecca Cran
6bccea7c2b Fix typos - remove duplicate "the".
PR:	bin/154928
Submitted by:	Eitan Adler <lists at eitanadler.com>
MFC after: 	3 days
2011-02-21 09:01:34 +00:00
Jack F Vogel
730d313078 Fix the shared code to be consistent with Intel-internal,
and now build.
2011-02-12 00:07:40 +00:00
Doug Barton
8c1bde8dd0 Restore 2 prototypes that seem to have been mistakenly removed in r218582.
I've manually twiddled the whitespace for e1000_commit_fc_settings_generic
to match the others in the file.

Submitted by:	dim
Tested by:	me
2011-02-11 23:08:34 +00:00
Jack F Vogel
15cb0f401a Somehow the RX ring depletion fix got partially removed,
replace the missing pieces.
2011-02-11 19:49:07 +00:00
Jack F Vogel
4b52dc6ad6 Revert changes made here, they will cause a conflict
later on with our shared code.
2011-02-11 19:03:00 +00:00
Jack F Vogel
fe954359d6 Inconsistencies in the updated igb shared code and the older
em/lem, breaking the build, correcting that.
2011-02-11 17:18:42 +00:00
Bjoern A. Zeeb
b60688be0f After r218530 export several functions which are no longer private to
e1000_mac.c but part of the e1000_api.

X-MFC with: 218530 by jfv
2011-02-11 09:58:38 +00:00
Jack F Vogel
f0ecc46d04 Add support for the new I350 family of 1G interfaces.
- this also includes virtualization support on these devices

Correct some vlan issues we were seeing in test, jumbo frames on vlans
did not work correctly, this was all due to confused logic around HW
filters, the new code should now work for all uses.

Important fix: when mbuf resources are depeleted, it was possible to
completely empty the RX ring, and then the RX engine would stall
forever. This is fixed by a flag being set whenever the refresh code
fails due to an mbuf shortage, also the local timer now makes sure
that all queues get an interrupt when it runs, the interrupt code
will then always call rxeof, and in that routine the first thing done
is now to check the refresh flag and call refresh_mbufs. This has been
verified to fix this type 'hang'. Similar code will follow in the other
drivers.

Finally, sync up shared code for the I350 support.

Thanks to everyone that has been reporting issues, and helping in the
debug/test process!!
2011-02-11 01:00:26 +00:00
Jack F Vogel
fbfbce8ae9 Fix for kern/152853, pullup at the wrong point
is breaking UDP. Thanks to Petr Lampa for the
patch.
2011-01-19 18:20:11 +00:00
Matthew D Fleming
5bc0787f29 Specify a CTLTYPE_FOO so that a future sysctl(8) change does not need
to rely on the format string.
2011-01-18 21:14:23 +00:00
Matthew D Fleming
8c49f18771 sysctl(9) cleanup checkpoint: amd64 GENERIC builds cleanly.
Commit the Intel drivers.
2011-01-12 19:53:23 +00:00
Jack F Vogel
599564e633 A couple problems discovered by Andrew Boyer:
- failure code in em_xmit got mangled along the way
     and was not properly handling errors.
   - local timer code had a leftover UNLOCK call that
     should be removed.

MFC after 3 days
2011-01-12 00:23:47 +00:00
Jack F Vogel
1ce42f7249 Correct build error. 2010-12-04 06:38:21 +00:00
Jack F Vogel
aa8dea58c6 Remove the bogus test in the TX context setup for IPV6,
the size can be smaller than the constant when you are
doing HW TAGGING, and you still need to process this
packet in a normal way. I'm not sure where the notion
to just return came from, but its wrong.

MFC after: 3 days
2010-12-04 02:04:02 +00:00
Jack F Vogel
9d43b64dbf Small cut and paste bug in flow control string fixed.
Second, correct the discard/refresh_mbufs code to behave
more like igb, there have been panics due to discards and
this should fix them.

MFC after: 3 days
2010-12-04 01:59:58 +00:00
Jack F Vogel
f4b4ba3ed2 Unwanted extra call to set_vlan_hw added back by mistake. 2010-11-26 22:36:47 +00:00
Jack F Vogel
12203744da The purpose of this change is to add a routine to
disable ASPM L0S and L1 LINK states on 82573, 82574,
and 82583. The theory is that this is behind certain
hangs being experienced by some customers.

Also included a small optimization in the rxeof routine
that was in my internal code.

Change the PBA size for pchlan, it was incorrect.

MFC after: 3 days
2010-11-24 22:24:07 +00:00
Jack F Vogel
ffdb071a24 Add shared code glue for new 82580 devices. 2010-11-24 01:13:55 +00:00
Jack F Vogel
46b9f1ff9f - New 82580 devices supported
- Fixes from John Baldwin: vlan shadow tables made per/interface,
  make vlan hw setup only happen when capability enabled, and
  finally, make a tuneable interrupt rate. Thanks John!
- Tweaked watchdog handling to avoid any false positives, now
  detection is in the TX clean path, with only the final check
  and init happening in the local timer.
- limit queues to 8 for all devices, with 82576 or 82580 on
  larger machines it can get greater than this, and it seems
  mostly a resource waste to do so. Even 8 might be high but
  it can be manually reduced.
- use 2k, 4k and now 9k clusters based on the MTU size.
- rework the igb_refresh_mbuf() code, its important to
  make sure the descriptor is rewritten even when reusing
  mbufs since writeback clobbers things.

MFC: in a few days, this delta needs to get to 8.2
2010-11-23 22:12:02 +00:00
Jack F Vogel
e4c690b4f0 Sync the lem code up with the vlan and other fixes in em.
Delete a unneeded test from the beginning of em_xmit.
CRITICAL: shared code fix for 82574, a mutex might not be
          released, this can cause hangs.
2010-11-01 20:19:25 +00:00
Jack F Vogel
35928b338e In the data setup code for doing offloads the
ip and tcp pointers were not reset after some
pullups. In practice this led to an NFS mount
failure when using UDP reported by Kevin Lo,
thanks Kevin. Fix from yongari, thank you!
2010-10-28 00:16:54 +00:00
Jack F Vogel
7deff7f9b4 Bug fix delta to the em driver:
- Chasin down bogus watchdogs has led to an improved
	  design to this handling, the hang decision takes
	  place in the tx cleanup, with only a simple report
	  check in local_timer. Our tests have shown no false
	  watchdogs with this code.
	- VLAN fixes from jhb, the shadow vfta should be per
	  interface, but as global it was not. Thanks John.
	- Bug fixes in the support for new PCH2 hardware.
	- Thanks for all the help and feedback on the driver,
	  changes to lem with be coming shortly as well.
2010-10-26 00:07:58 +00:00
Jack F Vogel
7d9119bdc4 Update code from Intel:
- Sync shared code with Intel internal
	- New client chipset support added
	- em driver - fixes to 82574, limit queues to 1 but use MSIX
	- em driver - large changes in TX checksum offload and tso
	  code, thanks to yongari.
	- some small changes for watchdog issues.
	- igb driver - local timer watchdog code was missing locking
	  this and a couple other watchdog related fixes.
	- bug in rx discard found by Andrew Boyer, check for null pointer

MFC: a week
2010-09-28 00:13:15 +00:00
John Baldwin
8385f4cf94 Tweak the stats exported by the e1000 drivers:
- Add a single sysctl procedure to all three drivers to read an arbitrary
  register (the register is passed as arg2).  Use it to replace existing
  routines in igb(4) that used a separate routine for each register, and
  to add support for missing stats in em(4) and lem(4).
- Move the 'rx_overruns' and 'watchdog_timeouts' stats out of the MAC stats
  section as they are driver stats, not MAC counters.
- Simplify the code that creates per-queue stats in igb(4) to use a single
  loop and remove duplicated code.
- Properly read all 64 bits of the 'good octets received/transmitted' in
  em(4) and lem(4).
- Actually read the interrupt count registers in em(4), and drop the
  'host to card' sysctl stats from em(4) as they are not implemented in
  any of the hardware this driver supports.
- Restore several stats to em(4) that were lost in the earlier stats
  conversion including per-queue stats.
- Export several MAC stats in em(4) that were exported in igb(4) but not
  in em(4).
- Export stats in lem(4) using individual sysctls as in em(4) and igb(4).

Reviewed by:	jfv
MFC after:	1 week
2010-09-20 16:04:44 +00:00
Jack F Vogel
26c88ee828 Code correction in refresh_mbufs, just continuing
without index recalc was wrong.
2010-09-07 21:28:45 +00:00
Jack F Vogel
d9f1a5aa8e Tighten up the rx mbuf refresh code, there were some
discrepencies from the igb version which was the target.

Change the message when neither MSI or MSIX are enabled
and a fallback to Legacy interrupts happen, the existing
message was confusing.
2010-09-07 20:13:08 +00:00
Pyun YongHyeon
dd20cce19a Do not allocate multicast array memory in multicast filter
configuration function. For failed memory allocations, em(4)/lem(4)
called panic(9) which is not acceptable on production box.
igb(4)/ixgb(4)/ix(4) allocated the required memory in stack which
consumed 768 bytes of stack memory which looks too big.

To address these issues, allocate multicast array memory in device
attach time and make multicast configuration success under any
conditions. This change also removes the excessive use of memory in
stack.

Reviewed by:	jfv
2010-08-28 00:34:22 +00:00
Pyun YongHyeon
880a50b513 If em(4) failed to allocate RX buffers, do not call panic(9).
Just showing some buffer allocation error is more appropriate
action for drivers. This should fix occasional panic reported on
em(4) when driver encountered resource shortage.

Reviewed by:	jfv
2010-08-28 00:16:49 +00:00
Pyun YongHyeon
ad1917be37 Do not call voluntary panic(9) in case of if_alloc() failure.
Reviewed by:	jfv
2010-08-28 00:09:19 +00:00
Pyun YongHyeon
5c8080f0d9 Make sure not to access unallocated stats memory.
Reviewed by:	jfv
2010-08-27 23:50:13 +00:00
Jack F Vogel
1ed622958d Eliminate the ambiguous queue setting logic for
the VF, it made it possible to have 2 queues which
we don't want, the HOST is unable to handle it.
2010-08-19 17:00:33 +00:00
Jack F Vogel
3b9b3fc3bf Put the early setting of the MAC type back, its
removal resulted in broken code in MSIX setup.
2010-08-06 20:55:49 +00:00
Matthew D Fleming
a767c78e4a Fix clang warning on empty statement.
Reviewed by:    rdivacky, zml
Approved by:    zml (mentor)
2010-07-28 16:24:06 +00:00
George V. Neville-Neil
6de27dc09f style(9) fix
MFC after:	1 week
2010-07-24 18:53:46 +00:00
George V. Neville-Neil
dc1ccafedd Fix a bug in the statistics code for tracking the head and
tail pointers of the tx and rx queues.   We needed a SYSCTL_PROC
to correctly get the values at run time.

Submitted by:	Andrew Boyer aboyer at averesystems.com
MFC after:	1 week
2010-07-23 17:53:39 +00:00
Jack F Vogel
9886a800fc Fix for a panic when TX checksum offload is done and
a packet has only a header in the first mbuf, the
checksum code will dereference a pointer into the
non-existing IP header. Do a check for the size and
pullup if needed. Thanks to Michael Tuexen for this
fix.

MFC: asap - should be in 8.1 IMHO
2010-07-12 21:47:30 +00:00
Jack F Vogel
d6ff3d4d13 Fix of a VLAN problem by jhb, the checksum capability
got lost along the way.

MFC: asap
2010-07-09 17:11:29 +00:00
Jack F Vogel
b827058579 OK, I was a bit sleep this morning and checked in
the core changes but left out the shared code, lol.
Well, and a couple fixes to the core... hopefully
this will all be complete now.

Happy happy joy joy :)
2010-06-30 21:05:51 +00:00
Jack F Vogel
911fddbcb5 SR-IOV support added to igb
What this provides is support for the 'virtual function'
interface that a FreeBSD VM may be assigned from a host
like KVM on Linux, or newer versions of Xen with such
support.

When the guest is set up with the capability, a special
limited function 82576 PCI device is present in its virtual
PCI space, so with this driver installed in the guest that
device will be detected and function nearly like the bare
metal, as it were.

The interface is only allowed a single queue in this configuration
however initial performance tests have looked very good.

Enjoy!!
2010-06-30 17:26:47 +00:00
George V. Neville-Neil
5ea4d522bb Make sure that all the exposed counters and variables are actually
being updated.

Pointed out by:	jfv
2010-06-24 21:17:58 +00:00
Jack F Vogel
b7741e7a13 Two stats were duplicated, thanks to Andrew Boyer
for pointing this out.
2010-06-17 17:38:39 +00:00
George V. Neville-Neil
fdbf7e3c5e Move statistics into the sysctl tree making it easier to find
and use them.
Add previously hidden statistics, some of which include interrupt
and host/card communication counters.
2010-06-16 20:57:41 +00:00
George V. Neville-Neil
9dc69a1589 Move statistics into the sysctl tree making it easier to find
and use them.
Add previously hidden statistics, some of which include interrupt
and host/card communication counters.
2010-06-16 17:36:53 +00:00
Jack F Vogel
dfc14ce06e Changes from John Baldwin adding to last commit,
change rxeof api for poll friendliness, and
eliminate unnecessary link tasklet use. Thanks John!
2010-06-16 16:37:36 +00:00
Jack F Vogel
0dee704f01 Change to have legacy interrupts use the same
handler had a flaw, thanks to John Baldwin for
finding it. Change which queue legacy tasks are
enqueued on.

MFC: soonest
2010-06-15 21:11:51 +00:00
Jack F Vogel
e6ec2d1fa6 Put back the lost bus_describe_intr() calls. 2010-06-11 21:35:19 +00:00
Jack F Vogel
2e6bd30b46 Add a couple fixes from Michael Tuexen.
Remove unneeded rxtx handler, make que handler generic.
Do not allocate header mbufs in rx ring if not doing hdr split.
Release the lock in rxeof call to stack.

MFC for 8.1 asap
2010-06-11 20:54:27 +00:00
John Baldwin
c29ae5520a Restore part of 200671 which was lost in previous driver changes:
- Add interrupt descriptions when using mulitple MSI-X interrupts.
2010-05-20 20:01:54 +00:00
Marius Strobl
876ab8b5e4 Fix a mismerge in r206001.
PR:		146614
Approved by:	jfv (implicit)
MFC afer:	3 days
2010-05-15 19:46:16 +00:00
Jack F Vogel
46168c5453 Small changes preparing for MFC, need to conditionalize
the buf_ring_free call, and lem is missing the WOL change
put into em.
2010-05-14 22:18:34 +00:00
Jack F Vogel
beef45ff88 Address the LOD that some are seeing, put the RX lock
back in rxeof (I could see little point in taking it out),
and now release it before the stack entry.

Also, make it so the 82574 does not configure for multiqueue
when its not used in the stack.
2010-04-28 19:22:52 +00:00
Jack F Vogel
1655af0a72 Change default WOL back to MAGIC only, having
multicast enabled causes problems in man environments.
2010-04-28 17:37:30 +00:00
Jack F Vogel
d43a118797 Add a missing fragment in the tx msix handler to invoke
another if all work is not done.

Sync the igb driver with changes suggested by yongari and
made in em, these made sense to be in both drivers.
2010-04-14 20:55:33 +00:00
Jack F Vogel
7ea05bb9a5 Remove multiqueue stack related stuff form lem, it is
unneeded for legacy hardware.
Also remove some TSO related cruft.
Add some watchdog_time setting that was missing, thanks
to Mikolaj Golub for pointing that out.
2010-04-14 18:29:01 +00:00
Jack F Vogel
3b4e5df82c The lock move in rxeof necessitated a couple
more places to do the locking, fixes a panic.
2010-04-10 19:25:55 +00:00
Jack F Vogel
b4ab02b842 Correct broken build. 2010-04-10 07:26:51 +00:00
Jack F Vogel
91ce5735bf A few more changes from yongari:
- code flow in handler could let interrupt be
	  reenabled when not wanted.
	- change where the RX lock is taken to improve
	  performance.
	- adapter->msix is true for MSI systems also,
	  it needs to explicitly test for 82574, good one :)
2010-04-09 23:15:37 +00:00
Jack F Vogel
99f8e467e8 DUH, must be tired, I missed the second instance...
time for the weekend :)
2010-04-09 21:18:46 +00:00
Jack F Vogel
af8a24c8d0 Thanks to Michael Tuexen for catching this, bit set that
keeps the clock from being reset when writing to EITR was
incorrect, also there is a shared code #define for it anyway.
2010-04-09 21:16:45 +00:00
Jack F Vogel
681ac9c018 Incorporate suggested improvements from yongari.
Also, from feedback, make the multiqueue code an
option (EM_MULTIQUEUE) that is off by default.
Problems have been seen with UDP when its on.
2010-04-09 18:42:15 +00:00
Jack F Vogel
476310d371 Three changes:
- add CRC stripping to the RX side, this was handled
	  by some obscure code in rxeof previously, its easier
	  to simply have the hardware strip it now.
	- Add back an ALTQ change that slipped between the cracks
	- Add an update to the watchdog_time in the xmit code, not
	  doing this in ixgbe caused problems, think its needed here
	  as well.
2010-04-08 19:13:42 +00:00
Jack F Vogel
500e8f26d7 Important fix got clobbered in the em driver, keeping
VLAN HWFILTER from being used by default, this breaks
stacked pseudo devices, and as it turns out, also breaks
virtual machines that happen to use VLANS (didn't know that
before :). Put the fix back into the em driver, and for good
measure add the same code to the igb driver where it should
have been anyway.
2010-04-08 00:50:43 +00:00
Jack F Vogel
657c04c3f8 The POLL code was missed in the queue conversion,
change the argument type to igb_rxeof() to the
correct type. Note, any users of POLLING must
be sure and set the number of queues to 1 for
things to work correctly.
2010-03-31 23:24:42 +00:00
Marius Strobl
79c7b7190f Hook the identification LEDs of igb(4), lem(4) and em(4) devices up with
led(4) so they can be lit or f.e. made blink via `echo f2 > /dev/led/em0`
for localization purposes.

Approved by:	jfv
MFC afer:	1 week (after r205869)
2010-03-31 20:43:24 +00:00
Jack F Vogel
772aec19a1 Fix poll handler declaration. 2010-03-31 16:42:22 +00:00
Jack F Vogel
eaa9db2bb6 Fix lint build problem. 2010-03-30 17:05:24 +00:00
Jack F Vogel
8ec87fc514 Update to igb and em:
em revision 7.0.0:
	- Using driver devclass, seperate legacy (pre-pcie) code
	  into a seperate source file. This will at least help
	  protect against regression issues. It compiles along
	  with em, and is transparent to end use, devices in each
	  appear to be 'emX'. When using em in a modular form this
	  also allows the legacy stuff to be defined out.
	- Add tx and rx rings as in igb, in the 82574 this becomes
	  actual multiqueue for the first time (2 queues) while in
	  other PCIE adapters its just make code cleaner.
	- Add RX mbuf handling logic that matches igb, this will
	  eliminate packet drops due to temporary mbuf shortage.

igb revision 1.9.3:
	- Following the ixgbe code, use a new approach in what
	  was called 'get_buf', the routine now has been made
	  independent of rxeof, it now does the update to the
	  engine TDT register, this design allows temporary
	  mbuf resources to become non-critical, not requiring
	  a packet to be discarded, instead it just returns and
	  does not increment the tail pointer.
	- With the above change it was also unnecessary to keep
	  'spare' maps around, since we do not have the discard
	  issue.
	- Performance tweaks and improvements to the code also.

MFC in a week
2010-03-29 23:36:34 +00:00
Max Laier
193cbc4d24 Fix drbr and altq interaction:
- introduce drbr_needs_enqueue that returns whether the interface/br needs
   an enqueue operation: returns true if altq is enabled or there are
   already packets in the ring (as we need to maintain packet order)
 - update all drbr consumers
 - fix drbr_flush
 - avoid using the driver queue (IFQ_DRV_*) in the altq case as the
   multiqueue consumer does not provide enough protection, serialize altq
   interaction with the main queue lock
 - make drbr_dequeue_cond work with altq

Discussed with:		kmacy, yongari, jfv
MFC after:		4 weeks
2010-02-13 16:04:58 +00:00
Jack F Vogel
29e1e1a3a7 A few minor changes: add altq option header, add missing conditional
around a buf_ring call that will break 7.3, and thanks to Fabien Thomas
add POLLING support for igb and a minor related fix in the em driver.
2010-02-01 19:28:43 +00:00
Jack F Vogel
146f25649d Fix for kern/141646: when stacking pseudo drivers like
lagg and vlan the vlan attach/detach event is not being
handed down to em, this caused some init code not to run,
and thus VLANs did not work. Ultimately having the event
get propagated would be nice, but for now the solution is
to have HWFILTER off by default, when this is the case
VLANs will work, ifconfig can be used to turn it on and
then get HW tag filtering.
2010-01-30 00:11:44 +00:00
Jack F Vogel
89d507fcf7 Add a link tasklet so updates can be sleepable. 2010-01-27 20:12:04 +00:00
Jack F Vogel
afb98829ac Two more build problems, missing includes and semicolon. 2010-01-27 18:00:24 +00:00
Jack F Vogel
6c8d4b16d4 Opps, completely wrong version of if_em.h got into
the checkin, sorry all :(
2010-01-27 17:35:58 +00:00
Jack F Vogel
ac95ee294c Missing a fix for the new watchdog handling. 2010-01-26 23:04:13 +00:00
Jack F Vogel
4b502b77de Remove some internal conditional defines that will
fail in kernel tree.
2010-01-26 22:38:31 +00:00
Jack F Vogel
a69ed8dfb3 Update the 1G drivers, shared code sync with Intel,
igb now has a queue notion that has a single interrupt
with an RX/TX pair, this will reduce the total interrupts
seen on a system. Both em and igb have a new watchdog
method. igb has fixes from Pyun Yong-Hyeon that have
improved stability, thank you :)

I wish to MFC this for 7.3 asap, please test if able.
2010-01-26 22:32:22 +00:00
Martin Blapp
c2ede4b379 Remove extraneous semicolons, no functional changes.
Submitted by:	Marc Balmer <marc@msys.ch>
MFC after:	1 week
2010-01-07 21:01:37 +00:00
John Baldwin
becf6b3db8 - Add missing newlines to some error messages.
- Add interrupt descriptions when using mulitple MSI-X interrupts.

Reviewed by:	jfv
2009-12-18 16:14:31 +00:00
Jack F Vogel
74dbc418c7 Add old read_mac_addr routine to this module since
a customer report of an Invalid MAC Address has occurred.
2009-12-14 19:05:16 +00:00
Jack F Vogel
92cc9dde8b Remove the MTX_SPIN flag to the shared code MUTEX
as it was causing a panic, also took the opportunity
to rename the lock for clarity.
2009-12-14 18:43:18 +00:00
Jack F Vogel
2b8dbb8253 Remove phantom line of code that somehow slipped
into the checkin.
2009-12-08 18:54:37 +00:00
Jack F Vogel
4edd8523d4 Resync with Intel versions of both the em and igb
drivers. These add new hardware support, most importantly
the pch (i5 chipset) in the em driver. Also, both drivers
now have the simplified (and I hope improved) watchdog
code. The igb driver uses the new RX cleanup that I
first implemented in ixgbe.

em  - version 6.9.24
igb - version 1.8.4
2009-12-08 01:07:44 +00:00
Jack F Vogel
466727ddbe With an i386 kernel the igb driver can cause a
page fault panic on initialization due to a large
number of bounce pages being allocated. This is due
to the dma tag requiring page alignment on mbuf mapping.
This was removed some time back from the ixgbe driver
and is not needed here either.
2009-11-11 19:13:40 +00:00
Jack F Vogel
7d21b9c117 Fix build issue with last commit. 2009-09-10 21:16:26 +00:00
Jack F Vogel
b53aa98e18 Fix build complaint from previous checkin 2009-09-10 21:14:55 +00:00
Jack F Vogel
7f9960edcb Fix an xmit mbuf leak, when transmit failed but you
still have an mbuf it was not being requeued.

MFC: 3 days
2009-09-10 18:56:45 +00:00
Jack F Vogel
f32888843f Fix for pr 138516
An mbuf is not requeued when a xmit fails.

MFC: 3 days
2009-09-10 18:53:34 +00:00
Poul-Henning Kamp
6778431478 Revert previous commit and add myself to the list of people who should
know better than to commit with a cat in the area.
2009-09-08 13:19:05 +00:00
Poul-Henning Kamp
b34421bf9c Add necessary include. 2009-09-08 13:16:55 +00:00
Xin LI
1886a6912d Temporarily enhance em(4) and igb(4) hack to take account for IFF_NOARP.
Without this changeset there will be no way to prevent these NICs from
sending ARP, which is harmful in server farms that is configured as
"Direct Server Return" behind a load balancer.

A better fix would remove the whole hack completely but it would be
later than 8.0-RELEASE.

Reviewed by:	jfv, yongari
Approved by:	re (kib)
2009-08-19 17:59:41 +00:00
Jack F Vogel
45289e2ded Improvement on the last change, this gives a precise
way to tell the one and only interface that a vlan
event is for. Thanks to John Baldwin for the patch.

Approved by: re
2009-07-24 21:35:52 +00:00
Jack F Vogel
387424df40 This delta fixes two bugs:
- When a vlan event occurs a check was not made that
    the event was actually for the interface, thus resulting
    in a panic. All three drivers have this vulnerability. Add
    a check for this condition.
  - Secondly, there was a duplicate buf_ring free in the em
    driver resulting in a panic on unload. Remove.

Approved by:  re
2009-07-24 16:57:49 +00:00