initialization. Prior to this change packets may be transmitted with an
incorrect checksum.
Em(4) already has an equivalent change in r213234.
Obtained From: Sandvine
MFC After: 1 week
Approved by: re (bz)
em drivers.
- Make the per-instance 'enable_aim' sysctl truly per-instance by having it
change a per-instance variable (which is used to control AIM) rather
than having all of the per-instance sysctls operate on a single global
variable.
Reviewed by: jfv (earlier version)
MFC after: 1 week
particular flow control and dma coalesce. Also improve the
sysctl operation on those too.
Add IPv6 detection in the ioctl code, this was done for
ixgbe first, carrying that over.
Add resource ability to disable particular adapter.
Add HW TSO capability so vlans can make use of TSO
method instead of reusing the existing per-queue interrupt task.
Reusing the per-queue interrupt task could result in both an interrupt
thread and the taskqueue thread trying to handle received packets on a
single queue resulting in out-of-order packet processing.
- Don't define igb_start() at all on 8.0 and where if_transmit is used.
Replace last remaining call to igb_start() with a loop to kick off
transmit on each queue instead.
- Call ether_ifdetach() earlier in igb_detach().
- Drain tasks and free taskqueues during igb_detach().
Reviewed by: jfv
MFC after: 1 week
this there is a rare return path that bogusly appears
to fail when it should not. Also white space correction.
Thanks to Arnaud Lacombe for noticing the problem.
- Add the change made in em to the actual unrefreshed number
of descriptors is used as a basis in rxeof on the way out
to determine if more refresh is needed. NOTE: there is a
difference in the ring setup in igb, this is not accidental,
it is necessitated by hardware behavior, when you reset the
newer adapters it will not let you write RDH, it ALWAYS sets
it to 0. Thus the way em does it is not possible.
- Change the sysctl handling of flow control, it will now make
the change dynamically when the variable setting changes rather
than requiring a reset.
- Change the eee sysctl naming, validation found the old unintuitive :)
- Last but not least, some important performance tweaks in the TX
path, I found that UDP behavior could be drastically hindered or
improved with just small changes in the start loop. What I have
here is what testing has shown to be the best overall. Its interesting
to note that changing the clean threshold to start at a full half of
the ring, made a BIG difference in performance. I hope that this
will prove to be advantageous for most workloads.
MFC in a week.
get's defragged due to a mapping failure the header
pointers will be invalidated and can result in a
TSO or other failure down the line. So, when the
remapping occurs force a retry thru the offload
calculation code. Thanks to Andrew Boyer for discovering
this and cooking up the fix!!
to calculate the outstanding descriptors that need to be
refreshed at any time, and use THAT in rxeof to determine
if refreshing needs to be done. Also change the local_timer
to simply fire off the appropriate interrupt rather than
schedule a tasklet, its simpler.
MFC in two weeks
been undergoing test for some weeks. This improves the RX
mbuf handling to avoid system hang due to depletion. Thanks
to all those who have been testing the code, and to Beezar
Liu for the design changes.
Next the igb driver is updated for similar RX changes, but
also to add new features support for our upcoming i350 family
of adapters.
MFC after a week
- this also includes virtualization support on these devices
Correct some vlan issues we were seeing in test, jumbo frames on vlans
did not work correctly, this was all due to confused logic around HW
filters, the new code should now work for all uses.
Important fix: when mbuf resources are depeleted, it was possible to
completely empty the RX ring, and then the RX engine would stall
forever. This is fixed by a flag being set whenever the refresh code
fails due to an mbuf shortage, also the local timer now makes sure
that all queues get an interrupt when it runs, the interrupt code
will then always call rxeof, and in that routine the first thing done
is now to check the refresh flag and call refresh_mbufs. This has been
verified to fix this type 'hang'. Similar code will follow in the other
drivers.
Finally, sync up shared code for the I350 support.
Thanks to everyone that has been reporting issues, and helping in the
debug/test process!!
- failure code in em_xmit got mangled along the way
and was not properly handling errors.
- local timer code had a leftover UNLOCK call that
should be removed.
MFC after 3 days
the size can be smaller than the constant when you are
doing HW TAGGING, and you still need to process this
packet in a normal way. I'm not sure where the notion
to just return came from, but its wrong.
MFC after: 3 days
Second, correct the discard/refresh_mbufs code to behave
more like igb, there have been panics due to discards and
this should fix them.
MFC after: 3 days
disable ASPM L0S and L1 LINK states on 82573, 82574,
and 82583. The theory is that this is behind certain
hangs being experienced by some customers.
Also included a small optimization in the rxeof routine
that was in my internal code.
Change the PBA size for pchlan, it was incorrect.
MFC after: 3 days
- Fixes from John Baldwin: vlan shadow tables made per/interface,
make vlan hw setup only happen when capability enabled, and
finally, make a tuneable interrupt rate. Thanks John!
- Tweaked watchdog handling to avoid any false positives, now
detection is in the TX clean path, with only the final check
and init happening in the local timer.
- limit queues to 8 for all devices, with 82576 or 82580 on
larger machines it can get greater than this, and it seems
mostly a resource waste to do so. Even 8 might be high but
it can be manually reduced.
- use 2k, 4k and now 9k clusters based on the MTU size.
- rework the igb_refresh_mbuf() code, its important to
make sure the descriptor is rewritten even when reusing
mbufs since writeback clobbers things.
MFC: in a few days, this delta needs to get to 8.2
ip and tcp pointers were not reset after some
pullups. In practice this led to an NFS mount
failure when using UDP reported by Kevin Lo,
thanks Kevin. Fix from yongari, thank you!
- Chasin down bogus watchdogs has led to an improved
design to this handling, the hang decision takes
place in the tx cleanup, with only a simple report
check in local_timer. Our tests have shown no false
watchdogs with this code.
- VLAN fixes from jhb, the shadow vfta should be per
interface, but as global it was not. Thanks John.
- Bug fixes in the support for new PCH2 hardware.
- Thanks for all the help and feedback on the driver,
changes to lem with be coming shortly as well.
- Sync shared code with Intel internal
- New client chipset support added
- em driver - fixes to 82574, limit queues to 1 but use MSIX
- em driver - large changes in TX checksum offload and tso
code, thanks to yongari.
- some small changes for watchdog issues.
- igb driver - local timer watchdog code was missing locking
this and a couple other watchdog related fixes.
- bug in rx discard found by Andrew Boyer, check for null pointer
MFC: a week
- Add a single sysctl procedure to all three drivers to read an arbitrary
register (the register is passed as arg2). Use it to replace existing
routines in igb(4) that used a separate routine for each register, and
to add support for missing stats in em(4) and lem(4).
- Move the 'rx_overruns' and 'watchdog_timeouts' stats out of the MAC stats
section as they are driver stats, not MAC counters.
- Simplify the code that creates per-queue stats in igb(4) to use a single
loop and remove duplicated code.
- Properly read all 64 bits of the 'good octets received/transmitted' in
em(4) and lem(4).
- Actually read the interrupt count registers in em(4), and drop the
'host to card' sysctl stats from em(4) as they are not implemented in
any of the hardware this driver supports.
- Restore several stats to em(4) that were lost in the earlier stats
conversion including per-queue stats.
- Export several MAC stats in em(4) that were exported in igb(4) but not
in em(4).
- Export stats in lem(4) using individual sysctls as in em(4) and igb(4).
Reviewed by: jfv
MFC after: 1 week
discrepencies from the igb version which was the target.
Change the message when neither MSI or MSIX are enabled
and a fallback to Legacy interrupts happen, the existing
message was confusing.
configuration function. For failed memory allocations, em(4)/lem(4)
called panic(9) which is not acceptable on production box.
igb(4)/ixgb(4)/ix(4) allocated the required memory in stack which
consumed 768 bytes of stack memory which looks too big.
To address these issues, allocate multicast array memory in device
attach time and make multicast configuration success under any
conditions. This change also removes the excessive use of memory in
stack.
Reviewed by: jfv
Just showing some buffer allocation error is more appropriate
action for drivers. This should fix occasional panic reported on
em(4) when driver encountered resource shortage.
Reviewed by: jfv
tail pointers of the tx and rx queues. We needed a SYSCTL_PROC
to correctly get the values at run time.
Submitted by: Andrew Boyer aboyer at averesystems.com
MFC after: 1 week
a packet has only a header in the first mbuf, the
checksum code will dereference a pointer into the
non-existing IP header. Do a check for the size and
pullup if needed. Thanks to Michael Tuexen for this
fix.
MFC: asap - should be in 8.1 IMHO
the core changes but left out the shared code, lol.
Well, and a couple fixes to the core... hopefully
this will all be complete now.
Happy happy joy joy :)
What this provides is support for the 'virtual function'
interface that a FreeBSD VM may be assigned from a host
like KVM on Linux, or newer versions of Xen with such
support.
When the guest is set up with the capability, a special
limited function 82576 PCI device is present in its virtual
PCI space, so with this driver installed in the guest that
device will be detected and function nearly like the bare
metal, as it were.
The interface is only allowed a single queue in this configuration
however initial performance tests have looked very good.
Enjoy!!
Remove unneeded rxtx handler, make que handler generic.
Do not allocate header mbufs in rx ring if not doing hdr split.
Release the lock in rxeof call to stack.
MFC for 8.1 asap
back in rxeof (I could see little point in taking it out),
and now release it before the stack entry.
Also, make it so the 82574 does not configure for multiqueue
when its not used in the stack.
unneeded for legacy hardware.
Also remove some TSO related cruft.
Add some watchdog_time setting that was missing, thanks
to Mikolaj Golub for pointing that out.
- code flow in handler could let interrupt be
reenabled when not wanted.
- change where the RX lock is taken to improve
performance.
- adapter->msix is true for MSI systems also,
it needs to explicitly test for 82574, good one :)
- add CRC stripping to the RX side, this was handled
by some obscure code in rxeof previously, its easier
to simply have the hardware strip it now.
- Add back an ALTQ change that slipped between the cracks
- Add an update to the watchdog_time in the xmit code, not
doing this in ixgbe caused problems, think its needed here
as well.
VLAN HWFILTER from being used by default, this breaks
stacked pseudo devices, and as it turns out, also breaks
virtual machines that happen to use VLANS (didn't know that
before :). Put the fix back into the em driver, and for good
measure add the same code to the igb driver where it should
have been anyway.
change the argument type to igb_rxeof() to the
correct type. Note, any users of POLLING must
be sure and set the number of queues to 1 for
things to work correctly.
em revision 7.0.0:
- Using driver devclass, seperate legacy (pre-pcie) code
into a seperate source file. This will at least help
protect against regression issues. It compiles along
with em, and is transparent to end use, devices in each
appear to be 'emX'. When using em in a modular form this
also allows the legacy stuff to be defined out.
- Add tx and rx rings as in igb, in the 82574 this becomes
actual multiqueue for the first time (2 queues) while in
other PCIE adapters its just make code cleaner.
- Add RX mbuf handling logic that matches igb, this will
eliminate packet drops due to temporary mbuf shortage.
igb revision 1.9.3:
- Following the ixgbe code, use a new approach in what
was called 'get_buf', the routine now has been made
independent of rxeof, it now does the update to the
engine TDT register, this design allows temporary
mbuf resources to become non-critical, not requiring
a packet to be discarded, instead it just returns and
does not increment the tail pointer.
- With the above change it was also unnecessary to keep
'spare' maps around, since we do not have the discard
issue.
- Performance tweaks and improvements to the code also.
MFC in a week
- introduce drbr_needs_enqueue that returns whether the interface/br needs
an enqueue operation: returns true if altq is enabled or there are
already packets in the ring (as we need to maintain packet order)
- update all drbr consumers
- fix drbr_flush
- avoid using the driver queue (IFQ_DRV_*) in the altq case as the
multiqueue consumer does not provide enough protection, serialize altq
interaction with the main queue lock
- make drbr_dequeue_cond work with altq
Discussed with: kmacy, yongari, jfv
MFC after: 4 weeks
lagg and vlan the vlan attach/detach event is not being
handed down to em, this caused some init code not to run,
and thus VLANs did not work. Ultimately having the event
get propagated would be nice, but for now the solution is
to have HWFILTER off by default, when this is the case
VLANs will work, ifconfig can be used to turn it on and
then get HW tag filtering.
igb now has a queue notion that has a single interrupt
with an RX/TX pair, this will reduce the total interrupts
seen on a system. Both em and igb have a new watchdog
method. igb has fixes from Pyun Yong-Hyeon that have
improved stability, thank you :)
I wish to MFC this for 7.3 asap, please test if able.
drivers. These add new hardware support, most importantly
the pch (i5 chipset) in the em driver. Also, both drivers
now have the simplified (and I hope improved) watchdog
code. The igb driver uses the new RX cleanup that I
first implemented in ixgbe.
em - version 6.9.24
igb - version 1.8.4
page fault panic on initialization due to a large
number of bounce pages being allocated. This is due
to the dma tag requiring page alignment on mbuf mapping.
This was removed some time back from the ixgbe driver
and is not needed here either.
Without this changeset there will be no way to prevent these NICs from
sending ARP, which is harmful in server farms that is configured as
"Direct Server Return" behind a load balancer.
A better fix would remove the whole hack completely but it would be
later than 8.0-RELEASE.
Reviewed by: jfv, yongari
Approved by: re (kib)
- When a vlan event occurs a check was not made that
the event was actually for the interface, thus resulting
in a panic. All three drivers have this vulnerability. Add
a check for this condition.
- Secondly, there was a duplicate buf_ring free in the em
driver resulting in a panic on unload. Remove.
Approved by: re
part that is made up of 8K banks rather than 4K, if these
systems are using bank 1 then the last change in this code
breaks the bank read, resulting in an invalid checksum of
the eeprom during driver load. This change fixes this.
Approved by: re
result was when the RX index wrapped it was converted into some
sort of gibberish and written into the RDT register, effectively
killing the RX side of the thing :)
Approved by: re
IF_ADDR_UNLOCK() across network device drivers when accessing the
per-interface multicast address list, if_multiaddrs. This will
allow us to change the locking strategy without affecting our driver
programming interface or binary interface.
For two wireless drivers, remove unnecessary locking, since they
don't actually access the multicast address list.
Approved by: re (kib)
MFC after: 6 weeks
CPU for too long period than necessary. Additively, interfaces are kept
polled (in the tick) even if no more packets are available.
In order to avoid such situations a new generic mechanism can be
implemented in proactive way, keeping track of the time spent on any
packet and fragmenting the time for any tick, stopping the processing
as soon as possible.
In order to implement such mechanism, the polling handler needs to
change, returning the number of packets processed.
While the intended logic is not part of this patch, the polling KPI is
broken by this commit, adding an int return value and the new flag
IFCAP_POLLING_NOCOUNT (which will signal that the return value is
meaningless for the installed handler and checking should be skipped).
Bump __FreeBSD_version in order to signal such situation.
Reviewed by: emaste
Sponsored by: Sandvine Incorporated
will sometimes fail to initialize problem due to a lock
contention with management hardware. However, in order to
deliver that fix it was necessary to take a shared code
update as a whole, and this required scattered changes in
the core code to be compatible.
The em driver now has VLAN HW support added as the igb
driver had previously.
MFC after: ASAP - in time for 7.1 RELEASE
- changes in support of the VLAN filter fix to 126850
- removal of a bunch of legacy code that was cruft, if not
possibly harmful.
- removal of POLLING from this driver, with multiqueue and
MSIX it just makes no sense here.
- Fix an LRO bug that I've been working on internally, intermittent
panics under stress, the problem was releasing the RX ring lock
before the LRO flushing.
- Following the above fix I now enable LRO by default
- For performance reasons increase the default number of RX queues
to 4.
- Add AIM - "Adaptive Interrupt Moderation", a fancy way of saying
that the EITR value is dynamically changed based on the size of
packets in the last interrupt interval.
- Much goodness to try, enjoy!!
proved to be necessary to make the static drivers work
in EITHER/OR or BOTH configurations. Modules will still
build in sys/modules/igb or em as before.
This also updates the igb driver for support for the 82576
adapter, adds shared code fixes, and etc....
MFC after: ASAP
