capability. One of reason using interrupt taskqueue in re(4) was
to reduce number of TX/RX interrupts under load because re(4)
controllers have no good TX/RX interrupt moderation mechanism.
Basic TX interrupt moderation is done by hardware for most
controllers but RX interrupt moderation through undocumented
register showed poor RX performance so it was disabled in r215025.
Using taskqueue to handle RX interrupt greatly reduced number of
interrupts but re(4) consumed all available CPU cycles to run the
taskqueue under high TX/RX network load. This can happen even with
RTL810x fast ethernet controller and I believe this is not
acceptable for most systems.
To mitigate the issue, use one-shot timer register to moderate RX
interrupts. The timer register provides programmable one-shot timer
and can be used to suppress interrupt generation. The timer runs at
125MHZ on PCIe controllers so the minimum time allowed for the
timer is 8ns. Data sheet says the register is 32 bits but
experimentation shows only lower 13 bits are valid so maximum time
that can be programmed is 65.528us. This yields theoretical maximum
number of RX interrupts that could be generated per second is about
15260. Combined with TX completion interrupts re(4) shall generate
less than 20k interrupts. This number is still slightly high
compared to other intelligent ethernet controllers but system is
very responsive even under high network load.
Introduce sysctl variable dev.re.%d.int_rx_mod that controls amount
of time to delay RX interrupt processing in units of us. Value 0
completely disables RX interrupt moderation. To provide old
behavior for controllers that have MSI/MSI-X capability, introduce
a new tunable hw.re.intr_filter. If the tunable is set to non-zero
value, driver will use interrupt taskqueue. The default value of
the tunable is 0. This tunable has no effect on controllers that
has no MSI/MSI-X capability or if MSI/MSI-X is explicitly disabled
by administrator.
While I'm here cleanup interrupt setup/teardown since re(4) uses
single MSI/MSI-X message at this moment.
recent PCIe controllers(RTL8102E or later and RTL8168/8111C or
later) supports either 2 or 4 MSI-X messages. Unfortunately vendor
did not publicly release RSS related information yet. However
switching to MSI-X is one-step forward to support RSS.
RTL8111C generated corrupted frames where TCP option header was
broken. All other sample controllers I have did not show such
problem so it could be RTL8111C specific issue. Because there are
too many variants it's hard to tell how many controllers have such
issue. Just disable TSO by default but have user override it.
controllers. Experimentation with RTL8102E, RTL8103E and RTL8105E
showed dramatic decrement of TX completion interrupts under high TX
load(e.g. from 147k interrupts/second to 10k interrupts/second)
With this change, TX interrupt moderation is applied to all
controllers except RTL8139C+.
GbE controllers. It seems these controllers no longer support
multi-fragmented RX buffers such that driver have to allocate
physically contiguous buffers.
o Retire RL_FLAG_NOJUMBO flag and introduce RL_FLAG_JUMBOV2 to
mark controllers that use new jumbo frame scheme.
o Configure PCIe max read request size to 4096 for standard frames
and reduce it to 512 for jumbo frames.
o TSO/checksum offloading is not supported for jumbo frames on
these controllers. Reflect it to ioctl handler and driver
initialization.
o Remove unused rl_stats_no_timeout in softc.
o Embed a pointer to structure rl_hwrev into softc to keep track
of controller MTU limitation and remove rl_hwrev in softc since
that information is available through a pointer to structure
rl_hwrev.
Special thanks to Realtek for donating sample hardwares which made
this possible.
H/W donated by: Realtek Semiconductor Corp.
limit maximum RX buffer size to RE_RX_DESC_BUFLEN instead of
blindly configuring it to 16KB. Due to lack of documentation, re(4)
didn't allow jumbo frame on these controllers. However it seems
controller is confused with jumbo frame such that it can DMA the
received frame to wrong address instead of splitting it into
multiple RX buffers. Of course, this caused panic.
Since re(4) does not support jumbo frames on these controllers,
make controller drop frame that is longer than RE_RX_DESC_BUFLEN
sized frame. Fortunately RTL810x controllers, which do not support
jumbo frame, have no such issues but this change also limited
maximum RX buffer size allowed to RTL810x controllers. Allowing
16KB RX buffer for controllers that have no such capability is
meaningless.
MFC after: 3 days
and just show old (cached) values. Controller will not respond to
the command unless MAC is enabled so DUMP request for down
interface caused request timeout.
RealTek changed TX descriptor format for later controllers so these
controllers require MSS configuration in different location of TX
descriptor. TSO is enabled by default for controllers that use new
descriptor format.
For old controllers, TSO is still disabled by default due to broken
frames under certain conditions but users can enable it.
Special thanks to Hayes Wang at RealTek.
MFC after: 2 weeks
not provide any MAC configuration interface for resolved flow
control parameters. There is even no register that configures water
mark which will control generation of pause frames.
However enabling flow control surely enhanced performance a lot.
It seems RTL8169/RTL8168/RTL810xE has a kind of interrupt
moderation mechanism but it is not documented at all. The magic
value dramatically reduced number of interrupts without noticeable
performance drops so apply it to all RTL8169/RTL8169 controllers.
Vendor's FreeBSD driver also applies it to RTL810xE controllers but
their Linux driver explicitly cleared the register, so do not
enable interrupt moderation for RTL810xE controllers.
While I'm here sort 8169 specific registers.
Obtained from: RealTek FreeBSD driver
There were a couple of attempts in the past to reduce it since it
took more than 1ms. Because mii_tick() periodically polls link
status, waiting more than 1ms for each GMII register access was
overkill. Unfortunately all previous attempts were failed with
various ways on different controllers.
This time, add additional 20us dealy at the end of GMII register
access which seems to requirement of all RealTek controllers to
issue next GMII register access request. This is the same way what
Linux does.
useful counters like rl_missed_pkts is 16 bits quantity which is
too small to hold meaningful information happened in a second. This
means driver should frequently read these counters in order not to
lose accuracy and that approach is too inefficient in driver's
view. Moreover it seems there is no way to trigger an interrupt to
detect counter near-full or wraparound event as well as lacking
clearing the MAC counters. Another limitation of reading the
counters from RealTek controllers is lack of interrupt firing at
the end of DMA cycle of MAC counter read request such that driver
have to poll the end of the DMA which is a time consuming process
as well as inefficient. The more severe issue of the MAC counter
read request is it takes too long to complete the DMA. All these
limitation made maintaining MAC counters in driver impractical. For
now, just provide simple sysctl interface to trigger reading the
MAC counters. These counters could be used to track down driver
issues. Users can read MAC counters maintained in controller with
the following command.
#sysctl dev.re.0.stats=1
While I'm here add check for validity of dma map and allocated
memory before unloading/freeing them.
Tested by: rmacklem
the NIC drivers as well as the PHY drivers to take advantage of the
mii_attach() introduced in r213878 to get rid of certain hacks. For
the most part these were:
- Artificially limiting miibus_{read,write}reg methods to certain PHY
addresses; we now let mii_attach() only probe the PHY at the desired
address(es) instead.
- PHY drivers setting MIIF_* flags based on the NIC driver they hang
off from, partly even based on grabbing and using the softc of the
parent; we now pass these flags down from the NIC to the PHY drivers
via mii_attach(). This got us rid of all such hacks except those of
brgphy() in combination with bce(4) and bge(4), which is way beyond
what can be expressed with simple flags.
While at it, I took the opportunity to change the NIC drivers to pass
up the error returned by mii_attach() (previously by mii_phy_probe())
and unify the error message used in this case where and as appropriate
as mii_attach() actually can fail for a number of reasons, not just
because of no PHY(s) being present at the expected address(es).
Reviewed by: jhb, yongari
frame, make sure to update VLAN capabilities whenever jumbo frame
is configured.
While I'm here rearrange interface capabilities configuration. The
controller requires VLAN hardware tagging to make TSO work on VLANs
so explicitly check this requirement.
Tx DMA burst size 2048, I beleive PCIe maximum read request size
also should match to the value of Tx DMA burst size. With this
change I can get more than 800Mbps for TCP bulk transfers.
Previously I was not able to get more than 700Mbps. If I enable TSO
it now shows 927Mbps.
reacquiring driver lock in Rx handler. re(4) drops a driver lock
before passing received frame to upper stack and reacquire the
lock. During the time window ioctl calls could be executed and if
the ioctl was interface down request, driver will stop the
controller and free allocated mbufs. After that when driver comes
back to Rx handler again it does not know what was happend so it
could access free mbufs which in turn cause panic.
Reported by: Norbert Papke < npapk <> acm dot org >
Tested by: Norbert Papke < npapk <> acm dot org >
IF_ADDR_UNLOCK() across network device drivers when accessing the
per-interface multicast address list, if_multiaddrs. This will
allow us to change the locking strategy without affecting our driver
programming interface or binary interface.
For two wireless drivers, remove unnecessary locking, since they
don't actually access the multicast address list.
Approved by: re (kib)
MFC after: 6 weeks
CPU for too long period than necessary. Additively, interfaces are kept
polled (in the tick) even if no more packets are available.
In order to avoid such situations a new generic mechanism can be
implemented in proactive way, keeping track of the time spent on any
packet and fragmenting the time for any tick, stopping the processing
as soon as possible.
In order to implement such mechanism, the polling handler needs to
change, returning the number of packets processed.
While the intended logic is not part of this patch, the polling KPI is
broken by this commit, adding an int return value and the new flag
IFCAP_POLLING_NOCOUNT (which will signal that the return value is
meaningless for the installed handler and checking should be skipped).
Bump __FreeBSD_version in order to signal such situation.
Reviewed by: emaste
Sponsored by: Sandvine Incorporated
checksum offload frames. Software workaround used for broken
controllers(RTL8169, RTL8168, RTL8168B) seem to cause watchdog
timeouts on RTL8139C+.
Introduce a new flag RL_FLAG_AUTOPAD to mark automatic padding
feature of controller and set it for RTL8139C+ and controllers that
use new descriptor format. This fixes watchdog timeouts seen on
RTL8139C+.
Reported by: Dimitri Rodis < DimitriR <> integritasystems dot com >
Tested by: Dimitri Rodis < DimitriR <> integritasystems dot com >
It seems that RTL8168D and RTL8102EL requires additional settle
time to complete RL_PHYAR register write. Accessing RL_PHYAR
register right after the write causes errors for subsequent PHY
register accesses.
Tested by: george at luckytele dot com,
Steve Wills < STEVE at stevenwills dot com >
mapping. The tunable is OFF for all controllers except RTL8169SC
family. RTL8169SC seems to require more magic to use memory
register mapping. r187483 added a fix for RTL8169SCe controller but
it does not looke like fix other variants of RTL8169SC.
Tested by: Gavin Stone-Tolcher g.stone-tolcher <> its dot uq dot edu dot au
controllers that lose Tx completion interrupts under certain
conditions. With this change it's safe to use MSI on PCIe
controllers so enable MSI on these controllers.
- Always program RX configuration register from scratch instead of
doing read/modify/write.
- Rename re_setmulti() to re_set_rxmode() to be reflect reality.
- Simplify hash filter logic a little while I am here.
Reviewed by: yongari (early version)
event from mii(4) may not be delivered if valid link was already
established. To address the issue, check current link state after
driving MII_TICK. This should fix a regression introduced in
r185753 on fast ethernet controllers.
Reported by: csjp, Bruce Cran < bruce <> cran DOT org DOT uk >
Tested by: csjp, Bruce Cran (initial version)
command whenever Tx completion interrupt is raised. The Tx poll
bit is cleared when all packets waiting to be transferred have been
processed. This means the second Tx poll command can be silently
ignored as the Tx poll bit could be still active while processing
of previous Tx poll command is in progress.
To address the issue re(4) used to invoke the Tx poll command in Tx
completion handler whenever it detects there are pending packets in
TxQ. However that still does not seem to completely eliminate
watchdog timeouts seen on RealTek PCIe controllers. To fix the
issue kick Tx poll command only after Tx completion interrupt is
raised as this would indicate Tx is now idle state such that it can
accept new Tx poll command again. While here apply this workaround
for PCIe based controllers as other controllers does not seem to
have this limitation.
Tested by: Victor Balada Diaz < victor <> bsdes DOT net >
out of sleep mode prior to accessing to PHY. This should fix device
attach failure seen on these controllers. Also enable the sleep
mode when device is put into sleep state.
PR: kern/123123, kern/123053
Waiting for 1ms for each GMII register access looks overkill and it
may also decrease overall performance of driver because re(4)
invokes mii_tick for every hz.
Tested by: rpaulo
established a valid link or not. In miibus_statchg handler add a
check for established link is valid one for the controller(e.g.
1000baseT is not a valid link for fastethernet controllers.)
o Added a flag RE_FLAG_FASTETHER to mark fastethernet controllers.
o Added additional check to know whether we've really encountered
watchdog timeouts or missed Tx completion interrupts. This change
may help to track down the cause of watchdog timeouts.
o In interrupt handler, removed a check for link state change
interrupt. Not all controllers have the bit and re(4) did not
rely on the event for a long time. In addition, re(4) didn't
request the interrupt in RL_IMR register.
Tested by: rpaulo
reading from EEPROM doesn't seem to work on these controllers.
Reported by: Milan Obuch ( freebsd-net at dino dot sk )
Tested by: Milan Obuch ( freebsd-net at dino dot sk )
watchdog timeout issues and the root cause seems to stem from
silicon bug of controller. Personally I couldn't reproduce it on
RTL8169 controller but it seems it's dependent on usage pattern.
For newer PCIe based controllers I have no TSO complaints but
turning off TSO would be more safe. Users who are sure that
their controller works with TSO can still reenable the TSO with
ifconfig(8).
Reported by: Oliver Lehmann (lehmann at ans-netz dot de), Eugene Butusov (ebutusov at gmail dot com)
11bits. This limits the maximum interface MTU size in TSO case
as upper stack should not generate TCP segments with MSS greater
than the limit. Armed with this information, disable TSO if
interface MTU is greater than the limit.
generation of RTL810x PCIe fast ethernet controller. Note, Tx/Rx
descriptor format is different from that of first generation of
RTL8101E series. Jumbo frame is not supported for RTL810x
family.
Tested by: NAGATA Shinya ( maya AT negeta DOT com )
ATM Tx/Rx checksum offload is supported but TSO and jumbo frame is
not yet supported. Because these newer controllers use different
descriptor formats, a flag RL_FLAG_DESCV2 flag was introduced to
handle that case in Tx/Rx handler. Also newer controllers seems to
require to not touch 'enable Tx/Rx bit' in RL_CPLUS_CMD register
so don't blindly try to set that bits.
Note, it seems that there is still power-saving related issue where
driver fails to attach PHY. Rebooting seems to fix that issue but
number of required reboots varys.
Many thanks to users that helped during developement. I really
appreciate their patient and test/feedbacks.
a dedicated flag that represents controller capabilities/events.
This will simplify many part of code that requires different
workaround for each controller revisions and will enhance
readability.
While I'm here move PHY wakeup code up before mii_phy_probe() which
seems to help to wake PHY in some cases.
RL_TXCFG register to identify a device in device probe. Reflect the
fact by modifing device description with general ethernet
controller family.
Note, rl_basetype in struct rl_type is not used and the more
detailed information is provided with rl_hwrev structure.
clearing MSI enable bit for MSI capable hardwares resulted in Tx
problems. MSI enable bit is set only when MSI is requested from
user.
Tested by: remko
offload bugs by manual padding for short IP/UDP frames. Unfortunately
it seems that these workaround does not work reliably on newer PCIe
variants of RealTek chips.
To workaround the hardware bug, always pad short frames if Tx IP
checksum offload is requested. It seems that the hardware has a
bug in IP checksum offload handling. NetBSD manually pads short
frames only when the length of IP frame is less than 28 bytes but I
chose 60 bytes to safety. Also unconditionally set IP checksum
offload bit in Tx descriptor if any TCP or UDP checksum offload is
requested. This is the same way as Linux does but it's not
mentioned in data sheet.
Obtained from: NetBSD
Tested by: remko, danger
multi-descriptor transmission attempt. Datasheet said nothing about
this requirements. This should fix a long-standing VLAN hardware
tagging issues with re(4).
Reported by: Giulio Ferro ( auryn AT zirakzigil DOT org )
Tested by: Giulio Ferro ( auryn AT zirakzigil DOT org )
o Increased number of Rx/Tx descriptors to 256 for 8169 GigEs
because it's hard to push the hardware to the limit with default
64 descriptors.
TSO requires large number of Tx descriptors to pass a full sized
TCP segment(65535 bytes IP packet) to hardware. Previously it
consumed 32 Tx descriptors, assuming MCLBYTES DMA segment size,
to send the TCP segment which means re(4) couldn't queue more
than two full sized IP packets.
For 8139C+ it still uses 64 Rx/Tx descriptors due to its hardware
limitations. With this changes there are (very) small waste of
memory for 8139C+ users but I don't think it would affect 8139C+
users for most cases.
o Various bus_dma(9) fixes.
- The hardware supports DAC so allow 64bit DMA operations.
- Removed BUS_DMA_ALLOC_NOW flag.
- Increased DMA segment size to 4096 from MCLBYTES because TSO
consumes too many descriptors with MCLBYTES DMA segment size.
- Tx/Rx side bus_dmamap_load_mbuf_sg(9) support. With these
changes the code is more readable than previous one and got a
(slightly) better performance as it doesn't need to pass/
decode arguments to/from callback function.
- Removed unnecessary callback function re_dmamap_desc() and
nuked rl_dmaload_arg structure which was used in the callback.
- Additional protection for DMA map load failure. In case of
failure reuse current map instead of returning a bogus DMA
map.
- Deferred DMA map unloading/sync operation for maximum
performance until we really need to load new DMA map. If we
happen to reuse current map(e.g. input error) there is no need
to sync/unload/load again.
- The number of allowable Tx DMA segments for a mbuf chains are
now 32 instead of magic nseg value. If the number of available
Tx descriptors are short enough to send highly fragmented mbuf
chains an optimized re_defrag() is called to collapse mbuf
chains which is supposed to be much faster than m_defrag(9).
re_defrag() was borrowed from ath(4).
- Separated Rx/Tx DMA tag from a common DMA tag such that Rx DMA
tag correctly uses DMA maps that were created with DMA alignment
restriction(8bytes alignments). Tx DMA tag does not have such
alignment limitation.
- Added additional sanity checks for DMA ring map load failure.
- Added additional spare Rx DMA map for graceful handling of Rx
DMA map load failure.
- Fixed misused bus_dmamap_sync(9) and added missing
bus_dmamap_sync(9) in re_encap()/re_txeof()/re_rxeof().
o Enabled TSO again as re(4) have reasonable number of Tx
descriptors.
o Don't touch DMA address of a Tx descriptor in re_txeof(). It's
not needed.
o Fix incorrect update of if_ierrors counter. For Rx buffer
shortage it should update if_qdrops as the buffer is reused.
o Added checks for unsupported H/W revisions and return ENXIO for
these hardwares. This is required to remove resource allocation
code in re_probe as other drivers do in device probe routine.
o Modified descriptor index manipulation macros as it's now possible
to have different number of descriptors for Rx/Tx.
o In re_start, to save a lock operation, use IFQ_DRV_IS_EMPTY before
trying to invoke IFQ_DRV_DEQUEUE. Also don't blindly call re_encap
since we already know the number of available Tx descriptors in
advance.
o Removed RL_TX_DESC_THLD which was used to reserve RL_TX_DESC_THLD
descriptors in Tx path. There is no such a limitation mentioned in
8139C+/8169/8110/8168/8101/8111 datasheet and it seems to work ok
without reserving RL_TX_DESC_THLD descriptors.
o Fix a comment for RL_GTXSTART. The register is 8bits register.
o Added comments for 8169/8139C+ hardware restrictions on descriptors.
o Removed forward declaration for "struct rl_softc", it's not needed.
o Added a new structure rl_txdesc for Tx descriptor managements and
a structure rl_rxdesc for Rx descriptor managements.
o Removed unused member variable rl_intlock in driver softc. There are
still several unused member variables which are supposed to be used
to access hardware statistics counters. But it seems that accessing
hardware counters were not implemented yet.
as multicast/broadcast frames. Previously re(4) ignored multicast
frames in promiscuous mode. The RTL8169 datasheet was not clear
how it handles multicast frames in promiscuous mode.
PR: kern/118572
MFC after: 3 days
Ethernet Controller. Multicast filtering wasn't tested and needs more
expore. While I'm here change complex if statements with switch
statement which would improve readability.
Reported by: Abdullah Ibn Hamad Al-Marri < wearabnet AT yahoo DOT ca >
Tested by: Abdullah Ibn Hamad Al-Marri < wearabnet AT yahoo DOT ca >
Without this the PHY wouldn't work as expected. This should fix
dual-boot Windows XP machine where RealTek Windows drivers put the
PHY in power down mode during shutdown. The magic PHY register
accesses come from RealTek driver. No datasheets mention the magic
PHY registers.
In general, the PHY wakeup code should go into PHY driver. However it
seems that it only apply to RTL8169S single chip and it would be
another hack if we have rgephy(4) check what parent driver/chip model
is attached.
Reported by: lofi, Laurens Timmermans ( laurens AT timkapel DOT nl )
Tested by: lofi
Obtained from: RealTek FreeBSD driver
Approved by: re (Ken Smith)
to clear RL_TDESC_VLANCTL_TAG). This fixes sending packets in the
native VLAN when running both tagged and an untagged VLAN over the
same trunk and descriptors are recycled.
Approved by: re (kensmith)
MFC after: 1 week
