Commit Graph

135 Commits

Author SHA1 Message Date
Andrew Rybchenko
29033246a8 sfxge: skip VPD info population if access is denied
The patch allows to run on unprivileged PF (PFIOV) passed to
a virtual machine.

Reviewed by:    gnn
Sponsored by:   Solarflare Communications, Inc.
MFC after:      2 days
Differential Revision: https://reviews.freebsd.org/D2891
2015-06-24 06:25:20 +00:00
Andrew Rybchenko
c378187825 sfxge: do not use DEBUG_FLAGS to enable extra debug checks
DEBUG_FLAGS are set to DEBUG option value when kernel is built.
For example, it is -g in GENERIC config to have debug symbols.
Also DEBUG_FLAGS are used to determine if ctfconvert should keep
debug symbols.
Since we redefined DEBUG_FLAGS, debug symbols were always missing.
ctfconvert complains about it during kernel build.
It is incorrect to append DEBUG_FLAGS, since if DEBUG has no -g (or
similar), we'll have no debug symbols and ctfconvert will complain.
If it incorrect to always have -g in our DEBUG_FLAGS, since debug
symbols presence should be controllable by kernel config.
So, just add disabled by default addition of -DDEBUG=1 to CFLAGS.

Reviewed by:    imp
Sponsored by:   Solarflare Communications, Inc.
MFC after:      2 days
Differential Revision: https://reviews.freebsd.org/D2666
2015-05-29 05:44:56 +00:00
Andrew Rybchenko
11d893928e sfxge: mask ifmedia options (pauses) when looking for matching mode
Sponsored by:   Solarflare Communications, Inc.
MFC after:      2 days
Differential Revision: https://reviews.freebsd.org/D2656
2015-05-27 08:57:48 +00:00
Andrew Rybchenko
3c838a9f51 sfxge: add 7xxx NICs family support
Support 7xxx adapters including firmware-assisted TSO and VLAN tagging:

  - Solarflare Flareon Ultra 7000 series 10/40G adapters:
    - Solarflare SFN7042Q QSFP+ Server Adapter
    - Solarflare SFN7142Q QSFP+ Server Adapter

  - Solarflare Flareon Ultra 7000 series 10G adapters:
    - Solarflare SFN7022F SFP+ Server Adapter
    - Solarflare SFN7122F SFP+ Server Adapter
    - Solarflare SFN7322F Precision Time Synchronization Server Adapter

  - Solarflare Flareon 7000 series 10G adapters:
    - Solarflare SFN7002F SFP+ Server Adapter

Support utilities to configure adapters and update firmware.

The work is done by Solarflare developers
(Andy Moreton, Andrew Lee and many others),
Artem V. Andreev <Artem.Andreev at oktetlabs.ru> and me.

Sponsored by:   Solarflare Communications, Inc.
MFC after:      2 weeks
Causually read by: gnn
Differential Revision: https://reviews.freebsd.org/D2618
2015-05-25 08:34:55 +00:00
Jung-uk Kim
fd90e2ed54 CALLOUT_MPSAFE has lost its meaning since r141428, i.e., for more than ten
years for head.  However, it is continuously misused as the mpsafe argument
for callout_init(9).  Deprecate the flag and clean up callout_init() calls
to make them more consistent.

Differential Revision:	https://reviews.freebsd.org/D2613
Reviewed by:	jhb
MFC after:	2 weeks
2015-05-22 17:05:21 +00:00
Andrew Rybchenko
1217b24e51 sfxge: relax assertion to allow RST flag in TSO packets
Kernel under stress load, mixed MC reboot and sfupdate really
generates TSO packet with RST flag.
It will generate many TCP packets with RST flag set.
May be RST flag should be set in the last segment only, but it could be
dropped. So, it is safer to keep the flag in all packets to be sure that
connection is reset.

Reviewed by:    gnn
Sponsored by:   Solarflare Communications, Inc.
MFC after:      2 days
Differential Revision: https://reviews.freebsd.org/D2609
2015-05-22 07:39:21 +00:00
Andrew Rybchenko
8a343f93e6 sfxge: do not advertise LRO capability if LRO is compiled out
Reviewed by:    gnn
Sponsored by:   Solarflare Communications, Inc.
MFC after:      2 days
Differential Revision: https://reviews.freebsd.org/D2565
2015-05-18 06:07:02 +00:00
Andrew Rybchenko
921f3bd2df sfxge: automatically turn off TSO when Tx checksum offload is disabled
Also return error if TSO is requested without Tx checksum offload.

Reviewed by:    gnn
Sponsored by:   Solarflare Communications, Inc.
MFC after:      2 days
Differential Revision: https://reviews.freebsd.org/D2564
2015-05-18 06:04:20 +00:00
Andrew Rybchenko
4cefd52431 sfxge: allow to disable checksum offloads over VLAN
It just affects capabilities of the created VLAN interface.

Sponsored by:   Solarflare Communications, Inc.
MFC after:      2 days
Differential Revision: https://reviews.freebsd.org/D2563
2015-05-18 06:03:21 +00:00
Andrew Rybchenko
f080384c8f sfxge: fix overflow queue freeze
If TxQ lock is obtained, deferred packet list shold be serviced even if
the packet addition fails because of overflow.

Without the patch freeze happens if:
 - queue is not blocked (i.e. completion does not trigger unblock and service)
 - put-list overflow (1024 entries)
 - sfxge_tx_packet_add() acquires TxQ lock just as it is released it in
   sfxge_tx_qdpl_service() on the second CPU but before pending check
 - sfxge_tx_packet_add() swizzles put-list to get-list, fails because of
   non-tcp get-list overflow and returns without packet list service
 - sfxge_tx_qdpl_service() on the second CPU checks that there are no
   pending packets in the put-list and returns

Other possible solution is to guaranee that maximum length of the put-list
is less than maximum length of any get-list.

Reviewed by:    gnn
Sponsored by:   Solarflare Communications, Inc.
MFC after:      2 days
Differential Revision: https://reviews.freebsd.org/D2562
2015-05-18 06:02:22 +00:00
Andrew Rybchenko
c379e930e7 sfxge: avoid usage of ifm_data
The driver uses ifm_data to save capabilities mask calculated during
initialization when supported phy modes are discovered.
The patch simply calculates it when either media or options are changed.

Reviewed by:    glebius
Sponsored by:   Solarflare Communications, Inc.
MFC after:      2 days
Differential Revision: https://reviews.freebsd.org/D2540
2015-05-16 10:35:30 +00:00
Andrew Rybchenko
ef97c04bc5 sfxge: add local variable with changed capabilities mask
It is required for the next patch which adds dependency of TSO
capabilities from Tx checksum offloads.

Reviewed by:    gnn
Sponsored by:   Solarflare Communications, Inc.
MFC after:      2 days
Differential Revision: https://reviews.freebsd.org/D2553
2015-05-16 06:04:53 +00:00
Andrew Rybchenko
0e4ebe6c4d sfxge: move mbuf free to sfxge_if_transmit()
It is a preparation to the next patch which will service packet queue even
if packet addtion fails.

Reviewed by:    gnn
Sponsored by:   Solarflare Communications, Inc.
MFC after:      2 days
Differential Revision: https://reviews.freebsd.org/D2552
2015-05-16 05:43:20 +00:00
Andrew Rybchenko
deee1de4bf sfxge: get rid of locked variable in sfxge_tx_packet_add()
Now each branch has one and only one possible TxQ lock state.
It simplifies understanding of the code.

Reviewed by:    gnn
Sponsored by:   Solarflare Communications, Inc.
MFC after:      2 days
Differential Revision: https://reviews.freebsd.org/D2551
2015-05-16 05:37:47 +00:00
Andrew Rybchenko
b5bae9f4f2 sfxge: support Rx checksum offloads disabling
We can't disable it in HW, but we can ignore result.
Discard Rx descriptor checksum flags if Rx checksum offload is off.

Reviewed by:    gnn
Sponsored by:   Solarflare Communications, Inc.
MFC after:      2 days
Differential Revision: https://reviews.freebsd.org/D2544
2015-05-16 05:36:40 +00:00
Andrew Rybchenko
d6e9f7362a sfxge: split sfxge_tx_qdpl_put() into *_locked() and *_unlocked()
It simplifies understanding of the sfxge_tx_packet_add() logic and
avoids passing of 'locked' to called function.

Reviewed by:    gnn
Sponsored by:   Solarflare Communications, Inc.
MFC after:      2 days
Differential Revision: https://reviews.freebsd.org/D2547
2015-05-15 06:50:59 +00:00
Andrew Rybchenko
22bc53cb48 sfxge: do not change CSUM_TSO when IFCAP_TSOx is changed
It is simply not required since the kernel checks corresponding
IFCAP_TSOx capability and CSUM_TSO in hw-assisted offloads.
Note that CSUM_TSO is two bits (CSUM_IP_TSO|CSUM_IP6_TSO) and both bits
are set in IPv4 and IPv6 mbufs.

Reviewed by:    gnn
Sponsored by:   Solarflare Communications, Inc.
MFC after:      2 days
Differential Revision: https://reviews.freebsd.org/D2546
2015-05-15 06:49:43 +00:00
Andrew Rybchenko
3b3390c115 sfxge: LRO may be done only if checksums are OK
Also it is cheaper to check Rx descriptor flags than TCP protocol in IP
header.

Reviewed by:    gnn
Sponsored by:   Solarflare Communications, Inc.
MFC after:      2 days
Differential Revision: https://reviews.freebsd.org/D2542
2015-05-15 06:48:36 +00:00
Andrew Rybchenko
db7e53aa3f sfxge: advertise IPv6 Rx and Tx checksum offload support
Tx checksum offload may be enabled/disabled.

Sponsored by:   Solarflare Communications, Inc.
MFC after:      2 days
Differential Revision: https://reviews.freebsd.org/D2543
2015-05-14 14:16:09 +00:00
Andrew Rybchenko
b2b5a4c060 sfxge: IPv4 Tx checksum offload may be disabled in fact
Split IFCAP_HWCSUM to IFCAP_RXCSUM and IFCAP_TXCSUM to highlight Tx and Rx.

Sponsored by:   Solarflare Communications, Inc.
MFC after:      2 days
Differential Revision: https://reviews.freebsd.org/D2541
2015-05-14 13:49:00 +00:00
Andrew Rybchenko
588644a4b6 sfxge: add local variable with Rx descriptor flags
Sponsored by:   Solarflare Communications, Inc.
MFC after:      2 days
2015-05-14 13:28:29 +00:00
Andrew Rybchenko
ab2310e8bf sfxge: add missing const qualifier to sfxge_link_mode
Sponsored by:   Solarflare Communications, Inc.
MFC after:      2 days
2015-05-14 12:59:17 +00:00
Andrew Rybchenko
cfaf34ff99 sfxge: fix bug in TSO when a DMA segment has both header and data
Reviewed by:    gnn
Sponsored by:   Solarflare Communications, Inc.
MFC after:      2 days
Differential Revision: https://reviews.freebsd.org/D2163
2015-03-29 15:18:04 +00:00
Andrew Rybchenko
dcf085860a sfxge: set correct RSS hash type instead of opaque
Reviewed by:    adrian, gnn
Sponsored by:   Solarflare Communications, Inc.
Differential Revision: https://reviews.freebsd.org/D2124
2015-03-28 10:20:20 +00:00
Andrew Rybchenko
cb552e880d sfxge: cleanup: fix index variable type to match upper boundary type
Sponsored by:   Solarflare Communications, Inc.
2015-03-24 11:33:34 +00:00
Andrew Rybchenko
919c8da4c5 sfxge: cleanup: add a blank line before each #if to improve readability
Sponsored by:   Solarflare Communications, Inc.
2015-03-24 11:25:19 +00:00
Andrew Rybchenko
cc5a55a29d sfxge: remove unnecessary and wrong prediction
Sponsored by:   Solarflare Communications, Inc.
Differential Revision: https://reviews.freebsd.org/D2085
2015-03-23 15:53:26 +00:00
Andrew Rybchenko
63424713e5 sfxge: do not check MCDI status word
This is a temporary workaround until we determine a reliable sequence
of operations for detecting MC reboots.

Sponsored by:   Solarflare Communications, Inc.
Differential Revision: https://reviews.freebsd.org/D2084
2015-03-23 15:52:57 +00:00
Andrew Rybchenko
5dadbb64d0 sfxge: FreeBSD before 10 does not have bus_space_*_8 on amd64
bus_space_*_8() are not always macros, so it is not correct to use
#ifndef.

Sponsored by:   Solarflare Communications, Inc.
Differential Revision: https://reviews.freebsd.org/D2083
2015-03-23 15:52:05 +00:00
Andrew Rybchenko
f6222d7bd0 sfxge: add statistics for each Tx queue
Sponsored by:   Solarflare Communications, Inc.
Differential Revision: https://reviews.freebsd.org/D2082
2015-03-23 15:49:11 +00:00
Andrew Rybchenko
a411fe4e80 sfxge: remove obsolete Tx non-multi queue support
Tx multi queue is added in FreeBSD 8.0. So, the changeset drops earlier
versions support.

Sponsored by:   Solarflare Communications, Inc.
Differential Revision: https://reviews.freebsd.org/D2081
2015-03-23 15:47:37 +00:00
Andrew Rybchenko
1cc8febf4c sfxge: add barriers to BAR write macros
In theory the barriers are required to cope with write combining and
reordering. Two barriers are added (sometimes merged to one):
 1. Before the first write to guarantee that previous writes to the region
    have been done
 2. Before the last write to guarantee that write to the last dword/qword is
    done after previous writes
Barriers are inserted before in the assumption that it is better to
postpone barriers as much as it is possible (more chances that the
operation has already been already done and barrier does not stall CPU).

On x86 and amd64 bus space write barriers are just compiler memory barriers
which are definitely required.

Sponsored by:   Solarflare Communications, Inc.
Differential Revision: https://reviews.freebsd.org/D2077
2015-03-23 15:45:39 +00:00
Andrew Rybchenko
87581ab817 sfxge: assert either kernel or internal copy of interface flags
ioctl to put interface down sets ifp->if_flags which holds the intended
administratively defined state and calls driver callback to apply it.
When everything is done, driver updates internal copy of
interface flags sc->if_flags which holds the operational state.
So, transmit from Rx path is possible when interface is intended to be
administratively down in accordance with ifp->if_flags, but not applied
yet and the operational state is up in accordance with sc->if_flags.

Sponsored by:   Solarflare Communications, Inc.
Differential Revision: https://reviews.freebsd.org/D2075
2015-03-23 15:44:17 +00:00
Andrew Rybchenko
22ff426cc2 sfxge: increase default put-list limit to 1024
Drops are observed under multi-stream TCP traffic due to put-list
overflow with limit equal to 64.

Sponsored by:   Solarflare Communications, Inc.
Approved by:    gnn (mentor)
2015-03-17 08:23:46 +00:00
Andrew Rybchenko
06824d2c88 sfxge: prefetch txq->common if TxQ is started only
Transmit may be called when TxQ is not started yet (i.e. txq->common is
invalid). TxQ state is checked below when mbuf is processed and dropped
if TxQ is not started.

Sponsored by:   Solarflare Communications, Inc.
Approved by:    gnn (mentor)
2015-03-17 08:23:15 +00:00
Andrew Rybchenko
ebfb5f2546 sfxge: adding version info to device description
The information is required for NIC update and config tools.

Submitted by:   Artem V. Andreev <Artem.Andreev at oktetlabs.ru>
Sponsored by:   Solarflare Communications, Inc.
Approved by:    gnn (mentor)
2015-03-17 08:21:31 +00:00
Andrew Rybchenko
95caaf0f54 sfxge: move deferred packet list statistics to dedicated node
It is done to structure sysctl and do not mix with Tx queue statistics
to be added.

Sponsored by:   Solarflare Communications, Inc.
Approved by:    gnn (mentor)
2015-03-17 08:16:37 +00:00
Andrew Rybchenko
245d157651 sfxge: add tunables to control LRO parameters on driver load time
Sponsored by:   Solarflare Communications, Inc.
Approved by:    gnn (mentor)
2015-03-17 08:14:46 +00:00
Enji Cooper
c14bdeb07e Unbreak 'make depend' with sfxge by removing debugging code activated in the
INET || INET6 case

X-MFC with: r279398
Pointyhat to: arybchik
2015-02-28 21:50:04 +00:00
Andrew Rybchenko
18daa0ee9a sfxge: compile out LRO if kernel is compiled without IPv4 and IPv6
Sponsored by:   Solarflare Communications, Inc.
Approved by:    gnn (mentor)
2015-02-28 19:01:43 +00:00
Andrew Rybchenko
851128b8de sfxge: expect required init_state on data path and in periodic calls
With the patch applied the number of instruction events is 1% less and
number of mispredicted branch events is 5% less under multistream TCP
traffic load close to line rate.

Sponsored by:   Solarflare Communications, Inc.
Approved by:    gnn (mentor)
2015-02-27 07:39:09 +00:00
Andrew Rybchenko
f86f4cd2ae sfxge: use tab to indent instead of spaces
Sponsored by:   Solarflare Communications, Inc.
Approved by:    gnn (mentor)
2015-02-25 06:19:23 +00:00
Andrew Rybchenko
75ba9e1e51 sfxge: correct limit for number of Rx queues
Sponsored by:   Solarflare Communications, Inc.
Approved by:    gnn (mentor)
2015-02-25 06:19:00 +00:00
Andrew Rybchenko
bce6d281d8 sfxge: add put-list high watermark
It is interesting to know how long put-list grows.

Sponsored by:   Solarflare Communications, Inc.
Approved by:    gnn (mentor)
2015-02-24 06:09:31 +00:00
Andrew Rybchenko
08664e0d60 sfxge: use goto to cleanup to avoid duplicate cleanup code
Sponsored by:   Solarflare Communications, Inc.
Approved by:    gnn (mentor)
2015-02-24 06:08:55 +00:00
Andrew Rybchenko
3d8fce2701 sfxge: implement if_get_counter callback
Sponsored by:   Solarflare Communications, Inc.
Approved by:    gnn (mentor)
2015-02-22 19:25:57 +00:00
Andrew Rybchenko
2fc13da641 sfxge: add common code support for changing TX queue pace
To delay packets from a particular TX queue by a particular time, write a value
into the TX Pace table s.t. pace time <= TX Pace Clock Period * (2 ^ pace value)
- the TX pace clock is 1/13 of the system clock, so its period should be 104 or
52 ns depending on whether turbo mode is active.

EFX_TX_PACE_CLOCK_BASE added by me.

Submitted by:   Mark Spender <mspender at solarflare.com>
Sponsored by:   Solarflare Communications, Inc.
Approved by:    gnn (mentor)
2015-02-22 19:24:08 +00:00
Andrew Rybchenko
af9078c3f1 sfxge: correct event queue interrupt moderation timer quanta
Submitted by:   Andrew Lee <alee at solarflare.com>
Sponsored by:   Solarflare Communications, Inc.
Approved by:    gnn (mentor)
2015-02-22 19:13:52 +00:00
Andrew Rybchenko
72ca6d78f1 sfxge: DMA allocated memory is set to zeros because of BUS_DMA_ZERO flag
It is not required to set it to zeros once again.

Sponsored by:   Solarflare Communications, Inc.
Approved by:    gnn (mentor)
2015-02-22 18:56:03 +00:00
Andrew Rybchenko
83488e1827 sfxge: do no allow EFSYS_MEM_ALLOC sleep
It solves locking problem when EFSYS_MEM_ALLOC is called in
the context holding a mutex (not allowed to sleep).
E.g. on interface bring up or multicast addresses addition.

Sponsored by:   Solarflare Communications, Inc.
Approved by:    gnn (mentor)
2015-02-22 18:54:16 +00:00