Commit Graph

60 Commits

Author SHA1 Message Date
Andrew Rybchenko
414dec5f0f sfxge(4): host byte order is required for IP ID in TSO descriptors
Submitted by:   Artem V. Andreev <Artem.Andreev at oktetlabs.ru>
Sponsored by:   Solarflare Communications, Inc.
MFC after:      1 week
2016-06-08 13:14:52 +00:00
Andrew Rybchenko
3bbc1e08ed sfxge(4): pick an RSS bucket for the packet enqueued and select TXQ accordingly
Submitted by:   Ivan Malov <Ivan.Malov at oktetlabs.ru>
Reviewed by:    gnn
Sponsored by:   Solarflare Communications, Inc.
Differential Revision:  https://reviews.freebsd.org/D6723
2016-06-06 09:08:16 +00:00
Andrew Rybchenko
929c7feb83 sfxge(4): cleanup: update copyright to 2016
Sponsored by:   Solarflare Communications, Inc.
MFC after:      1 week
Differential Revision:  https://reviews.freebsd.org/D6509
2016-05-24 12:16:57 +00:00
Pedro F. Giffuni
453130d9bf sys/dev: minor spelling fixes.
Most affect comments, very few have user-visible effects.
2016-05-03 03:41:25 +00:00
Pedro F. Giffuni
057b4402bf sys/dev: extend use of the howmany() macro when available.
We have a howmany() macro in the <sys/param.h> header that is
convenient to re-use as it makes things easier to read.
2016-04-26 15:03:15 +00:00
Gleb Smirnoff
8ec07310fa These files were getting sys/malloc.h and vm/uma.h with header pollution
via sys/mbuf.h
2016-02-01 17:41:21 +00:00
Andrew Rybchenko
a45a0da19c sfxge: support FATSOv2
Reviewed by:    gnn
Sponsored by:   Solarflare Communications, Inc.
MFC after:      2 days
Differential Revision: https://reviews.freebsd.org/D4934
2016-01-15 06:25:26 +00:00
Andrew Rybchenko
9dd0e15fab sfxge: switch to TxQ creation specific flags
It is better do not mix TxQ creation and receive event flags since only
checksum flags are applicable to TxQ.
Also it will allow to add a new TxQ creation specific flags.

Reviewed by:    gnn
Sponsored by:   Solarflare Communications, Inc.
MFC after: 2    days
Differential Revision: https://reviews.freebsd.org/D4389
2015-12-07 06:04:24 +00:00
Andrew Rybchenko
a32efb9731 sfxge: parse packets for TSO early in if_transmit
Submitted by:   Artem V. Andreev <Artem.Andreev at oktetlabs.ru>
Sponsored by:   Solarflare Communications, Inc.
MFC after:      1 week
Differential Revision: https://reviews.freebsd.org/D4309
2015-12-01 14:55:24 +00:00
Andrew Rybchenko
3c838a9f51 sfxge: add 7xxx NICs family support
Support 7xxx adapters including firmware-assisted TSO and VLAN tagging:

  - Solarflare Flareon Ultra 7000 series 10/40G adapters:
    - Solarflare SFN7042Q QSFP+ Server Adapter
    - Solarflare SFN7142Q QSFP+ Server Adapter

  - Solarflare Flareon Ultra 7000 series 10G adapters:
    - Solarflare SFN7022F SFP+ Server Adapter
    - Solarflare SFN7122F SFP+ Server Adapter
    - Solarflare SFN7322F Precision Time Synchronization Server Adapter

  - Solarflare Flareon 7000 series 10G adapters:
    - Solarflare SFN7002F SFP+ Server Adapter

Support utilities to configure adapters and update firmware.

The work is done by Solarflare developers
(Andy Moreton, Andrew Lee and many others),
Artem V. Andreev <Artem.Andreev at oktetlabs.ru> and me.

Sponsored by:   Solarflare Communications, Inc.
MFC after:      2 weeks
Causually read by: gnn
Differential Revision: https://reviews.freebsd.org/D2618
2015-05-25 08:34:55 +00:00
Andrew Rybchenko
1217b24e51 sfxge: relax assertion to allow RST flag in TSO packets
Kernel under stress load, mixed MC reboot and sfupdate really
generates TSO packet with RST flag.
It will generate many TCP packets with RST flag set.
May be RST flag should be set in the last segment only, but it could be
dropped. So, it is safer to keep the flag in all packets to be sure that
connection is reset.

Reviewed by:    gnn
Sponsored by:   Solarflare Communications, Inc.
MFC after:      2 days
Differential Revision: https://reviews.freebsd.org/D2609
2015-05-22 07:39:21 +00:00
Andrew Rybchenko
f080384c8f sfxge: fix overflow queue freeze
If TxQ lock is obtained, deferred packet list shold be serviced even if
the packet addition fails because of overflow.

Without the patch freeze happens if:
 - queue is not blocked (i.e. completion does not trigger unblock and service)
 - put-list overflow (1024 entries)
 - sfxge_tx_packet_add() acquires TxQ lock just as it is released it in
   sfxge_tx_qdpl_service() on the second CPU but before pending check
 - sfxge_tx_packet_add() swizzles put-list to get-list, fails because of
   non-tcp get-list overflow and returns without packet list service
 - sfxge_tx_qdpl_service() on the second CPU checks that there are no
   pending packets in the put-list and returns

Other possible solution is to guaranee that maximum length of the put-list
is less than maximum length of any get-list.

Reviewed by:    gnn
Sponsored by:   Solarflare Communications, Inc.
MFC after:      2 days
Differential Revision: https://reviews.freebsd.org/D2562
2015-05-18 06:02:22 +00:00
Andrew Rybchenko
0e4ebe6c4d sfxge: move mbuf free to sfxge_if_transmit()
It is a preparation to the next patch which will service packet queue even
if packet addtion fails.

Reviewed by:    gnn
Sponsored by:   Solarflare Communications, Inc.
MFC after:      2 days
Differential Revision: https://reviews.freebsd.org/D2552
2015-05-16 05:43:20 +00:00
Andrew Rybchenko
deee1de4bf sfxge: get rid of locked variable in sfxge_tx_packet_add()
Now each branch has one and only one possible TxQ lock state.
It simplifies understanding of the code.

Reviewed by:    gnn
Sponsored by:   Solarflare Communications, Inc.
MFC after:      2 days
Differential Revision: https://reviews.freebsd.org/D2551
2015-05-16 05:37:47 +00:00
Andrew Rybchenko
d6e9f7362a sfxge: split sfxge_tx_qdpl_put() into *_locked() and *_unlocked()
It simplifies understanding of the sfxge_tx_packet_add() logic and
avoids passing of 'locked' to called function.

Reviewed by:    gnn
Sponsored by:   Solarflare Communications, Inc.
MFC after:      2 days
Differential Revision: https://reviews.freebsd.org/D2547
2015-05-15 06:50:59 +00:00
Andrew Rybchenko
db7e53aa3f sfxge: advertise IPv6 Rx and Tx checksum offload support
Tx checksum offload may be enabled/disabled.

Sponsored by:   Solarflare Communications, Inc.
MFC after:      2 days
Differential Revision: https://reviews.freebsd.org/D2543
2015-05-14 14:16:09 +00:00
Andrew Rybchenko
cfaf34ff99 sfxge: fix bug in TSO when a DMA segment has both header and data
Reviewed by:    gnn
Sponsored by:   Solarflare Communications, Inc.
MFC after:      2 days
Differential Revision: https://reviews.freebsd.org/D2163
2015-03-29 15:18:04 +00:00
Andrew Rybchenko
cb552e880d sfxge: cleanup: fix index variable type to match upper boundary type
Sponsored by:   Solarflare Communications, Inc.
2015-03-24 11:33:34 +00:00
Andrew Rybchenko
f6222d7bd0 sfxge: add statistics for each Tx queue
Sponsored by:   Solarflare Communications, Inc.
Differential Revision: https://reviews.freebsd.org/D2082
2015-03-23 15:49:11 +00:00
Andrew Rybchenko
a411fe4e80 sfxge: remove obsolete Tx non-multi queue support
Tx multi queue is added in FreeBSD 8.0. So, the changeset drops earlier
versions support.

Sponsored by:   Solarflare Communications, Inc.
Differential Revision: https://reviews.freebsd.org/D2081
2015-03-23 15:47:37 +00:00
Andrew Rybchenko
87581ab817 sfxge: assert either kernel or internal copy of interface flags
ioctl to put interface down sets ifp->if_flags which holds the intended
administratively defined state and calls driver callback to apply it.
When everything is done, driver updates internal copy of
interface flags sc->if_flags which holds the operational state.
So, transmit from Rx path is possible when interface is intended to be
administratively down in accordance with ifp->if_flags, but not applied
yet and the operational state is up in accordance with sc->if_flags.

Sponsored by:   Solarflare Communications, Inc.
Differential Revision: https://reviews.freebsd.org/D2075
2015-03-23 15:44:17 +00:00
Andrew Rybchenko
06824d2c88 sfxge: prefetch txq->common if TxQ is started only
Transmit may be called when TxQ is not started yet (i.e. txq->common is
invalid). TxQ state is checked below when mbuf is processed and dropped
if TxQ is not started.

Sponsored by:   Solarflare Communications, Inc.
Approved by:    gnn (mentor)
2015-03-17 08:23:15 +00:00
Andrew Rybchenko
95caaf0f54 sfxge: move deferred packet list statistics to dedicated node
It is done to structure sysctl and do not mix with Tx queue statistics
to be added.

Sponsored by:   Solarflare Communications, Inc.
Approved by:    gnn (mentor)
2015-03-17 08:16:37 +00:00
Andrew Rybchenko
851128b8de sfxge: expect required init_state on data path and in periodic calls
With the patch applied the number of instruction events is 1% less and
number of mispredicted branch events is 5% less under multistream TCP
traffic load close to line rate.

Sponsored by:   Solarflare Communications, Inc.
Approved by:    gnn (mentor)
2015-02-27 07:39:09 +00:00
Andrew Rybchenko
bce6d281d8 sfxge: add put-list high watermark
It is interesting to know how long put-list grows.

Sponsored by:   Solarflare Communications, Inc.
Approved by:    gnn (mentor)
2015-02-24 06:09:31 +00:00
Andrew Rybchenko
3d8fce2701 sfxge: implement if_get_counter callback
Sponsored by:   Solarflare Communications, Inc.
Approved by:    gnn (mentor)
2015-02-22 19:25:57 +00:00
Andrew Rybchenko
72ca6d78f1 sfxge: DMA allocated memory is set to zeros because of BUS_DMA_ZERO flag
It is not required to set it to zeros once again.

Sponsored by:   Solarflare Communications, Inc.
Approved by:    gnn (mentor)
2015-02-22 18:56:03 +00:00
Andrew Rybchenko
4944888783 sfxge: TxQ block level should use EFX_TXQ_LIMIT as maximum TxQ size
Sponsored by:   Solarflare Communications, Inc.
Approved by:    gnn (mentor)
2015-02-22 07:20:04 +00:00
Andrew Rybchenko
4d0b294b00 sfxge: packet can't require more than n_dma_seg descriptors plus one
Use remaining number of DMA segment instead of maximum number in mapping
when checking space for one more TSO segment packet.

Sponsored by:   Solarflare Communications, Inc.
Approved by:    gnn (mentor)
2015-02-22 07:16:11 +00:00
Andrew Rybchenko
c071447ac1 sfxge: style fixes and cleanup
Sync endif comment with conditional.
BOOTROM and SIENA_BOOTROM are the same, but highlight that it is Siena.
Restore commented out assertion.
Sync comments with out-of-tree driver.

Sponsored by:   Solarflare Communications, Inc.
Approved by:    gnn (mentor)
2015-02-22 07:08:57 +00:00
Andrew Rybchenko
45923e7690 sfxge: more accurate calculation of maximum number of TSO segments
Sponsored by:   Solarflare Communications, Inc.
Approved by:    gnn (mentor)
2015-02-21 06:24:48 +00:00
Andrew Rybchenko
6d73545e5d sfxge: reap Tx descriptors on unblock
Otherwise when processing finally comes to efx_tx_qdesc_post() it could
be insufficient space between reaped and added to post pending
descriptors.

Sponsored by:   Solarflare Communications, Inc.
Approved by:    gnn (mentor)
2015-02-20 19:44:02 +00:00
Andrew Rybchenko
612d8e2828 sfxge: style fixes
Use nitem() to get number of array elements.
Remove unused define.
Use TAB to indent.

Sponsored by:   Solarflare Communications, Inc.
Approved by:    gnn (mentor)
2015-02-20 19:39:40 +00:00
Andrew Rybchenko
b2c43c38e7 sfxge: handle fragmented TCP header in mbuf
TCP header is fragmented in the case of VLAN tagged IPv6 traffic without
HW VLAN tagging.

Sponsored by:   Solarflare Communications, Inc.
Approved by:    gnn (mentor)
2015-02-20 07:53:46 +00:00
Andrew Rybchenko
e2b05fe21c sfxge: add driver context member with number of transmit queues
Sponsored by:   Solarflare Communications, Inc.
Approved by:    gnn (mentor)
2015-02-18 06:19:35 +00:00
Andrew Rybchenko
d0f7387777 sfxge: add TCP segment size to sfxge_tso_state
It avoids access to m_pkthdr when TSO packet is started and also makes
tso_start_new_packet() function smaller.

Sponsored by:   Solarflare Communications, Inc.
Approved by:    gnn (mentor)
2015-02-18 06:18:51 +00:00
Andrew Rybchenko
2cd617a883 sfxge: flush all Tx queues from if_qflush
Sponsored by:   Solarflare Communications, Inc.
Approved by:    gnn (mentor), glebius
2015-02-16 06:16:38 +00:00
Andrew Rybchenko
0b28bbdca9 sfxge: remove inline specifiers
Now compiler does not need any help.
The patch does not change generated code.

Sponsored by:   Solarflare Communications, Inc.
Approved by:    gnn (mentor), glebius
2015-02-16 06:08:13 +00:00
Andrew Rybchenko
fd59fc3478 sfxge: remove used sfxge_tso_state member dma_seg_i
Sponsored by:   Solarflare Communications, Inc.
Approved by:    gnn (mentor)
2015-02-16 06:07:01 +00:00
Andrew Rybchenko
3f1e3499ed sfxge: remove full_packet_size from sfxge_tso_state
It makes sfxge_tso_state smaller and even makes tso_start_new_packet()
few bytes smaller. Data used to calculate packet size are used nearby,
so it should be no problems with cache etc.

Sponsored by:   Solarflare Communications, Inc.
Approved by:    gnn (mentor), glebius
2015-02-16 06:04:26 +00:00
Andrew Rybchenko
c9795115dc sfxge: remove unused variable
Sponsored by:   Solarflare Communications, Inc.
Approved by:    gnn (mentor), glebius
2015-02-16 06:00:43 +00:00
Andrew Rybchenko
e1a3d10e9e sfxge: Add statistics for partially dropped TSO packets
Sponsored by:   Solarflare Communications, Inc.
Approved by:    gnn (mentor)
2015-02-05 12:10:23 +00:00
Andrew Rybchenko
33d45dc5e4 sfxge: make lock names unique
Lock name should include interface name.
Tx queue and event queue lock name should include queue number.

Sponsored by:   Solarflare Communications, Inc.
Approved by:    gnn (mentor)
2015-02-05 11:39:15 +00:00
Andrew Rybchenko
763cab7173 sfxge: Add macros to init, destroy, acquire, release and assert locks
Sponsored by:   Solarflare Communications, Inc.
Approved by:    gnn (mentor)
2015-02-04 20:03:57 +00:00
Andrew Rybchenko
93929f253d sfxge: Separate software Tx queue limit for non-TCP traffic
Add separate software Tx queue limit for non-TCP traffic to make total
limit higher and avoid local drops of TCP packets because of no
backpressure.
There is no point to make non-TCP limit high since without backpressure
UDP stream easily overflows any sensible limit.

Split early drops statistics since it is better to have separate counter
for each drop reason to make it unabmiguous.

Add software Tx queue high watermark. The information is very useful to
understand how big queues grow under traffic load.

Sponsored by:   Solarflare Communications, Inc.
Approved by:    gnn (mentor)
2015-01-29 19:11:37 +00:00
Andrew Rybchenko
cc933626d4 sfxge: Add evq argument to sfxge_tx_qcomplete()
It removes necessity to get evq pointer by its index in soft context.

Sponsored by:   Solarflare Communications, Inc.
Approved by:    gnn (mentor)
2015-01-29 19:01:09 +00:00
Andrew Rybchenko
a35485aa72 sfxge: fixed TSO code to cope with VLAN headers
Submitted by:   Artem V. Andreev <Artem.Andreev at oktetlabs.ru>
Sponsored by:   Solarflare Communications, Inc.
Approved by:    gnn (mentor)
2015-01-29 18:59:22 +00:00
Hans Petter Selasky
c25290420e Start process of removing the use of the deprecated "M_FLOWID" flag
from the FreeBSD network code. The flag is still kept around in the
"sys/mbuf.h" header file, but does no longer have any users. Instead
the "m_pkthdr.rsstype" field in the mbuf structure is now used to
decide the meaning of the "m_pkthdr.flowid" field. To modify the
"m_pkthdr.rsstype" field please use the existing "M_HASHTYPE_XXX"
macros as defined in the "sys/mbuf.h" header file.

This patch introduces new behaviour in the transmit direction.
Previously network drivers checked if "M_FLOWID" was set in "m_flags"
before using the "m_pkthdr.flowid" field. This check has now now been
replaced by checking if "M_HASHTYPE_GET(m)" is different from
"M_HASHTYPE_NONE". In the future more hashtypes will be added, for
example hashtypes for hardware dedicated flows.

"M_HASHTYPE_OPAQUE" indicates that the "m_pkthdr.flowid" value is
valid and has no particular type. This change removes the need for an
"if" statement in TCP transmit code checking for the presence of a
valid flowid value. The "if" statement mentioned above is now a direct
variable assignment which is then later checked by the respective
network drivers like before.

Additional notes:
- The SCTP code changes will be committed as a separate patch.
- Removal of the "M_FLOWID" flag will also be done separately.
- The FreeBSD version has been bumped.

MFC after:	1 month
Sponsored by:	Mellanox Technologies
2014-12-01 11:45:24 +00:00
George V. Neville-Neil
060a95ef01 Support tunable to control Tx deferred packet list limits
Also increase default for Tx queue get-list limit.
Too small limit results in TCP packets drops especiall when many
streams are running simultaneously.
Put list may be kept small enough since it is just a temporary
location if transmit function can't get Tx queue lock.

Submitted by:   Andrew Rybchenko <arybchenko at solarflare.com>
Sponsored by:   Solarflare Communications, Inc.
2014-09-30 20:57:25 +00:00
George V. Neville-Neil
bc85c897c7 The patch allows to check state of the software Tx queues at run time.
Submitted by:   Andrew Rybchenko <arybchenko at solarflare.com>
Sponsored by:   Solarflare Communications, Inc.
2014-09-30 20:43:21 +00:00