Commit Graph

115 Commits

Author SHA1 Message Date
Sepherosa Ziehau
eacb70ba70 mxge: Setup mbuf flowid before calling tcp_lro_rx().
Reviewed by:	gallatin
MFC after:	1 week
Sponsored by:	Microsoft OSTC
Differential Revision:	https://reviews.freebsd.org/D6320
2016-05-12 03:36:49 +00:00
Pedro F. Giffuni
73a1170a8c sys/dev: use our nitems() macro when it is avaliable through param.h.
No functional change, only trivial cases are done in this sweep,
Drivers that can get further enhancements will be done independently.

Discussed in:	freebsd-current
2016-04-19 23:37:24 +00:00
Sepherosa Ziehau
6dd38b8716 tcp/lro: Use tcp_lro_flush_all in device drivers to avoid code duplication
And factor out tcp_lro_rx_done, which deduplicates the same logic with
netinet/tcp_lro.c

Reviewed by:	gallatin (1st version), hps, zbb, np, Dexuan Cui <decui microsoft com>
Sponsored by:	Microsoft OSTC
Differential Revision:	https://reviews.freebsd.org/D5725
2016-04-01 06:28:33 +00:00
Justin Hibbits
da1b038af9 Use uintmax_t (typedef'd to rman_res_t type) for rman ranges.
On some architectures, u_long isn't large enough for resource definitions.
Particularly, powerpc and arm allow 36-bit (or larger) physical addresses, but
type `long' is only 32-bit.  This extends rman's resources to uintmax_t.  With
this change, any resource can feasibly be placed anywhere in physical memory
(within the constraints of the driver).

Why uintmax_t and not something machine dependent, or uint64_t?  Though it's
possible for uintmax_t to grow, it's highly unlikely it will become 128-bit on
32-bit architectures.  64-bit architectures should have plenty of RAM to absorb
the increase on resource sizes if and when this occurs, and the number of
resources on memory-constrained systems should be sufficiently small as to not
pose a drastic overhead.  That being said, uintmax_t was chosen for source
clarity.  If it's specified as uint64_t, all printf()-like calls would either
need casts to uintmax_t, or be littered with PRI*64 macros.  Casts to uintmax_t
aren't horrible, but it would also bake into the API for
resource_list_print_type() either a hidden assumption that entries get cast to
uintmax_t for printing, or these calls would need the PRI*64 macros.  Since
source code is meant to be read more often than written, I chose the clearest
path of simply using uintmax_t.

Tested on a PowerPC p5020-based board, which places all device resources in
0xfxxxxxxxx, and has 8GB RAM.
Regression tested on qemu-system-i386
Regression tested on qemu-system-mips (malta profile)

Tested PAE and devinfo on virtualbox (live CD)

Special thanks to bz for his testing on ARM.

Reviewed By: bz, jhb (previous)
Relnotes:	Yes
Sponsored by:	Alex Perez/Inertial Computing
Differential Revision: https://reviews.freebsd.org/D4544
2016-03-18 01:28:41 +00:00
Justin Hibbits
43cd61606b Replace several bus_alloc_resource() calls using default arguments with bus_alloc_resource_any()
Since these calls only use default arguments, bus_alloc_resource_any() is the
right call.

Differential Revision: https://reviews.freebsd.org/D5306
2016-02-19 03:37:56 +00:00
Hans Petter Selasky
e936121d31 Add optimizing LRO wrapper:
- Add optimizing LRO wrapper which pre-sorts all incoming packets
  according to the hash type and flowid. This prevents exhaustion of
  the LRO entries due to too many connections at the same time.
  Testing using a larger number of higher bandwidth TCP connections
  showed that the incoming ACK packet aggregation rate increased from
  ~1.3:1 to almost 3:1. Another test showed that for a number of TCP
  connections greater than 16 per hardware receive ring, where 8 TCP
  connections was the LRO active entry limit, there was a significant
  improvement in throughput due to being able to fully aggregate more
  than 8 TCP stream. For very few very high bandwidth TCP streams, the
  optimizing LRO wrapper will add CPU usage instead of reducing CPU
  usage. This is expected. Network drivers which want to use the
  optimizing LRO wrapper needs to call "tcp_lro_queue_mbuf()" instead
  of "tcp_lro_rx()" and "tcp_lro_flush_all()" instead of
  "tcp_lro_flush()". Further the LRO control structure must be
  initialized using "tcp_lro_init_args()" passing a non-zero number
  into the "lro_mbufs" argument.

- Make LRO statistics 64-bit. Previously 32-bit integers were used for
  statistics which can be prone to wrap-around. Fix this while at it
  and update all SYSCTL's which expose LRO statistics.

- Ensure all data is freed when destroying a LRO control structures,
  especially leftover LRO entries.

- Reduce number of memory allocations needed when setting up a LRO
  control structure by precomputing the total amount of memory needed.

- Add own memory allocation counter for LRO.

- Bump the FreeBSD version to force recompilation of all KLDs due to
  change of the LRO control structure size.

Sponsored by:	Mellanox Technologies
Reviewed by:	gallatin, sbruno, rrs, gnn, transport
Tested by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D4914
2016-01-19 15:33:28 +00:00
Craig Rodrigues
d9db52256e Move zlib.c from net to libkern.
It is not network-specific code and would
be better as part of libkern instead.
Move zlib.h and zutil.h from net/ to sys/
Update includes to use sys/zlib.h and sys/zutil.h instead of net/

Submitted by:		Steve Kiernan stevek@juniper.net
Obtained from:		Juniper Networks, Inc.
GitHub Pull Request:	https://github.com/freebsd/freebsd/pull/28
Relnotes:		yes
2015-04-22 14:38:58 +00:00
Hans Petter Selasky
c25290420e Start process of removing the use of the deprecated "M_FLOWID" flag
from the FreeBSD network code. The flag is still kept around in the
"sys/mbuf.h" header file, but does no longer have any users. Instead
the "m_pkthdr.rsstype" field in the mbuf structure is now used to
decide the meaning of the "m_pkthdr.flowid" field. To modify the
"m_pkthdr.rsstype" field please use the existing "M_HASHTYPE_XXX"
macros as defined in the "sys/mbuf.h" header file.

This patch introduces new behaviour in the transmit direction.
Previously network drivers checked if "M_FLOWID" was set in "m_flags"
before using the "m_pkthdr.flowid" field. This check has now now been
replaced by checking if "M_HASHTYPE_GET(m)" is different from
"M_HASHTYPE_NONE". In the future more hashtypes will be added, for
example hashtypes for hardware dedicated flows.

"M_HASHTYPE_OPAQUE" indicates that the "m_pkthdr.flowid" value is
valid and has no particular type. This change removes the need for an
"if" statement in TCP transmit code checking for the presence of a
valid flowid value. The "if" statement mentioned above is now a direct
variable assignment which is then later checked by the respective
network drivers like before.

Additional notes:
- The SCTP code changes will be committed as a separate patch.
- Removal of the "M_FLOWID" flag will also be done separately.
- The FreeBSD version has been bumped.

MFC after:	1 month
Sponsored by:	Mellanox Technologies
2014-12-01 11:45:24 +00:00
Hans Petter Selasky
f0188618f2 Fix multiple incorrect SYSCTL arguments in the kernel:
- Wrong integer type was specified.

- Wrong or missing "access" specifier. The "access" specifier
sometimes included the SYSCTL type, which it should not, except for
procedural SYSCTL nodes.

- Logical OR where binary OR was expected.

- Properly assert the "access" argument passed to all SYSCTL macros,
using the CTASSERT macro. This applies to both static- and dynamically
created SYSCTLs.

- Properly assert the the data type for both static and dynamic
SYSCTLs. In the case of static SYSCTLs we only assert that the data
pointed to by the SYSCTL data pointer has the correct size, hence
there is no easy way to assert types in the C language outside a
C-function.

- Rewrote some code which doesn't pass a constant "access" specifier
when creating dynamic SYSCTL nodes, which is now a requirement.

- Updated "EXAMPLES" section in SYSCTL manual page.

MFC after:	3 days
Sponsored by:	Mellanox Technologies
2014-10-21 07:31:21 +00:00
Gleb Smirnoff
56ec302f3c Whitespace cleanup. 2014-09-25 05:47:33 +00:00
Gleb Smirnoff
f3f040d929 - Provide mxge_get_counter() to return counters that are not collected,
but taken from hardware.
- Mechanically convert to if_inc_counter() the rest of counters.
2014-09-25 05:45:52 +00:00
Gleb Smirnoff
56b61ca27a Remove ifq_drops from struct ifqueue. Now queue drops are accounted in
struct ifnet if_oqdrops.

Some netgraph modules used ifqueue w/o ifnet. Accounting of queue drops
is simply removed from them. There were no API to read this statistic.

Sponsored by:	Netflix
Sponsored by:	Nginx, Inc.
2014-09-19 09:01:19 +00:00
Gleb Smirnoff
b245f96c44 Since 32-bit if_baudrate isn't enough to describe a baud rate of a 10 Gbit
interface, in the r241616 a crutch was provided. It didn't work well, and
finally we decided that it is time to break ABI and simply make if_baudrate
a 64-bit value. Meanwhile, the entire struct if_data was reviewed.

o Remove the if_baudrate_pf crutch.

o Make all fields of struct if_data fixed machine independent size. The
  notion of data (packet counters, etc) are by no means MD. And it is a
  bug that on amd64 we've got a 64-bit counters, while on i386 32-bit,
  which at modern speeds overflow within a second.

  This also removes quite a lot of COMPAT_FREEBSD32 code.

o Give 16 bit for the ifi_datalen field. This field was provided to
  make future changes to if_data less ABI breaking. Unfortunately the
  8 bit size of it had effectively limited sizeof if_data to 256 bytes.

o Give 32 bits to ifi_mtu and ifi_metric.
o Give 64 bits to the rest of fields, since they are counters.

__FreeBSD_version bumped.

Discussed with:	emax
Sponsored by:	Netflix
Sponsored by:	Nginx, Inc.
2014-03-13 03:42:24 +00:00
Gleb Smirnoff
76039bc84f The r48589 promised to remove implicit inclusion of if_var.h soon. Prepare
to this event, adding if_var.h to files that do need it. Also, include
all includes that now are included due to implicit pollution via if_var.h

Sponsored by:	Netflix
Sponsored by:	Nginx, Inc.
2013-10-26 17:58:36 +00:00
Scott Long
c68534f1d5 Update PCI drivers to no longer look at the MEMIO-enabled bit in the PCI
command register.  The lazy BAR allocation code in FreeBSD sometimes
disables this bit when it detects a range conflict, and will re-enable
it on demand when a driver allocates the BAR.  Thus, the bit is no longer
a reliable indication of capability, and should not be checked.  This
results in the elimination of a lot of code from drivers, and also gives
the opportunity to simplify a lot of drivers to use a helper API to set
the busmaster enable bit.

This changes fixes some recent reports of disk controllers and their
associated drives/enclosures disappearing during boot.

Submitted by:	jhb
Reviewed by:	jfv, marius, achadd, achim
MFC after:	1 day
2013-08-12 23:30:01 +00:00
Gabor Kovesdan
b78540b1c7 - Correct mispellings of word resource
Submitted by:	Christoph Mallon <christoph.mallon@gmx.de>
2013-04-17 11:47:32 +00:00
Andrew Gallatin
dedbe8362f Several cleanups and fixes to mxge:
- Remove vestigial null pointer tests after malloc(..., M_WAITOK).

- Remove vestigal qualhack union

- Use strlcpy() instead of the error-prone strncpy() when parsing
  EEPROM and copying strings

- Check the MAC address in the EEPROM strings more strictly.

- Expand the macro MXGE_NEXT_STRING() at its only user. Due to a typo,
  the macro was very confusing.

- Remove unnecessary buffer limit check.  The buffer is double-NUL
  terminated per construction.

PR:		kern/176369
Submitted by:	Christoph Mallon <christoph.mallon gmx.de>
2013-02-25 16:22:40 +00:00
Andrew Gallatin
cabc512fe4 Bump mxge copyright.
Sponsored by: Myricom

MFC After: 7 days
2013-02-22 19:23:33 +00:00
Andrew Gallatin
a4b233dd06 Improvements for newer mxge nics:
- Some mxge nics may store the serial number in the SN2 field of the
  EEPROM.  These will also have an SN=0 field, so parse the SN2 field,
  and give it precedence.

- Skip MXGEFW_CMD_UNALIGNED_TEST on mxge nics which do not require it.
  This saves roughly 10ms per port at device attach time.

Sponsored by: Myricom

MFC After: 7 days
2013-02-22 19:21:29 +00:00
Andrew Gallatin
abc5b96b99 Try harder to make mxge safe for all combinations of INET and INET6
- Re-fix build by restoring local removed in r247151, but protected
  by #if defined(INET) || defined(INET6) so that the compile
  succeeds in the !(INET||INET6) case.

- Protect call to in_pseudo() with an #ifdef INET, to allow
  a kernel to link with mxge when INET is not compiled in.

- Also remove an errant (improperly commented) obsolete debugging printf

Thanks to Glebius for pointing out the !(INET||INET6) build issue.

Sponsored by: Myricom

MFC After: 7 days
2013-02-22 16:46:28 +00:00
Gleb Smirnoff
60f2751670 Fix build. 2013-02-22 12:41:24 +00:00
Andrew Gallatin
26dd49c61d Improve mxge's receive performance for IPv6:
- Add support for IPv6 rx csum offload
- Finally switch mxge from using its own driver lro, to
	using tcp_lro

MFC after:	7 days
Sponsored by: Myricom Inc.
2013-02-21 21:28:33 +00:00
Andrew Gallatin
0a7a780eee Add support to mxge for IPv6 TX csum offload & IPv6 TSO.
Sponsored by: Myricom, Inc.
MFC after: 7 days
2013-02-19 21:33:21 +00:00
Sofian Brabez
61bfd86762 Use DEVMETHOD_END macro defined in sys/bus.h instead of {0, 0} sentinel on device_method_t arrays
Reviewed by:	cognet
Approved by:	cognet
2013-01-30 18:01:20 +00:00
Gleb Smirnoff
c6499eccad Mechanically substitute flags from historic mbuf allocator with
malloc(9) flags in sys/dev.
2012-12-04 09:32:43 +00:00
Gleb Smirnoff
73b160cf5b Utilize new macro to initialize if_baudrate. 2012-10-18 13:57:28 +00:00
Gleb Smirnoff
063efed28c The drbr(9) API appeared to be so unclear, that most drivers in
tree used it incorrectly, which lead to inaccurate overrated
if_obytes accounting. The drbr(9) used to update ifnet stats on
drbr_enqueue(), which is not accurate since enqueuing doesn't
imply successful processing by driver. Dequeuing neither mean
that. Most drivers also called drbr_stats_update() which did
accounting again, leading to doubled if_obytes statistics. And
in case of severe transmitting, when a packet could be several
times enqueued and dequeued it could have been accounted several
times.

o Thus, make drbr(9) API thinner. Now drbr(9) merely chooses between
  ALTQ queueing or buf_ring(9) queueing.
  - It doesn't touch the buf_ring stats any more.
  - It doesn't touch ifnet stats anymore.
  - drbr_stats_update() no longer exists.

o buf_ring(9) handles its stats itself:
  - It handles br_drops itself.
  - br_prod_bytes stats are dropped. Rationale: no one ever
    reads them but update of a common counter on every packet
    negatively affects performance due to excessive cache
    invalidation.
  - buf_ring_enqueue_bytes() reduced to buf_ring_enqueue(), since
    we no longer account bytes.

o Drivers handle their stats theirselves: if_obytes, if_omcasts.

o mlx4(4), igb(4), em(4), vxge(4), oce(4) and  ixv(4) no longer
  use drbr_stats_update(), and update ifnet stats theirselves.

o bxe(4) was the most correct driver, it didn't call
  drbr_stats_update(), thus it was the only driver accurate under
  moderate load. Now it also maintains stats itself.

o ixgbe(4) had already taken stats from hardware, so just
  - drop software stats updating.
  - take multicast packet count from hardware as well.

o mxge(4) just no longer needs NO_SLOW_STATS define.

o cxgb(4), cxgbe(4) need no change, since they obtain stats
  from hardware.

Reviewed by:	jfv, gnn
2012-09-28 18:28:27 +00:00
Scott Long
62ce43ccc8 More conversions of drivers to use the PCI parent DMA tag. 2012-03-12 18:15:08 +00:00
Ed Schouten
dc15eac046 Use strchr() and strrchr().
It seems strchr() and strrchr() are used more often than index() and
rindex(). Therefore, simply migrate all kernel code to use it.

For the XFS code, remove an empty line to make the code identical to
the code in the Linux kernel.
2012-01-02 12:12:10 +00:00
Andrew Gallatin
30882b10fc Fix media reporting for dual port CX4 myri10ge NICs
MFC after: 7 days
Sponsored by: Myricom, Inc.
2011-07-12 14:43:49 +00:00
Andrew Gallatin
3cae73118e Implement mxge_init()
This fixes a long standing bug in mxge(4) where "ifconfig mxge0 $IP"
did not bring the interface into a RUNNING state, like it does on
most (all?) other FreeBSD NIC drivers.

Thanks to gnn for mentioning the bug, and yongari for pointing out that
ether_ioctl() invokes ifp->if_init() in SIOCSIFADDR.

MFC after: 7 days
2011-04-06 15:45:32 +00:00
John Baldwin
3b0a4aef96 Do a sweep of the tree replacing calls to pci_find_extcap() with calls to
pci_find_cap() instead.
2011-03-23 13:10:15 +00:00
John Baldwin
a4a75d679c Use a regular taskqueue rather than a fast taskqueue for mxge(4).
Reviewed by:	gallatin
2011-01-07 16:07:29 +00:00
Andrew Gallatin
4ed8ca8f25 Fix a TSO checksum bug on mxge(4):
The Myri10GE NIC will assume all TSO frames contain partial checksum,
and will emit TSO segments with bad TCP checksums if a TSO frame
contains a full checksum.  The mxge driver takes care to make sure
that TSO is disabled when checksum offload is disabled for this
reason.  However, modules that modify packet contents (like pf) may
end up completing a checksum on a TSO frame, leading to the NIC emitting
TSO segments with bad checksums.

To workaround this, restore the partial checksum in the mxge driver
when we're fed a TSO frame with a full checksum.

Reported by: Bob Healey

MFC after:	3 days
2010-11-22 16:43:05 +00:00
Andrew Gallatin
2108913743 Add interrupt descriptions for mxge's msi-x vectors 2010-05-21 15:57:24 +00:00
Andrew Gallatin
56b6785885 Correctly identify some twinax cables, which report
a media type of 1.
2010-05-19 20:00:15 +00:00
Fabien Thomas
b92990ef88 Add a fastpath to allocate from packet zone when using m_getjcl.
This will add support for packet zone for at least igb and ixgbe
and will avoid to check for that in bce and mxge.

MFC after: 1 week
2010-05-07 22:09:17 +00:00
Andrew Gallatin
2a29e09f38 Add missing IFCAP_LINKSTATE to mxge
Submitted by: yongari
2010-04-15 14:26:52 +00:00
Andrew Gallatin
c406ad2ecc Cleanup if_media handling in mxge(4)
- Re-probe xfp / sfp+ socket on link events, in case user
    has changed transceiver
- correctly report current media to avoid confusing lagg (reported by Panasas)
- Report link speed  (submitted by yongari)

Reviewed by: yongari (earlier version)

MFC after: 7 days
2010-04-15 13:50:55 +00:00
Andrew Gallatin
2e08479882 Fix 2 bugs in mxge_attach()
- Don't leak slice resources when mxge_alloc_rings() fails

- Start taskq threads only after we know attach will succeed.  At
  boot time, taskqueue_terminate() will loop infinately, waiting
  for the threads to exit, and hang the system.

Submitted by: Panasas
MFC After: 3 days
2010-03-17 20:13:09 +00:00
Andrew Gallatin
0dce6781ae Update mxge to support IFCAP_VLAN_HWTSO.
Note: If/when FreeBSD supports TSO over IPv6, the minimal mxge fw
rev to enable IFCAP_VLAN_HWTSO will need to be increased to 1.4.37
2010-02-22 16:57:03 +00:00
Max Laier
193cbc4d24 Fix drbr and altq interaction:
- introduce drbr_needs_enqueue that returns whether the interface/br needs
   an enqueue operation: returns true if altq is enabled or there are
   already packets in the ring (as we need to maintain packet order)
 - update all drbr consumers
 - fix drbr_flush
 - avoid using the driver queue (IFQ_DRV_*) in the altq case as the
   multiqueue consumer does not provide enough protection, serialize altq
   interaction with the main queue lock
 - make drbr_dequeue_cond work with altq

Discussed with:		kmacy, yongari, jfv
MFC after:		4 weeks
2010-02-13 16:04:58 +00:00
Andrew Gallatin
5769c5efb0 Use better default RSS hash (src + dst, rather than just src port)
MFC after:3 days
2010-01-11 22:28:40 +00:00
Andrew Gallatin
51bc2092fe Fix reporting of 10G Twinax media
Reported by: mjacob
MFC after: 3 days
2010-01-11 22:08:59 +00:00
Martin Blapp
c2ede4b379 Remove extraneous semicolons, no functional changes.
Submitted by:	Marc Balmer <marc@msys.ch>
MFC after:	1 week
2010-01-07 21:01:37 +00:00
Andrew Gallatin
57e42c95a4 Don't take the driver mutex in mxge_tick(), as it
is run with the mutex held.

Submitted by: rwatson
MFC after:	3 days
2009-12-22 15:41:58 +00:00
Andrew Gallatin
6b484a49cd Make mxge do a better job recovering from NIC h/w faults
by checking PCI config space when the NIC is not
transmitting.  Previously, a h/w fault would not have been
detected if the NIC was down, or handling an RX only
workload.
2009-10-20 18:58:28 +00:00
Andrew Gallatin
72c042dfb5 Move mxge(4)'s NIC watchdog reset handler from
a callout to a taskqueue
2009-10-19 20:51:27 +00:00
Andrew Gallatin
83d54b590f Two more mxge watchdog fixes:
1) Restore the PCI Express control register after a watchdog
   reset.  This is required because the device will come out
   of watchdog reset with the pectl reg at its default state,
   and important BIOS configuration (like max payload size)
   could be lost.

2) Call mxge_start_locked() for every tx queue before dropping
   the lock in the watchdog handler.   This is required, as
   the queue's buf ring may have filled during the reset.
2009-09-30 14:42:06 +00:00
Andrew Gallatin
a393336b87 Improve mxge watchdog routine's ability to reliably reset a failed NIC:
- Mark the link as down, so if watchdog reset fails, link watching
    failover software can notice it
- Don't send MXGEFW_CMD_ETHERNET_DOWN if the NIC has been reset, it is
    not needed, and will fail on a freshly reset NIC.
- Ensure the transmit routines aren't attempting to PIO write to doorbells
    while the NIC is being reset.
- Download the correct f/w, rather than using the EEPROM f/w after reset.
- Export a count of the number of watchdog resets via sysctl
- Zero all f/w stats at reset.  This will lead to less confusing
    diagnostic output when investigating NIC failures.

MFC after:	3 days
2009-09-21 20:16:10 +00:00