Commit Graph

36 Commits

Author SHA1 Message Date
Matt Macy
d7c5a620e2 ifnet: Replace if_addr_lock rwlock with epoch + mutex
Run on LLNW canaries and tested by pho@

gallatin:
Using a 14-core, 28-HTT single socket E5-2697 v3 with a 40GbE MLX5
based ConnectX 4-LX NIC, I see an almost 12% improvement in received
packet rate, and a larger improvement in bytes delivered all the way
to userspace.

When the host receiving 64 streams of netperf -H $DUT -t UDP_STREAM -- -m 1,
I see, using nstat -I mce0 1 before the patch:

InMpps OMpps  InGbs  OGbs err TCP Est %CPU syscalls csw     irq GBfree
4.98   0.00   4.42   0.00 4235592     33   83.80 4720653 2149771   1235 247.32
4.73   0.00   4.20   0.00 4025260     33   82.99 4724900 2139833   1204 247.32
4.72   0.00   4.20   0.00 4035252     33   82.14 4719162 2132023   1264 247.32
4.71   0.00   4.21   0.00 4073206     33   83.68 4744973 2123317   1347 247.32
4.72   0.00   4.21   0.00 4061118     33   80.82 4713615 2188091   1490 247.32
4.72   0.00   4.21   0.00 4051675     33   85.29 4727399 2109011   1205 247.32
4.73   0.00   4.21   0.00 4039056     33   84.65 4724735 2102603   1053 247.32

After the patch

InMpps OMpps  InGbs  OGbs err TCP Est %CPU syscalls csw     irq GBfree
5.43   0.00   4.20   0.00 3313143     33   84.96 5434214 1900162   2656 245.51
5.43   0.00   4.20   0.00 3308527     33   85.24 5439695 1809382   2521 245.51
5.42   0.00   4.19   0.00 3316778     33   87.54 5416028 1805835   2256 245.51
5.42   0.00   4.19   0.00 3317673     33   90.44 5426044 1763056   2332 245.51
5.42   0.00   4.19   0.00 3314839     33   88.11 5435732 1792218   2499 245.52
5.44   0.00   4.19   0.00 3293228     33   91.84 5426301 1668597   2121 245.52

Similarly, netperf reports 230Mb/s before the patch, and 270Mb/s after the patch

Reviewed by:	gallatin
Sponsored by:	Limelight Networks
Differential Revision:	https://reviews.freebsd.org/D15366
2018-05-18 20:13:34 +00:00
Mark Peek
3eeb751122 Update VMCI license based on comments from core, the FreeBSD Foundation,
and VMware legal:
- Add a dual BSD-2 Clause/GPLv2 LICENSE file in the VMCI directory
- Remove the use of "All Rights Reserved"
- Per best practice, remove copyright/license info from Makefile

Reviewed by: imp, emaste, jhb, Vishnu Dasa <vdasa@vmware.com>
Approved by: VMware legal via Mark Peek <markpeek@vmware.com>
Differential Revision:  https://reviews.freebsd.org/D14979
2018-04-08 01:32:56 +00:00
Mark Peek
8c302b2e86 Rectify VMCI SPDX license
Approved by: Vishnu Dasa <vdasa@vmware.com>
2018-03-27 06:33:00 +00:00
Conrad Meyer
7953f98e23 vmci(4): Fix GCC build and rationalize vmci_kernel_defs.h
To fix the GCC build, remove multiple redundant declarations of
vmci_send_datagram() (the copy in vmci.h as well as the extern definition in
vmci_queue_pair.c were wholly redundant).

Also to fix the GCC build, include a non-empty format string in the vmci(4)
definition of ASSERT().  It seems harmless either way, but adding the
stringified invariant is easier than masking the warning.

The other vmci_kernel_defs.h changes are cosmetic and simply match macros to
existing definitions.

Reported by:	GCC 6.4.0
Sponsored by:	Dell EMC Isilon
2018-03-26 19:53:02 +00:00
Mark Peek
63a938566d Add VMCI (Virtual Machine Communication Interface) driver
In a virtual machine, VMCI is exposed as a regular PCI device. The primary
communication mechanisms supported are a point-to-point bidirectional
transport based on a pair of memory-mapped queues, and asynchronous
notifications in the form of datagrams and doorbells. These features are
available to kernel level components such as vSockets through the VMCI
kernel API. In addition to this, the VMCI kernel API provides support for
receiving events related to the state of the VMCI communication channels,
and the virtual machine itself.

Submitted by: Vishnu Dasa <vdasa@vmware.com>
Reviewed by: bcr, imp
Obtained from: VMware
Differential Revision: https://reviews.freebsd.org/D14289
2018-03-25 00:57:00 +00:00
Pedro F. Giffuni
ac2fffa4b7 Revert r327828, r327949, r327953, r328016-r328026, r328041:
Uses of mallocarray(9).

The use of mallocarray(9) has rocketed the required swap to build FreeBSD.
This is likely caused by the allocation size attributes which put extra pressure
on the compiler.

Given that most of these checks are superfluous we have to choose better
where to use mallocarray(9). We still have more uses of mallocarray(9) but
hopefully this is enough to bring swap usage to a reasonable level.

Reported by:	wosch
PR:		225197
2018-01-21 15:42:36 +00:00
Pedro F. Giffuni
26c1d774b5 dev: make some use of mallocarray(9).
Focus on code where we are doing multiplications within malloc(9). None of
these is likely to overflow, however the change is still useful as some
static checkers can benefit from the allocation attributes we use for
mallocarray.

This initial sweep only covers malloc(9) calls with M_NOWAIT. No good
reason but I started doing the changes before r327796 and at that time it
was convenient to make sure the sorrounding code could handle NULL values.
2018-01-13 22:30:30 +00:00
Bryan Venteicher
1b108b19ed Correctly report the vmxnet3 link down media status
Reported by:	lew@perftech.com
MFC after:	1 week
2017-11-28 06:21:37 +00:00
Andriy Gapon
ced98d784b fix vmxnet3 crash when LRO is enabled
The crash can occur when all of the following conditions are true:
- a packet consists of multiple segements (requires LRO enabled)
- there has been a failure to allocate an mbuf for the packet and
  the packet has to be dropped
- a host (vmware) still owned at least one segment of the packet,
  so the driver had to wait for another interrupt to proceed to
  discarding the remaning segment(s)

Reviewed by:	rstone
MFC after:	2 weeks
Sponsored by:	Panzura
Differential Revision: https://reviews.freebsd.org/D10874
2017-05-25 10:49:56 +00:00
Luiz Otavio O Souza
959bd29aa8 Spell VMXNET3_LEGACY_TX correctly.
Obtained from:	pfSense
MFC after:	2 weeks
Sponsored by:	Rubicon Communications, LLC (Netgate)
2017-02-18 06:18:14 +00:00
Alexander Motin
d0cefbdc0d Update if_hwassist when interface options are changed.
In particular for me this fixes checksum problem when if_bridge attached
to the interface requests TXCSUM to be disabled, but effectively ignored.

MFC after:	3 days
Sponsored by:	iXsystems, Inc.
2016-07-18 14:40:13 +00:00
John Baldwin
1941909336 Only use a power of 2 for the number of receive and transmit queues.
Using other values causes VMXNET3_CMD_ENABLE to fail.  The Linux
driver also enforces this restriction.

Reviewed by:	bryanv
MFC after:	1 week
Sponsored by:	Norse
Differential Revision:	https://reviews.freebsd.org/D4139
2015-11-16 21:36:50 +00:00
Hans Petter Selasky
c25290420e Start process of removing the use of the deprecated "M_FLOWID" flag
from the FreeBSD network code. The flag is still kept around in the
"sys/mbuf.h" header file, but does no longer have any users. Instead
the "m_pkthdr.rsstype" field in the mbuf structure is now used to
decide the meaning of the "m_pkthdr.flowid" field. To modify the
"m_pkthdr.rsstype" field please use the existing "M_HASHTYPE_XXX"
macros as defined in the "sys/mbuf.h" header file.

This patch introduces new behaviour in the transmit direction.
Previously network drivers checked if "M_FLOWID" was set in "m_flags"
before using the "m_pkthdr.flowid" field. This check has now now been
replaced by checking if "M_HASHTYPE_GET(m)" is different from
"M_HASHTYPE_NONE". In the future more hashtypes will be added, for
example hashtypes for hardware dedicated flows.

"M_HASHTYPE_OPAQUE" indicates that the "m_pkthdr.flowid" value is
valid and has no particular type. This change removes the need for an
"if" statement in TCP transmit code checking for the presence of a
valid flowid value. The "if" statement mentioned above is now a direct
variable assignment which is then later checked by the respective
network drivers like before.

Additional notes:
- The SCTP code changes will be committed as a separate patch.
- Removal of the "M_FLOWID" flag will also be done separately.
- The FreeBSD version has been bumped.

MFC after:	1 month
Sponsored by:	Mellanox Technologies
2014-12-01 11:45:24 +00:00
Gleb Smirnoff
0d286aa4cb Provide vmxnet3_get_counter() to return counters that are not collected,
but taken from hardware.
2014-09-25 08:36:11 +00:00
Hans Petter Selasky
9fd573c39d Improve transmit sending offload, TSO, algorithm in general.
The current TSO limitation feature only takes the total number of
bytes in an mbuf chain into account and does not limit by the number
of mbufs in a chain. Some kinds of hardware is limited by two
factors. One is the fragment length and the second is the fragment
count. Both of these limits need to be taken into account when doing
TSO. Else some kinds of hardware might have to drop completely valid
mbuf chains because they cannot loaded into the given hardware's DMA
engine. The new way of doing TSO limitation has been made backwards
compatible as input from other FreeBSD developers and will use
defaults for values not set.

Reviewed by:	adrian, rmacklem
Sponsored by:	Mellanox Technologies
MFC after:	1 week
2014-09-22 08:27:27 +00:00
Hans Petter Selasky
72f3100047 Revert r271504. A new patch to solve this issue will be made.
Suggested by:	adrian @
2014-09-13 20:52:01 +00:00
Hans Petter Selasky
eb93b77ae4 Improve transmit sending offload, TSO, algorithm in general.
The current TSO limitation feature only takes the total number of
bytes in an mbuf chain into account and does not limit by the number
of mbufs in a chain. Some kinds of hardware is limited by two
factors. One is the fragment length and the second is the fragment
count. Both of these limits need to be taken into account when doing
TSO. Else some kinds of hardware might have to drop completely valid
mbuf chains because they cannot loaded into the given hardware's DMA
engine. The new way of doing TSO limitation has been made backwards
compatible as input from other FreeBSD developers and will use
defaults for values not set.

MFC after:	1 week
Sponsored by:	Mellanox Technologies
2014-09-13 08:26:09 +00:00
Bryan Venteicher
a88f19d8f1 Remove some write only variables
MFC after:	3 days
2014-07-02 23:28:21 +00:00
Bryan Venteicher
efd48b30f1 Give each interrupt a descriptive name when using MSIX
MFC after:	3 days
2014-06-29 01:04:11 +00:00
Bryan Venteicher
2a87457c75 Increment the pending packets more aggressively for TSO
Assume the number of description used is reasonable value to
increment this otherwise opaque field by.

While here, reduce a minor difference between the legacy and
multiqueue transmit paths.

MFC after:	1 week
2014-06-20 02:54:04 +00:00
Bryan Venteicher
1204e3745a Handle multiple calls to rxq_eof for single packet completion
This requires the VMware vmxnet3 device to flip the start of packet
descriptor's generation before the rest of the packet's descriptors
have been loaded into the Rx ring. I've never observed this behavior,
and it seems to make the most sense not to do it this way. But it is
not a lot of work for the driver to handle this situation just in case.

MFC after:	1 week
2014-06-20 02:49:03 +00:00
Bryan Venteicher
d701c1992d Remove an unnecessary kick of the host at the end of transmitting
MFC after:	1 week
2014-06-20 02:31:52 +00:00
Hans Petter Selasky
5098dbb20d Fix GCC compile warning: Variable(s) can be used uninitialized. 2014-06-19 05:10:03 +00:00
Bryan Venteicher
9945063d70 Don't check the dma map address for a static DMA mapping against NULL
to determine if the mapping is valid.

Submitted by:	jhb
2014-06-14 23:36:17 +00:00
Bryan Venteicher
e41318fc7b Fix TSO support on VMware Fusion
Apparently for VMware Fusion (and presumably VMware Workstation/Player
since the PR states TSO is broken there too, but I cannot test), the
TCP header pseudo checksum calculated should only include the protocol
(IPPROTO_TCP) value, not also the lengths as the stack does instead.

VMware ESXi seems to ignore whatever value is in the TCP header checksum,
and it is a bit surprising there is a different behavior between the
VMware products. And it is unfortunate that on ESXi we are forced to do
this extra bit of work.

PR:		kern/185849
MFC after:	3 days
2014-06-09 02:45:53 +00:00
Bryan Venteicher
75f4c88677 Remove an unnecessary variable reassignment
And it would be bad if 'm' was different from '*m0' at this
point, since we've already populated the SG list.

MFC after:	3 days
2014-06-09 02:39:05 +00:00
Luigi Rizzo
c7156fe92f make sure if_transmit returns 0 if the mbuf is enqueued.
ixgbe/ixv.c still needs a similar fix but it takes a little
more restructuring of the code.

MFC after:	3 days
2014-06-06 20:49:56 +00:00
Bryan Venteicher
e557c1dd90 Add Tx/Rx multiqueue support to vmx(4)
As a prerequisite for multiple queues, the guest must have MSIX enabled.
Unfortunately, to work around device passthrough bugs, FreeBSD disables
MSIX when running as a VMWare guest due to the hw.pci.honor_msi_blacklist
tunable; this tunable must be disabled for multiple queues.

Also included is various minor changes from the projects/vmxnet branch.

MFC after:	1 month
2014-03-17 05:45:29 +00:00
Gleb Smirnoff
b245f96c44 Since 32-bit if_baudrate isn't enough to describe a baud rate of a 10 Gbit
interface, in the r241616 a crutch was provided. It didn't work well, and
finally we decided that it is time to break ABI and simply make if_baudrate
a 64-bit value. Meanwhile, the entire struct if_data was reviewed.

o Remove the if_baudrate_pf crutch.

o Make all fields of struct if_data fixed machine independent size. The
  notion of data (packet counters, etc) are by no means MD. And it is a
  bug that on amd64 we've got a 64-bit counters, while on i386 32-bit,
  which at modern speeds overflow within a second.

  This also removes quite a lot of COMPAT_FREEBSD32 code.

o Give 16 bit for the ifi_datalen field. This field was provided to
  make future changes to if_data less ABI breaking. Unfortunately the
  8 bit size of it had effectively limited sizeof if_data to 256 bytes.

o Give 32 bits to ifi_mtu and ifi_metric.
o Give 64 bits to the rest of fields, since they are counters.

__FreeBSD_version bumped.

Discussed with:	emax
Sponsored by:	Netflix
Sponsored by:	Nginx, Inc.
2014-03-13 03:42:24 +00:00
Gleb Smirnoff
c3322cb91c Include necessary headers that now are available due to pollution
via if_var.h.

Sponsored by:	Netflix
Sponsored by:	Nginx, Inc.
2013-10-28 07:29:16 +00:00
Gleb Smirnoff
76039bc84f The r48589 promised to remove implicit inclusion of if_var.h soon. Prepare
to this event, adding if_var.h to files that do need it. Also, include
all includes that now are included due to implicit pollution via if_var.h

Sponsored by:	Netflix
Sponsored by:	Nginx, Inc.
2013-10-26 17:58:36 +00:00
Bryan Venteicher
ce3be286f0 Do not provide a hint of the guest's OS version
The calculation can overflow if __FreeBSD_version is big
enough, and it does not appear to be required.

Reported by:	grehan
Approved by:	re (gjb)
2013-10-11 06:48:40 +00:00
Bryan Venteicher
c02d19b6b6 Use correct type for the vmx vlan filter table
Approved by:	re (glebius, gjb)
2013-09-08 19:13:06 +00:00
Bryan Venteicher
3c5dfe892d Few more minor if_vmx tweaks
- Allow the Rx/Tx queue sizes to be configured by tunables
 - Bail out earlier if the Tx queue unlikely has enough free
   descriptors to hold the frame
 - Cleanup some of the offloading capabilities handling
2013-08-30 05:53:00 +00:00
Bryan Venteicher
3c9657753e Couple minor if_vmx tweaks
- Use queue size fields from the Tx/Rx queues in various places
    instead of (currently the same values) from the softc.
  - Fix potential crash in detach if the attached failed to alloc
    queue memory.
  - Move the VMXNET3_MAX_RX_SEGS define to a better spot.
  - Tweak frame size calculation w.r.t. ETHER_ALIGN. This could be
    tweaked some more, or removed since it probably doesn't matter
    much for x86 (and the x86 class of machines this driver will
    be used on).
2013-08-27 04:05:18 +00:00
Bryan Venteicher
e3c97c2cc2 Add vmx(4), a VMware VMXNET3 ethernet driver ported from OpenBSD 2013-08-23 20:47:16 +00:00