d/freebsd-skq
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
377421df96
freebsd-skq/sys/dev/qlxgbe
History
Matt Macy d7c5a620e2 ifnet: Replace if_addr_lock rwlock with epoch + mutex 
Run on LLNW canaries and tested by pho@

gallatin:
Using a 14-core, 28-HTT single socket E5-2697 v3 with a 40GbE MLX5
based ConnectX 4-LX NIC, I see an almost 12% improvement in received
packet rate, and a larger improvement in bytes delivered all the way
to userspace.

When the host receiving 64 streams of netperf -H $DUT -t UDP_STREAM -- -m 1,
I see, using nstat -I mce0 1 before the patch:

InMpps OMpps  InGbs  OGbs err TCP Est %CPU syscalls csw     irq GBfree
4.98   0.00   4.42   0.00 4235592     33   83.80 4720653 2149771   1235 247.32
4.73   0.00   4.20   0.00 4025260     33   82.99 4724900 2139833   1204 247.32
4.72   0.00   4.20   0.00 4035252     33   82.14 4719162 2132023   1264 247.32
4.71   0.00   4.21   0.00 4073206     33   83.68 4744973 2123317   1347 247.32
4.72   0.00   4.21   0.00 4061118     33   80.82 4713615 2188091   1490 247.32
4.72   0.00   4.21   0.00 4051675     33   85.29 4727399 2109011   1205 247.32
4.73   0.00   4.21   0.00 4039056     33   84.65 4724735 2102603   1053 247.32

After the patch

InMpps OMpps  InGbs  OGbs err TCP Est %CPU syscalls csw     irq GBfree
5.43   0.00   4.20   0.00 3313143     33   84.96 5434214 1900162   2656 245.51
5.43   0.00   4.20   0.00 3308527     33   85.24 5439695 1809382   2521 245.51
5.42   0.00   4.19   0.00 3316778     33   87.54 5416028 1805835   2256 245.51
5.42   0.00   4.19   0.00 3317673     33   90.44 5426044 1763056   2332 245.51
5.42   0.00   4.19   0.00 3314839     33   88.11 5435732 1792218   2499 245.52
5.44   0.00   4.19   0.00 3293228     33   91.84 5426301 1668597   2121 245.52

Similarly, netperf reports 230Mb/s before the patch, and 270Mb/s after the patch

Reviewed by:	gallatin
Sponsored by:	Limelight Networks
Differential Revision:	https://reviews.freebsd.org/D15366
2018-05-18 20:13:34 +00:00
..
ql_boot.c Upgraded FW Related Files to version 5.4.67 2018-04-25 17:51:17 +00:00
ql_dbg.c
ql_dbg.h 1. Add additional debug prints. 2018-03-29 17:36:34 +00:00
ql_def.h
ql_fw.c Upgraded FW Related Files to version 5.4.67 2018-04-25 17:51:17 +00:00
ql_glbl.h
ql_hw.c 1. Add additional debug prints. 2018-03-29 17:36:34 +00:00
ql_hw.h
ql_inline.h
ql_ioctl.c 1. Add additional debug prints. 2018-03-29 17:36:34 +00:00
ql_ioctl.h
ql_isr.c 1. Add additional debug prints. 2018-03-29 17:36:34 +00:00
ql_minidump.c Upgraded FW Related Files to version 5.4.67 2018-04-25 17:51:17 +00:00
ql_minidump.h
ql_misc.c
ql_os.c ifnet: Replace if_addr_lock rwlock with epoch + mutex 2018-05-18 20:13:34 +00:00
ql_os.h
ql_reset.c Upgraded FW Related Files to version 5.4.67 2018-04-25 17:51:17 +00:00
ql_tmplt.h
ql_ver.h Upgraded FW Related Files to version 5.4.67 2018-04-25 17:51:17 +00:00
README.txt

README.txt 

# $FreeBSD$

			README File
		QLogic 8300 series Dual Port
10 Gigabit Ethernet & CNA Adapter Driver for FreeBSD 9.x/10.x

		QLogic Corporation.
		All rights reserved. 


Table of Contents
1. Package Contents 
2. OS Support
3. Supported Features
4. Using the Driver
   4.1 Installing the driver
   4.2 Removing the driver
5. Driver Parameters
6. Additional Notes
7. Contacting Support

1. Package Contents 
 * Documentation 
   - README (this document) version:1.0
   - Release Notes Version:1.0
 * Driver (if_qlxgbe.ko)
	- FreeBSD 9.x/10.x
 * Firmware: pre-flashed on QLogic adapter;

2. OS Support

The Qlogic 83xx 10Gigabit Ethernet/CNA driver is compatible with the 
following OS platforms:
 * FreeBSD 9.x/10.x (64-bit) [Intel EM64T, AMD64]

3. Supported Features
10Gigabit Ethernet NIC/CNA driver supports following features

* Large Segment Offload over TCP IPV4
* Large Segment Offload over TCP IPV6
* Receive Side scaling
* TCP over IPv4 checksum offload
* UDP over IPv4 checksum offload
* IPV4 checksum offload
* TCP over IPv6 checksum offload
* UDP over IPv6 checksum offload
* Jumbo frames
* VLAN Tag


4. Using the driver

 4.1 Installing the driver

   - copy the driver file (if_qlxgbe.ko) into some directory (say qla_driver)
   - cd <to qla_driver>
   - kldload -v ./if_qlxgbe.ko

 4.2 Removing the driver
 
  - kldunload if_qlxgbe

5. Parameters to set prior to installing the driver
     Please run  "sysctl kern.ipc" and "sysctl net.inet.tcp" and see if these
     values are already greater than shown below. Change only those which
     are less than shown below.

   - Add the following lines to /etc/sysctl.conf and reboot the machine prior
     to installing the driver
   
	kern.ipc.nmbjumbo9=2000000
	kern.ipc.nmbclusters=1000000
	net.inet.tcp.recvbuf_max=262144
	net.inet.tcp.recvbuf_inc=16384
	kern.ipc.maxsockbuf=2097152
	net.inet.tcp.recvspace=131072
	net.inet.tcp.sendbuf_max=262144
	net.inet.tcp.sendspace=65536
 
   - If you do not want to reboot the system please run the following commands

	login or su to root

	sysctl kern.ipc.nmbjumbo9=2000000
	sysctl kern.ipc.nmbclusters=1000000
	sysctl net.inet.tcp.recvbuf_max=262144
	sysctl net.inet.tcp.recvbuf_inc=16384
	sysctl kern.ipc.maxsockbuf=2097152
	sysctl net.inet.tcp.recvspace=131072
	sysctl net.inet.tcp.sendbuf_max=262144
	sysctl net.inet.tcp.sendspace=65536

6. Compile options Makefile if building driver from sources
	None

7. Contacting Support 
Please feel free to contact your QLogic approved reseller or QLogic 
Technical Support at any phase of integration for assistance. QLogic
Technical Support can be reached by the following methods: 
Web:    http://support.qlogic.com
E-mail: support@qlogic.com
(c) Copyright 2013-14. All rights reserved worldwide. QLogic, the QLogic 
logo, and the Powered by QLogic logo are registered trademarks of
QLogic Corporation. All other brand and product names are trademarks 
or registered trademarks of their respective owners.