d/freebsd-dev
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
af2ffa3d52
freebsd-dev/sys/netinet
History
Matt Macy d7c5a620e2 ifnet: Replace if_addr_lock rwlock with epoch + mutex 
Run on LLNW canaries and tested by pho@

gallatin:
Using a 14-core, 28-HTT single socket E5-2697 v3 with a 40GbE MLX5
based ConnectX 4-LX NIC, I see an almost 12% improvement in received
packet rate, and a larger improvement in bytes delivered all the way
to userspace.

When the host receiving 64 streams of netperf -H $DUT -t UDP_STREAM -- -m 1,
I see, using nstat -I mce0 1 before the patch:

InMpps OMpps  InGbs  OGbs err TCP Est %CPU syscalls csw     irq GBfree
4.98   0.00   4.42   0.00 4235592     33   83.80 4720653 2149771   1235 247.32
4.73   0.00   4.20   0.00 4025260     33   82.99 4724900 2139833   1204 247.32
4.72   0.00   4.20   0.00 4035252     33   82.14 4719162 2132023   1264 247.32
4.71   0.00   4.21   0.00 4073206     33   83.68 4744973 2123317   1347 247.32
4.72   0.00   4.21   0.00 4061118     33   80.82 4713615 2188091   1490 247.32
4.72   0.00   4.21   0.00 4051675     33   85.29 4727399 2109011   1205 247.32
4.73   0.00   4.21   0.00 4039056     33   84.65 4724735 2102603   1053 247.32

After the patch

InMpps OMpps  InGbs  OGbs err TCP Est %CPU syscalls csw     irq GBfree
5.43   0.00   4.20   0.00 3313143     33   84.96 5434214 1900162   2656 245.51
5.43   0.00   4.20   0.00 3308527     33   85.24 5439695 1809382   2521 245.51
5.42   0.00   4.19   0.00 3316778     33   87.54 5416028 1805835   2256 245.51
5.42   0.00   4.19   0.00 3317673     33   90.44 5426044 1763056   2332 245.51
5.42   0.00   4.19   0.00 3314839     33   88.11 5435732 1792218   2499 245.52
5.44   0.00   4.19   0.00 3293228     33   91.84 5426301 1668597   2121 245.52

Similarly, netperf reports 230Mb/s before the patch, and 270Mb/s after the patch

Reviewed by:	gallatin
Sponsored by:	Limelight Networks
Differential Revision:	https://reviews.freebsd.org/D15366
2018-05-18 20:13:34 +00:00
..
cc Plug a memory leak and potential NULL-pointer dereference introduced in r331214. 2018-05-17 02:46:27 +00:00
khelp sys: general adoption of SPDX licensing ID tags. 2017-11-27 15:23:17 +00:00
libalias libalias: small memory allocation cleanups. 2018-01-12 23:12:30 +00:00
netdump Fix netdump configuration when VIMAGE is enabled. 2018-05-17 04:08:57 +00:00
tcp_stacks This commit brings in the TCP high precision timer system (tcp_hpts). 2018-04-19 13:37:59 +00:00
accf_data.c sys: general adoption of SPDX licensing ID tags. 2017-11-27 15:23:17 +00:00
accf_dns.c sys: general adoption of SPDX licensing ID tags. 2017-11-27 15:23:17 +00:00
accf_http.c sys: general adoption of SPDX licensing ID tags. 2017-11-27 15:23:17 +00:00
icmp6.h Correct the comment describing badrs which is bad router solicitiation, 2017-12-29 07:23:18 +00:00
icmp_var.h sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
if_ether.c ifnet: Replace if_addr_lock rwlock with epoch + mutex 2018-05-18 20:13:34 +00:00
if_ether.h sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
igmp_var.h Separate list manipulation locking from state change in multicast 2018-05-02 19:36:29 +00:00
igmp.c ifnet: Replace if_addr_lock rwlock with epoch + mutex 2018-05-18 20:13:34 +00:00
igmp.h sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
in_cksum.c sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
in_debug.c ifnet: Replace if_addr_lock rwlock with epoch + mutex 2018-05-18 20:13:34 +00:00
in_fib.c Renumber copyright clause 4 2017-02-28 23:42:47 +00:00
in_fib.h Renumber copyright clause 4 2017-02-28 23:42:47 +00:00
in_gif.c sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
in_jail.c Move most of the contents of opt_compat.h to opt_global.h. 2018-04-06 17:35:35 +00:00
in_kdtrace.c Use tcpinfoh_t for TCP headers in the tcp:::debug-{drop,input} probes. 2018-01-25 15:35:34 +00:00
in_kdtrace.h sys: general adoption of SPDX licensing ID tags. 2017-11-27 15:23:17 +00:00
in_mcast.c ifnet: Replace if_addr_lock rwlock with epoch + mutex 2018-05-18 20:13:34 +00:00
in_pcb.c ifnet: Replace if_addr_lock rwlock with epoch + mutex 2018-05-18 20:13:34 +00:00
in_pcb.h Revert r332894 at the request of the submitter. 2018-04-24 19:55:12 +00:00
in_pcbgroup.c sys: general adoption of SPDX licensing ID tags. 2017-11-27 15:23:17 +00:00
in_prot.c Move most of the contents of opt_compat.h to opt_global.h. 2018-04-06 17:35:35 +00:00
in_proto.c sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
in_rmx.c Code duplication but rib_head is special. Not found an easy way to go 2016-02-03 21:56:51 +00:00
in_rss.c Rename rss_soft_m2cpuid() -> rss_soft_m2cpuid_v4() in preparation for 2015-08-29 06:58:30 +00:00
in_rss.h Rename rss_soft_m2cpuid() -> rss_soft_m2cpuid_v4() in preparation for 2015-08-29 06:58:30 +00:00
in_systm.h sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
in_var.h ifnet: Replace if_addr_lock rwlock with epoch + mutex 2018-05-18 20:13:34 +00:00
in.c ifnet: Replace if_addr_lock rwlock with epoch + mutex 2018-05-18 20:13:34 +00:00
in.h sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
ip6.h sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
ip_carp.c ifnet: Replace if_addr_lock rwlock with epoch + mutex 2018-05-18 20:13:34 +00:00
ip_carp.h sys: general adoption of SPDX licensing ID tags. 2017-11-27 15:23:17 +00:00
ip_divert.c ifnet: Replace if_addr_lock rwlock with epoch + mutex 2018-05-18 20:13:34 +00:00
ip_divert.h sys: general adoption of SPDX licensing ID tags. 2017-11-27 15:23:17 +00:00
ip_dummynet.h sys: general adoption of SPDX licensing ID tags. 2017-11-27 15:23:17 +00:00
ip_ecn.c sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
ip_ecn.h sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
ip_encap.c sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
ip_encap.h sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
ip_fastfwd.c netpfil: Introduce PFIL_FWD flag 2018-03-23 16:56:44 +00:00
ip_fw.h Rework ipfw dynamic states implementation to be lockless on fast path. 2018-02-07 18:59:54 +00:00
ip_gre.c sys: general adoption of SPDX licensing ID tags. 2017-11-27 15:23:17 +00:00
ip_icmp.c ifnet: Replace if_addr_lock rwlock with epoch + mutex 2018-05-18 20:13:34 +00:00
ip_icmp.h sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
ip_id.c sys: general adoption of SPDX licensing ID tags. 2017-11-27 15:23:17 +00:00
ip_input.c ifnet: Replace if_addr_lock rwlock with epoch + mutex 2018-05-18 20:13:34 +00:00
ip_mroute.c style: Remove remaining deprecated MALLOC/FREE macros 2018-01-25 22:25:13 +00:00
ip_mroute.h sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
ip_options.c sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
ip_options.h sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
ip_output.c Revert r332894 at the request of the submitter. 2018-04-24 19:55:12 +00:00
ip_reass.c Do not generate illegal mbuf chains during IP fragment reassembly. Only 2018-01-24 05:09:21 +00:00
ip_var.h Currently in_pcbfree will unconditionally wunlock the pcbinfo lock 2018-05-05 22:40:40 +00:00
ip.h sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
pim_var.h sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
pim.h sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
raw_ip.c ifnet: Replace if_addr_lock rwlock with epoch + mutex 2018-05-18 20:13:34 +00:00
sctp_asconf.c sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
sctp_asconf.h sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
sctp_auth.c sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
sctp_auth.h sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
sctp_bsd_addr.c ifnet: Replace if_addr_lock rwlock with epoch + mutex 2018-05-18 20:13:34 +00:00
sctp_bsd_addr.h sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
sctp_cc_functions.c sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
sctp_constants.h Add constant for the PAD chunk as defined in RFC 4820. 2018-01-27 13:46:55 +00:00
sctp_crc32.c When adding support for sending SCTP packets containing an ABORT chunk 2017-12-26 12:35:02 +00:00
sctp_crc32.h When adding support for sending SCTP packets containing an ABORT chunk 2017-12-26 12:35:02 +00:00
sctp_dtrace_declare.h sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
sctp_dtrace_define.h sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
sctp_header.h sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
sctp_indata.c Ensure we are not dereferencing a NULL pointer. 2018-05-06 14:19:50 +00:00
sctp_indata.h sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
sctp_input.c Fix a signed/unsigned warning showing up for the userland stack 2018-04-08 11:37:00 +00:00
sctp_input.h Retire SCTP_WITH_NO_CSUM option. 2017-12-07 22:19:08 +00:00
sctp_lock_bsd.h sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
sctp_os_bsd.h Move most of the contents of opt_compat.h to opt_global.h. 2018-04-06 17:35:35 +00:00
sctp_os.h sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
sctp_output.c sctp_get_mbuf_for_msg() should honor the allinone parameter. 2018-05-14 15:16:51 +00:00
sctp_output.h sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
sctp_pcb.c sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
sctp_pcb.h sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
sctp_peeloff.c sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
sctp_peeloff.h sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
sctp_ss_functions.c Fix SPDX line as suggested by pfg 2017-11-24 19:38:59 +00:00
sctp_structs.h sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
sctp_syscalls.c Move most of the contents of opt_compat.h to opt_global.h. 2018-04-06 17:35:35 +00:00
sctp_sysctl.c Retire SCTP_WITH_NO_CSUM option. 2017-12-07 22:19:08 +00:00
sctp_sysctl.h sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
sctp_timer.c sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
sctp_timer.h sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
sctp_uio.h Retire SCTP_WITH_NO_CSUM option. 2017-12-07 22:19:08 +00:00
sctp_usrreq.c Fix a logical inversion bug. 2018-04-08 12:08:20 +00:00
sctp_var.h sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
sctp.h sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
sctputil.c Ensure that the MTU's used are multiple of 4. 2018-05-14 13:50:17 +00:00
sctputil.h sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
siftr.c sys: general adoption of SPDX licensing ID tags. 2017-11-27 15:23:17 +00:00
tcp_debug.c sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
tcp_debug.h sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
tcp_fastopen.c When checking the TCP fast cookie length, conststently also check 2018-02-27 22:12:38 +00:00
tcp_fastopen.h Greatly reduce the number of #ifdefs supporting the TCP_RFC7413 kernel option. 2018-02-26 03:03:41 +00:00
tcp_fsm.h sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
tcp_hostcache.c sys: general adoption of SPDX licensing ID tags. 2017-11-27 15:23:17 +00:00
tcp_hostcache.h sys: general adoption of SPDX licensing ID tags. 2017-11-27 15:23:17 +00:00
tcp_hpts.c Minor style nits 2018-05-09 14:11:35 +00:00
tcp_hpts.h Use the full year, for real this time. 2018-05-09 20:26:37 +00:00
tcp_input.c Fix spurious retransmit recovery on low latency networks 2018-05-08 02:22:34 +00:00
tcp_log_buf.c Clean up some debugging code left in tcp_log_buf.c from r331347. 2018-04-10 15:51:37 +00:00
tcp_log_buf.h Make the TCP blackbox code committed in r331347 be an optional feature 2018-03-24 12:48:10 +00:00
tcp_lro.c Update tcp_lro with tested bugfixes from Netflix and LLNW: 2018-03-09 00:08:43 +00:00
tcp_lro.h sys: general adoption of SPDX licensing ID tags. 2017-11-27 15:23:17 +00:00
tcp_offload.c Add a hook to allow the toedev handling an offloaded connection to 2018-04-03 01:08:54 +00:00
tcp_offload.h Add a hook to allow the toedev handling an offloaded connection to 2018-04-03 01:08:54 +00:00
tcp_output.c Fix spurious retransmit recovery on low latency networks 2018-05-08 02:22:34 +00:00
tcp_pcap.c The TCPPCAP debugging feature caches recently-used mbufs for use in 2016-07-06 16:17:13 +00:00
tcp_pcap.h The TCPPCAP debugging feature caches recently-used mbufs for use in 2016-07-06 16:17:13 +00:00
tcp_reass.c sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
tcp_sack.c sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
tcp_seq.h r330675 introduced an extra window check in the LRO code to ensure it 2018-04-03 13:54:38 +00:00
tcp_subr.c Revert r332894 at the request of the submitter. 2018-04-24 19:55:12 +00:00
tcp_syncache.c This commit brings in the TCP high precision timer system (tcp_hpts). 2018-04-19 13:37:59 +00:00
tcp_syncache.h Greatly reduce the number of #ifdefs supporting the TCP_RFC7413 kernel option. 2018-02-26 03:03:41 +00:00
tcp_timer.c Fix spurious retransmit recovery on low latency networks 2018-05-08 02:22:34 +00:00
tcp_timer.h Export tcp_always_keepalive for use by the Chelsio TOM module. 2018-01-30 23:01:37 +00:00
tcp_timewait.c Fix LINT-NOINET build initializing local to false. This is 2018-03-22 05:07:57 +00:00
tcp_usrreq.c These two modules need the tcp_hpts.h file for 2018-04-19 15:03:48 +00:00
tcp_var.h This change re-arranges the fields within the tcp-pcb so that 2018-04-26 21:41:16 +00:00
tcp.h Add the "TCP Blackbox Recorder" which we discussed at the developer 2018-03-22 09:40:08 +00:00
tcpip.h sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
toecore.c Add a hook to allow the toedev handling an offloaded connection to 2018-04-03 01:08:54 +00:00
toecore.h Add a hook to allow the toedev handling an offloaded connection to 2018-04-03 01:08:54 +00:00
udp_usrreq.c Revert r332894 at the request of the submitter. 2018-04-24 19:55:12 +00:00
udp_var.h sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
udp.h sys: further adoption of SPDX licensing ID tags. 2017-11-20 19:43:44 +00:00
udplite.h