Fix a heap corruption issue leading to a server-side crash. (#564)

Commit 5ab2132c (PR #551) fixed, among other things, a memory
leak.  The solution, however, causes a hazard where a free() of
an invalid pointer can corrupt the heap.  We've observed this
fairly repeatably while running the test_commands.sh script on
CentOS 7.

To remedy this, we NULL out a pointer after the object it
pointed to has been free-d, just like a number of other similar
objects.

(cherry picked from commit 103d4318e85783b99d6984c091ea878fc0d844b2)

.travis.yml

@@ -0,0 +1,7 @@
+language: c
+compiler:
+   - gcc
+   - clang
+os: linux
+ 
+script: ./configure && make && make check

LICENSE

@@ -1,6 +1,6 @@
-"iperf, Copyright (c) 2014, The Regents of the University of California, through
-Lawrence Berkeley National Laboratory (subject to receipt of any required
-approvals from the U.S. Dept. of Energy).  All rights reserved."
+"iperf, Copyright (c) 2014-2017, The Regents of the University of California,
+through Lawrence Berkeley National Laboratory (subject to receipt of any 
+required approvals from the U.S. Dept. of Energy).  All rights reserved."
 
 Redistribution and use in source and binary forms, with or without
 modification, are permitted provided that the following conditions are met:

README.md

@@ -13,7 +13,7 @@ This version, sometimes referred to as iperf3, is a redesign of an
 original version developed at NLANR/DAST.  iperf3 is a new
 implementation from scratch, with the goal of a smaller, simpler code
 base, and a library version of the functionality that can be used in
-other programs. iperf3 also a number of features found in other tools
+other programs. iperf3 also has a number of features found in other tools
 such as nuttcp and netperf, but were missing from the original iperf.
 These include, for example, a zero-copy mode and optional JSON output.
 Note that iperf3 is *not* backwards compatible with the original iperf.
@@ -26,6 +26,13 @@ OpenBSD, NetBSD, Android, Solaris, and other Linux distributions.
 iperf3 is principally developed by ESnet / Lawrence Berkeley National
 Laboratory.  It is released under a three-clause BSD license.
 
+Note that at this point, ESnet plans to support iperf3 in "maintenance
+mode".  At this point, there are no definite plans for further iperf3
+releases, and ESnet will be providing a very limited amount of
+resources for support and development, going forward.  However, ESnet
+could issue new iperf3 releases to deal with security issues or
+high-impact bug fixes.
+
 For more information see: http://software.es.net/iperf
 
 Source code and issue tracker: https://github.com/esnet/iperf
@@ -137,26 +144,9 @@ variables.
 Known Issues
 ------------
 
-* UDP performance: Some problems have been noticed with iperf3 on the
-ESnet 100G testbed at high UDP rates (above 10Gbps).  The symptom is
-that on any particular run of iperf3 the receiver reports a loss rate
-of about 20%, regardless of the -b option used on the client side.
-This problem appears not to be iperf3-specific, and may be due to the
-placement of the iperf3 process on a CPU and its relation to the
-inbound NIC.  In some cases this problem can be mitigated by an
-appropriate use of the CPU affinity (-A) option.  (Issue #55)
+A set of known issues is maintained on the iperf3 Web pages:
 
-* The -Z flag sometimes causes the iperf3 client to hang on OSX.
-(Issue #129)
-
-* When specifying the TCP buffer size using the "-w" flag on Linux, Linux 
-doubles the value you pass in. (You can see this using iperf3's debug flag). 
-But then the CWND does not actually ramp up to the doubled value, but only
-to about 75% of the doubled value. This appears to be by design.
-
-* On some platforms, it might be necessary to invoke "ldconfig" 
-manually after doing a "make install" before the iperf3 executable can 
-find its shared library.  (Issue #153)
+http://software.es.net/iperf/dev.html#known-issues
 
 Links
 -----
@@ -173,7 +163,7 @@ responsibility for the content of these pages.
 Copyright
 ---------
 
-iperf, Copyright (c) 2014, The Regents of the University of
+iperf, Copyright (c) 2014-2017, The Regents of the University of
 California, through Lawrence Berkeley National Laboratory (subject
 to receipt of any required approvals from the U.S. Dept. of
 Energy).  All rights reserved.

RELEASE_NOTES

@@ -1,3 +1,181 @@
+== iperf 3.1.7 2017-03-06 ==
+
+iperf 3.1.7 is functionally identical to iperf 3.1.6.  Its only
+changes consist of updated documentation files and text in the RPM
+spec file.
+
+== iperf 3.1.6 2017-02-02 ==
+
+The release notes for iperf 3.1.6 describe changes, including bug
+fixes and new functionality, made since iperf 3.1.5.
+
+* User-visible changes
+
+  * Specifying --fq-rate or --no-fq-socket-pacing on a system where
+    these options are not supported now generate an error instead of a
+    warning.  This change makes diagnosing issues related to pacing
+    more apparent.
+
+  * Fixed a bug where two recently-added diagnostic messages spammed
+    the JSON output on UDP tests.
+
+== iperf 3.1.5 2017-01-12 ==
+
+The release notes for iperf 3.1.5 describe changes, including bug
+fixes and new functionality, made since iperf 3.1.4.
+
+Compatibility note: Fair-queueing is now specified differently in
+iperf 3.1.5 than in prior versions (which include 3.1.3 and 3.1.4).
+
+Compatibility note: UDP tests may yield different results from all
+prior versions of iperf3 (through 3.1.4) due to the new default UDP
+sending size.
+
+* User-visible changes
+
+  * The fair-queueing per-socket based pacing available on recent
+    Linux systems has been reimplemented with a different user
+    interface (#325, #467, #488).  The --bandwidth command-line flag
+    now controls only the application-level pacing, as was the case in
+    iperf 3.1.2 and all earlier versions.  Fair-queueing per-socket
+    based pacing is now controlled via a new --fq-rate command-line
+    flag.  Note that TCP and UDP tests may use --bandwidth, --fq-rate,
+    both flags, or neither flag.  SCTP tests currently support
+    --bandwidth only.  The --no-fq-socket-pacing flag, which was
+    introduced in iperf 3.1.3, has now been deprecated, and is
+    equivalent to --fq-rate=0.  iperf3 now reacts more gracefully if
+    --no-fq-socket-pacing or --fq-rate are specified on platforms that
+    don't support these options.
+
+    For UDP tests, note that the default --bandwidth is 1 Mbps.  Using
+    the fair-queueing-based pacing will probably require explicitly
+    setting both --bandwidth and --fq-rate, preferably to the same
+    value.  (While setting different values for --bandwidth and
+    --fq-rate can certainly be done, the results can range from
+    entertaining to perplexing.)
+
+  * iperf3 now chooses a more sane default UDP send size (#496, #498).
+    The former default (8KB) caused IP packet fragmentation on paths
+    having smaller MTUs (including any Ethernet network not configured
+    for jumbo frames).  This could have effects ranging from increased
+    burstiness, to packet loss, to complete failure of the test.
+    iperf3 now attempts to use the MSS of the control connection to
+    determine a default UDP send size if no sending length was
+    explicitly specified with --length.
+
+  * Several checks are now made when setting the socket buffer sizes
+    with the -w option, to verify that the settings have been applied
+    correctly.  The results of these checks can been seen when the
+    --debug flag is specified.  (#356)
+
+  * A --forceflush flag has been added to flush the output stream
+    after every statistics reporting interval.
+
+* Developer-visible changes
+
+  * A systemd service file has been added (#340, #430).
+
+== iperf 3.1.4 2016-10-31 ==
+
+The release notes for iperf 3.1.4 describe changes, including bug
+fixes and new functionality, made since iperf 3.1.3.
+
+* User-visible changes
+
+  * On systems that support setting the congestion control algorithm,
+    iperf3 now keeps track of the congestion control algorithm and
+    print it in the JSON output in the members sender_tcp_congestion
+    and receiver_tcp_congestion (issue #461).  A few bugs (probably
+    not user-visible) with setting the congestion control algorithm
+    were also fixed.
+
+* Developer-visible changes
+
+  * Fixed a buffer overflow in the cJSON library (issue #466).  It is
+    not believed that this bug created any security vulnerabilities in
+    the context of iperf3.
+
+  * Travis CI builds are now enabled on this codeline (pull request #424).
+
+  * Various bug fixes (issue #459, pull request #429, issue #388).
+
+== iperf 3.1.3 2016-06-08 ==
+
+The release notes for iperf 3.1.3 describe changes, including bug
+fixes and new functionality, made since iperf 3.1.2.
+
+* Security
+
+  * Fixed a buffer overflow / heap corruption issue that could occur
+    if a malformed JSON string was passed on the control channel.  In
+    theory, this vulnerability could be leveraged to create a heap
+    exploit.  This issue, present in the cJSON library, was already
+    fixed upstream, so was addressed in iperf3 by importing a newer
+    version of cJSON (plus local ESnet modifications).  Discovered and
+    reported by Dave McDaniel, Cisco Talos.  Cross-references:
+    TALOS-CAN-0164, ESNET-SECADV-2016-0001, CVE-2016-4303.
+
+* User-visible changes
+
+  * On supported platforms (recent Linux), iperf3 can use
+    fair-queueing-based per-socket pacing instead of its own
+    application-level pacing for the --bandwidth option.
+    Application-level pacing can be forced with the
+    -no-fq-socket-pacing flag.
+
+  * A bug that could show negative loss counters with --udp and --omit
+    has been fixed (issue #412, pull request #414).
+
+  * Error handling has been made slightly more robust.  Also, the
+    iperf3 server will no longer exit after five consecutive errors,
+    but will only exit for certain types of errors that prevent it
+    from participating in any tests at all.
+
+* Developer-visible changes
+
+  * Fixed the build on FreeBSD 11-CURRENT (issue #413).
+
+  * Fixed various coding errors (issue #423, issue #425).
+
+== iperf 3.1.2 2016-02-01 ==
+
+The release notes for iperf 3.1.2 describe changes, including bug
+fixes and new functionality, made since iperf 3.1.1.
+
+* User-visible changes
+
+  * Fixed a bug that caused nan values to be emitted (incorrectly)
+    into JSON, particularly at the end of UDP tests (issue #278).
+
+  * Fixed a bug that caused the wrong value to be printed for
+    out-of-order UDP packets (issue #329).
+
+  * Added a contrib/ directory containing a few submitted graphing
+    scripts.
+
+* Developer-visible changes
+
+== iperf 3.1.1 2015-11-19 ==
+
+The release notes for iperf 3.1.1 describe changes and new
+functionality in iperf 3.1.1, but not present in 3.1.
+
+* User-visible changes
+
+  * Some markup fixes have been made in the manpages for Debian
+    compatibility (issue #291).
+
+  * A bug where the -T title option was not being output correctly
+    in JSON output has been fixed (issue #292).
+
+  * Argument handling for some command-line options has been improved
+    (issue #316).
+
+* Developer-visible changes
+
+  * A regression with C++ compatibility in one of the iperf header
+    files has been fixed (issue #323).
+
 == iperf 3.1 2015-10-16 ==
 
 The release notes for iperf 3.1 describe changes and new

configure

@@ -1,6 +1,6 @@
 #! /bin/sh
 # Guess values for system-dependent variables and create Makefiles.
-# Generated by GNU Autoconf 2.69 for iperf 3.1.
+# Generated by GNU Autoconf 2.69 for iperf 3.1.7.
 #
 # Report bugs to <https://github.com/esnet/iperf>.
 #
@@ -590,8 +590,8 @@ MAKEFLAGS=
 # Identity of this package.
 PACKAGE_NAME='iperf'
 PACKAGE_TARNAME='iperf'
-PACKAGE_VERSION='3.1'
-PACKAGE_STRING='iperf 3.1'
+PACKAGE_VERSION='3.1.7'
+PACKAGE_STRING='iperf 3.1.7'
 PACKAGE_BUGREPORT='https://github.com/esnet/iperf'
 PACKAGE_URL='http://software.es.net/iperf/'
 
@@ -1316,7 +1316,7 @@ if test "$ac_init_help" = "long"; then
   # Omit some internal or obsolete options to make the list less imposing.
   # This message is too long to be a string in the A/UX 3.1 sh.
   cat <<_ACEOF
-\`configure' configures iperf 3.1 to adapt to many kinds of systems.
+\`configure' configures iperf 3.1.7 to adapt to many kinds of systems.
 
 Usage: $0 [OPTION]... [VAR=VALUE]...
 
@@ -1386,7 +1386,7 @@ fi
 
 if test -n "$ac_init_help"; then
   case $ac_init_help in
-     short | recursive ) echo "Configuration of iperf 3.1:";;
+     short | recursive ) echo "Configuration of iperf 3.1.7:";;
    esac
   cat <<\_ACEOF
 
@@ -1500,7 +1500,7 @@ fi
 test -n "$ac_init_help" && exit $ac_status
 if $ac_init_version; then
   cat <<\_ACEOF
-iperf configure 3.1
+iperf configure 3.1.7
 generated by GNU Autoconf 2.69
 
 Copyright (C) 2012 Free Software Foundation, Inc.
@@ -1869,7 +1869,7 @@ cat >config.log <<_ACEOF
 This file contains any messages produced by compilers while
 running configure, to aid debugging if configure makes a mistake.
 
-It was created by iperf $as_me 3.1, which was
+It was created by iperf $as_me 3.1.7, which was
 generated by GNU Autoconf 2.69.  Invocation command line was
 
   $ $0 $@
@@ -2744,7 +2744,7 @@ fi
 
 # Define the identity of the package.
  PACKAGE='iperf'
- VERSION='3.1'
+ VERSION='3.1.7'
 
 
 cat >>confdefs.h <<_ACEOF
@@ -12185,6 +12185,69 @@ fi
 # Check for systems which need -lsocket and -lnsl
 #AX_LIB_SOCKET_NSL
 
+# Check for the math library (needed by cjson on some platforms)
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for library containing floor" >&5
+$as_echo_n "checking for library containing floor... " >&6; }
+if ${ac_cv_search_floor+:} false; then :
+  $as_echo_n "(cached) " >&6
+else
+  ac_func_search_save_LIBS=$LIBS
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h.  */
+
+/* Override any GCC internal prototype to avoid an error.
+   Use char because int might match the return type of a GCC
+   builtin and then its argument prototype would still apply.  */
+#ifdef __cplusplus
+extern "C"
+#endif
+char floor ();
+int
+main ()
+{
+return floor ();
+  ;
+  return 0;
+}
+_ACEOF
+for ac_lib in '' m; do
+  if test -z "$ac_lib"; then
+    ac_res="none required"
+  else
+    ac_res=-l$ac_lib
+    LIBS="-l$ac_lib  $ac_func_search_save_LIBS"
+  fi
+  if ac_fn_c_try_link "$LINENO"; then :
+  ac_cv_search_floor=$ac_res
+fi
+rm -f core conftest.err conftest.$ac_objext \
+    conftest$ac_exeext
+  if ${ac_cv_search_floor+:} false; then :
+  break
+fi
+done
+if ${ac_cv_search_floor+:} false; then :
+
+else
+  ac_cv_search_floor=no
+fi
+rm conftest.$ac_ext
+LIBS=$ac_func_search_save_LIBS
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_search_floor" >&5
+$as_echo "$ac_cv_search_floor" >&6; }
+ac_res=$ac_cv_search_floor
+if test "$ac_res" != no; then :
+  test "$ac_res" = "none required" || LIBS="$ac_res $LIBS"
+
+else
+
+echo "floor()"
+exit 1
+
+fi
+
+
 # Solaris puts nanosleep in -lrt
 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for library containing nanosleep" >&5
 $as_echo_n "checking for library containing nanosleep... " >&6; }
@@ -12248,69 +12311,6 @@ exit 1
 fi
 
 
-# Solaris puts hstrerror in -lresolv
-{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for library containing hstrerror" >&5
-$as_echo_n "checking for library containing hstrerror... " >&6; }
-if ${ac_cv_search_hstrerror+:} false; then :
-  $as_echo_n "(cached) " >&6
-else
-  ac_func_search_save_LIBS=$LIBS
-cat confdefs.h - <<_ACEOF >conftest.$ac_ext
-/* end confdefs.h.  */
-
-/* Override any GCC internal prototype to avoid an error.
-   Use char because int might match the return type of a GCC
-   builtin and then its argument prototype would still apply.  */
-#ifdef __cplusplus
-extern "C"
-#endif
-char hstrerror ();
-int
-main ()
-{
-return hstrerror ();
-  ;
-  return 0;
-}
-_ACEOF
-for ac_lib in '' resolv; do
-  if test -z "$ac_lib"; then
-    ac_res="none required"
-  else
-    ac_res=-l$ac_lib
-    LIBS="-l$ac_lib  $ac_func_search_save_LIBS"
-  fi
-  if ac_fn_c_try_link "$LINENO"; then :
-  ac_cv_search_hstrerror=$ac_res
-fi
-rm -f core conftest.err conftest.$ac_objext \
-    conftest$ac_exeext
-  if ${ac_cv_search_hstrerror+:} false; then :
-  break
-fi
-done
-if ${ac_cv_search_hstrerror+:} false; then :
-
-else
-  ac_cv_search_hstrerror=no
-fi
-rm conftest.$ac_ext
-LIBS=$ac_func_search_save_LIBS
-fi
-{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_search_hstrerror" >&5
-$as_echo "$ac_cv_search_hstrerror" >&6; }
-ac_res=$ac_cv_search_hstrerror
-if test "$ac_res" != no; then :
-  test "$ac_res" = "none required" || LIBS="$ac_res $LIBS"
-
-else
-
-echo "nanosleep() required for timing operations."
-exit 1
-
-fi
-
-
 # On illumos we need -lsocket
 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for library containing socket" >&5
 $as_echo_n "checking for library containing socket... " >&6; }
@@ -12774,6 +12774,38 @@ fi
 done
 
 
+# Check for packet pacing socket option (Linux only for now).
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking SO_MAX_PACING_RATE socket option" >&5
+$as_echo_n "checking SO_MAX_PACING_RATE socket option... " >&6; }
+if ${iperf3_cv_header_so_max_pacing_rate+:} false; then :
+  $as_echo_n "(cached) " >&6
+else
+  cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h.  */
+#include <sys/socket.h>
+#ifdef SO_MAX_PACING_RATE
+  yes
+#endif
+
+_ACEOF
+if (eval "$ac_cpp conftest.$ac_ext") 2>&5 |
+  $EGREP "yes" >/dev/null 2>&1; then :
+  iperf3_cv_header_so_max_pacing_rate=yes
+else
+  iperf3_cv_header_so_max_pacing_rate=no
+fi
+rm -f conftest*
+
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $iperf3_cv_header_so_max_pacing_rate" >&5
+$as_echo "$iperf3_cv_header_so_max_pacing_rate" >&6; }
+if test "x$iperf3_cv_header_so_max_pacing_rate" = "xyes"; then
+
+$as_echo "#define HAVE_SO_MAX_PACING_RATE 1" >>confdefs.h
+
+fi
+
+
 ac_config_files="$ac_config_files Makefile src/Makefile src/version.h examples/Makefile iperf3.spec"
 
 cat >confcache <<\_ACEOF
@@ -13309,7 +13341,7 @@ cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1
 # report actual input values of CONFIG_FILES etc. instead of their
 # values after options handling.
 ac_log="
-This file was extended by iperf $as_me 3.1, which was
+This file was extended by iperf $as_me 3.1.7, which was
 generated by GNU Autoconf 2.69.  Invocation command line was
 
   CONFIG_FILES    = $CONFIG_FILES
@@ -13376,7 +13408,7 @@ _ACEOF
 cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1
 ac_cs_config="`$as_echo "$ac_configure_args" | sed 's/^ //; s/[\\""\`\$]/\\\\&/g'`"
 ac_cs_version="\\
-iperf config.status 3.1
+iperf config.status 3.1.7
 configured by $0, generated by GNU Autoconf 2.69,
   with options \\"\$ac_cs_config\\"
 

configure.ac

@@ -1,4 +1,4 @@
-# iperf, Copyright (c) 2014, 2015, The Regents of the University of
+# iperf, Copyright (c) 2014, 2015, 2016, 2017, The Regents of the University of
 # California, through Lawrence Berkeley National Laboratory (subject
 # to receipt of any required approvals from the U.S. Dept. of
 # Energy).  All rights reserved.
@@ -24,7 +24,7 @@
 # file for complete information.
 
 # Initialize the autoconf system for the specified tool, version and mailing list
-AC_INIT(iperf, 3.1, https://github.com/esnet/iperf, iperf, http://software.es.net/iperf/)
+AC_INIT(iperf, 3.1.7, https://github.com/esnet/iperf, iperf, http://software.es.net/iperf/)
 AC_LANG(C)
 
 # Specify where the auxiliary files created by configure should go. The config
@@ -57,14 +57,14 @@ AC_HEADER_STDC
 # Check for systems which need -lsocket and -lnsl
 #AX_LIB_SOCKET_NSL
 
-# Solaris puts nanosleep in -lrt
-AC_SEARCH_LIBS(nanosleep, [rt], [], [
-echo "nanosleep() required for timing operations."
+# Check for the math library (needed by cjson on some platforms)
+AC_SEARCH_LIBS(floor, [m], [], [
+echo "floor()"
 exit 1
 ])
 
-# Solaris puts hstrerror in -lresolv
-AC_SEARCH_LIBS(hstrerror, [resolv], [], [
+# Solaris puts nanosleep in -lrt
+AC_SEARCH_LIBS(nanosleep, [rt], [], [
 echo "nanosleep() required for timing operations."
 exit 1
 ])
@@ -141,4 +141,18 @@ AC_CHECK_FUNCS([cpuset_setaffinity sched_setaffinity],
 # it needs and what arguments it expects.
 AC_CHECK_FUNCS([sendfile])
 
+# Check for packet pacing socket option (Linux only for now).
+AC_CACHE_CHECK([SO_MAX_PACING_RATE socket option],
+[iperf3_cv_header_so_max_pacing_rate],
+AC_EGREP_CPP(yes,
+[#include <sys/socket.h>
+#ifdef SO_MAX_PACING_RATE
+  yes
+#endif
+],iperf3_cv_header_so_max_pacing_rate=yes,iperf3_cv_header_so_max_pacing_rate=no))
+if test "x$iperf3_cv_header_so_max_pacing_rate" = "xyes"; then
+    AC_DEFINE([HAVE_SO_MAX_PACING_RATE], [1], [Have SO_MAX_PACING_RATE sockopt.])
+fi
+
+
 AC_OUTPUT([Makefile src/Makefile src/version.h examples/Makefile iperf3.spec])

contrib/README.txt

@@ -0,0 +1,12 @@
+
+This directory contains files that might be useful to analyze iperf3 results
+
+iperf3_to_gnuplot.py: converts iperf3 JSON output to format easy to plot in gnuplot
+
+iperf3.gp: sample gnuplot commands to plot throught and retransmits
+
+
+Other iperf3 related projects that might be of interest:
+
+https://github.com/kaihendry/iperf3chart
+

contrib/iperf3.gp

@@ -0,0 +1,37 @@
+#
+# sample Gnuplot command file for iperf3 results
+set term x11
+#set term png 
+#set term postscript landscape color
+set key width -12
+
+# iperf3 data fields
+#start bytes bits_per_second retransmits snd_cwnd
+
+set output "iperf3.png"
+#set output "iperf3.eps"
+
+#set nokey
+
+set grid xtics
+set grid ytics
+set grid linewidth 1
+set title "TCP performance: 40G to 10G host"
+set xlabel "time (seconds)"
+set ylabel "Bandwidth (Gbits/second)"
+set xrange [0:60] 
+set yrange [0:15] 
+set ytics nomirror
+set y2tics
+set y2range [0:2500] 
+# dont plot when retransmits = 0
+set datafile missing '0'
+set pointsize 1.6
+
+plot "40Gto10G.old.dat" using 1:3 title '3.10 kernel' with linespoints lw 3 pt 5, \
+	 "40Gto10G.new.dat" using 1:3 title '4.2 kernel' with linespoints lw 3 pt 7, \
+ 	 "40Gto10G.old.dat" using 1:4 title 'retransmits' with points pt 7 axes x1y2
+
+#plot "iperf3.old.dat" using 1:3 title '3.10 kernel' with linespoints lw 3 pt 5, \
+#	 "iperf3.new.dat" using 1:3 title '4.2 kernel' with linespoints lw 3 pt 7
+

contrib/iperf3.service

@@ -0,0 +1,10 @@
+[Unit]
+Description=iperf3
+Requires=network.target
+
+[Service]
+ExecStart=/usr/bin/iperf3 -s
+Restart=on-failure
+
+[Install]
+WantedBy=multi-user.target

contrib/iperf3_to_gnuplot.py

@@ -0,0 +1,77 @@
+#!/usr/bin/env python
+
+"""
+Extract iperf data from json blob and format for gnuplot.
+"""
+
+import json
+import os
+import sys
+
+import os.path
+from optparse import OptionParser
+
+import pprint
+# for debugging, so output to stderr to keep verbose
+# output out of any redirected stdout.
+pp = pprint.PrettyPrinter(indent=4, stream=sys.stderr)
+
+def generate_output(iperf, options):
+    for i in iperf.get('intervals'):
+        for ii in i.get('streams'):
+            if options.verbose: pp.pprint(ii)
+            row = '{0} {1} {2} {3} {4}\n'.format(
+                        round(float(ii.get('start')), 4),
+                        ii.get('bytes'),
+                        # to Gbits/sec
+                        round(float(ii.get('bits_per_second')) / (1000*1000*1000), 3),
+                        ii.get('retransmits'),
+                        round(float(ii.get('snd_cwnd')) / (1000*1000), 2)
+                    )
+            yield row
+
+def main():
+    usage = '%prog [ -f FILE | -o OUT | -v ]'
+    parser = OptionParser(usage=usage)
+    parser.add_option('-f', '--file', metavar='FILE',
+        type='string', dest='filename', 
+        help='Input filename.')
+    parser.add_option('-o', '--output', metavar='OUT',
+            type='string', dest='output', 
+            help='Optional file to append output to.')
+    parser.add_option('-v', '--verbose',
+        dest='verbose', action='store_true', default=False,
+        help='Verbose debug output to stderr.')
+    options, args = parser.parse_args()
+    
+    if not options.filename:
+        parser.error('Filename is required.')
+    
+    file_path = os.path.normpath(options.filename)
+    
+    if not os.path.exists(file_path):
+        parser.error('{f} does not exist'.format(f=file_path))
+
+    with open(file_path,'r') as fh:
+        data = fh.read()
+    
+    try:
+        iperf = json.loads(data)
+    except Exception as e:
+        parser.error('Could not parse JSON from file (ex): {0}'.format(str(e)))
+
+    if options.output:
+        absp = os.path.abspath(options.output)
+        d,f = os.path.split(absp)
+        if not os.path.exists(d):
+            parser.error('Output file directory path {0} does not exist'.format(d))
+        fh = open(absp, 'a')
+    else:
+        fh = sys.stdout
+
+    for i in generate_output(iperf, options):
+        fh.write(i)
+
+
+if __name__ == '__main__':
+    main()

docs/invoking.rst

@@ -82,6 +82,10 @@ the executable.
    ‐‐logfile file
    send output to a log file.
 
+   --forceflush
+   force flushing output at every interval. Used to avoid buffering when sending
+   output to pipe.
+
    ‐d, ‐‐debug
    emit  debugging  output.  Primarily (perhaps exclusively) of use
    to developers.

iperf3.spec.in

@@ -14,9 +14,10 @@ BuildRequires:  e2fsprogs-devel
 %endif
 
 %description
-Iperf is a tool to measure maximum TCP bandwidth, allowing the tuning of
-various parameters and UDP characteristics. Iperf reports bandwidth, delay
-jitter, data-gram loss.
+iperf3 is a tool for active measurements of the maximum achievable
+bandwidth between two IP hosts.  It supports tuning of various
+parameters related to timing, protocols, and buffers.  For each test,
+it reports the bandwidth, loss, and other parameters.
 
 %package        devel
 Summary:        Development files for %{name}

src/Makefile.am

@@ -25,6 +25,7 @@ libiperf_la_SOURCES     = \
 	                iperf_sctp.h \
                         iperf_util.c \
                         iperf_util.h \
+			dscp.c \
                         net.c \
                         net.h \
 			portable_endian.h \

src/Makefile.in

@@ -139,8 +139,8 @@ LTLIBRARIES = $(lib_LTLIBRARIES)
 libiperf_la_LIBADD =
 am_libiperf_la_OBJECTS = cjson.lo iperf_api.lo iperf_error.lo \
 	iperf_client_api.lo iperf_locale.lo iperf_server_api.lo \
-	iperf_tcp.lo iperf_udp.lo iperf_sctp.lo iperf_util.lo net.lo \
-	tcp_info.lo tcp_window_size.lo timer.lo units.lo
+	iperf_tcp.lo iperf_udp.lo iperf_sctp.lo iperf_util.lo dscp.lo \
+	net.lo tcp_info.lo tcp_window_size.lo timer.lo units.lo
 libiperf_la_OBJECTS = $(am_libiperf_la_OBJECTS)
 AM_V_lt = $(am__v_lt_@AM_V@)
 am__v_lt_ = $(am__v_lt_@AM_DEFAULT_V@)
@@ -163,7 +163,8 @@ am__objects_1 = iperf3_profile-cjson.$(OBJEXT) \
 	iperf3_profile-iperf_udp.$(OBJEXT) \
 	iperf3_profile-iperf_sctp.$(OBJEXT) \
 	iperf3_profile-iperf_util.$(OBJEXT) \
-	iperf3_profile-net.$(OBJEXT) iperf3_profile-tcp_info.$(OBJEXT) \
+	iperf3_profile-dscp.$(OBJEXT) iperf3_profile-net.$(OBJEXT) \
+	iperf3_profile-tcp_info.$(OBJEXT) \
 	iperf3_profile-tcp_window_size.$(OBJEXT) \
 	iperf3_profile-timer.$(OBJEXT) iperf3_profile-units.$(OBJEXT)
 am_iperf3_profile_OBJECTS = iperf3_profile-main.$(OBJEXT) \
@@ -585,6 +586,7 @@ libiperf_la_SOURCES = \
 	                iperf_sctp.h \
                         iperf_util.c \
                         iperf_util.h \
+			dscp.c \
                         net.c \
                         net.h \
 			portable_endian.h \
@@ -803,8 +805,10 @@ distclean-compile:
 	-rm -f *.tab.c
 
 @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/cjson.Plo@am__quote@
+@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/dscp.Plo@am__quote@
 @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/iperf3-main.Po@am__quote@
 @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/iperf3_profile-cjson.Po@am__quote@
+@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/iperf3_profile-dscp.Po@am__quote@
 @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/iperf3_profile-iperf_api.Po@am__quote@
 @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/iperf3_profile-iperf_client_api.Po@am__quote@
 @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/iperf3_profile-iperf_error.Po@am__quote@
@@ -1027,6 +1031,20 @@ iperf3_profile-iperf_util.obj: iperf_util.c
 @AMDEP_TRUE@@am__fastdepCC_FALSE@	DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
 @am__fastdepCC_FALSE@	$(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(AM_CPPFLAGS) $(CPPFLAGS) $(iperf3_profile_CFLAGS) $(CFLAGS) -c -o iperf3_profile-iperf_util.obj `if test -f 'iperf_util.c'; then $(CYGPATH_W) 'iperf_util.c'; else $(CYGPATH_W) '$(srcdir)/iperf_util.c'; fi`
 
+iperf3_profile-dscp.o: dscp.c
+@am__fastdepCC_TRUE@	$(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(AM_CPPFLAGS) $(CPPFLAGS) $(iperf3_profile_CFLAGS) $(CFLAGS) -MT iperf3_profile-dscp.o -MD -MP -MF $(DEPDIR)/iperf3_profile-dscp.Tpo -c -o iperf3_profile-dscp.o `test -f 'dscp.c' || echo '$(srcdir)/'`dscp.c
+@am__fastdepCC_TRUE@	$(AM_V_at)$(am__mv) $(DEPDIR)/iperf3_profile-dscp.Tpo $(DEPDIR)/iperf3_profile-dscp.Po
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	$(AM_V_CC)source='dscp.c' object='iperf3_profile-dscp.o' libtool=no @AMDEPBACKSLASH@
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
+@am__fastdepCC_FALSE@	$(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(AM_CPPFLAGS) $(CPPFLAGS) $(iperf3_profile_CFLAGS) $(CFLAGS) -c -o iperf3_profile-dscp.o `test -f 'dscp.c' || echo '$(srcdir)/'`dscp.c
+
+iperf3_profile-dscp.obj: dscp.c
+@am__fastdepCC_TRUE@	$(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(AM_CPPFLAGS) $(CPPFLAGS) $(iperf3_profile_CFLAGS) $(CFLAGS) -MT iperf3_profile-dscp.obj -MD -MP -MF $(DEPDIR)/iperf3_profile-dscp.Tpo -c -o iperf3_profile-dscp.obj `if test -f 'dscp.c'; then $(CYGPATH_W) 'dscp.c'; else $(CYGPATH_W) '$(srcdir)/dscp.c'; fi`
+@am__fastdepCC_TRUE@	$(AM_V_at)$(am__mv) $(DEPDIR)/iperf3_profile-dscp.Tpo $(DEPDIR)/iperf3_profile-dscp.Po
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	$(AM_V_CC)source='dscp.c' object='iperf3_profile-dscp.obj' libtool=no @AMDEPBACKSLASH@
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
+@am__fastdepCC_FALSE@	$(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(AM_CPPFLAGS) $(CPPFLAGS) $(iperf3_profile_CFLAGS) $(CFLAGS) -c -o iperf3_profile-dscp.obj `if test -f 'dscp.c'; then $(CYGPATH_W) 'dscp.c'; else $(CYGPATH_W) '$(srcdir)/dscp.c'; fi`
+
 iperf3_profile-net.o: net.c
 @am__fastdepCC_TRUE@	$(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(AM_CPPFLAGS) $(CPPFLAGS) $(iperf3_profile_CFLAGS) $(CFLAGS) -MT iperf3_profile-net.o -MD -MP -MF $(DEPDIR)/iperf3_profile-net.Tpo -c -o iperf3_profile-net.o `test -f 'net.c' || echo '$(srcdir)/'`net.c
 @am__fastdepCC_TRUE@	$(AM_V_at)$(am__mv) $(DEPDIR)/iperf3_profile-net.Tpo $(DEPDIR)/iperf3_profile-net.Po

src/cjson.h

@@ -19,107 +19,137 @@
   OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
   THE SOFTWARE.
 */
+#include "iperf_config.h"
 
 #ifndef cJSON__h
 #define cJSON__h
 
+#ifdef HAVE_STDINT_H
+#include <stdint.h>
+#endif
+
 #ifdef __cplusplus
 extern "C"
 {
 #endif
 
 /* cJSON Types: */
-#define cJSON_False 0
-#define cJSON_True 1
-#define cJSON_NULL 2
-#define cJSON_Number 3
-#define cJSON_String 4
-#define cJSON_Array 5
-#define cJSON_Object 6
+#define cJSON_False  (1 << 0)
+#define cJSON_True   (1 << 1)
+#define cJSON_NULL   (1 << 2)
+#define cJSON_Number (1 << 3)
+#define cJSON_String (1 << 4)
+#define cJSON_Array  (1 << 5)
+#define cJSON_Object (1 << 6)
 	
 #define cJSON_IsReference 256
+#define cJSON_StringIsConst 512
 
 /* The cJSON structure: */
 typedef struct cJSON {
-	struct cJSON *next, *prev;	/* next/prev allow you to walk array/object chains. Alternatively, use GetArraySize/GetArrayItem/GetObjectItem */
+	struct cJSON *next,*prev;	/* next/prev allow you to walk array/object chains. Alternatively, use GetArraySize/GetArrayItem/GetObjectItem */
 	struct cJSON *child;		/* An array or object item will have a child pointer pointing to a chain of the items in the array/object. */
 
-	int type;			/* The type of the item, as above. */
+	int type;					/* The type of the item, as above. */
 
-	char *valuestring;		/* The item's string, if type==cJSON_String */
-	int64_t valueint;		/* The item's number, if type==cJSON_Number */
-	double valuefloat;		/* The item's number, if type==cJSON_Number */
+	char *valuestring;			/* The item's string, if type==cJSON_String */
+	int64_t valueint;			/* The item's number, if type==cJSON_Number */
+	double valuedouble;			/* The item's number, if type==cJSON_Number */
 
-	char *string;			/* The item's name string, if this item is the child of, or is in the list of subitems of an object. */
+	char *string;				/* The item's name string, if this item is the child of, or is in the list of subitems of an object. */
 } cJSON;
 
 typedef struct cJSON_Hooks {
-      void *(*malloc_fn)(size_t sz );
-      void (*free_fn)( void *ptr );
+      void *(*malloc_fn)(size_t sz);
+      void (*free_fn)(void *ptr);
 } cJSON_Hooks;
 
 /* Supply malloc, realloc and free functions to cJSON */
-extern void cJSON_InitHooks( cJSON_Hooks* hooks );
+extern void cJSON_InitHooks(cJSON_Hooks* hooks);
 
 
 /* Supply a block of JSON, and this returns a cJSON object you can interrogate. Call cJSON_Delete when finished. */
-extern cJSON *cJSON_Parse( const char *value );
+extern cJSON *cJSON_Parse(const char *value);
 /* Render a cJSON entity to text for transfer/storage. Free the char* when finished. */
-extern char *cJSON_Print( cJSON *item );
+extern char  *cJSON_Print(cJSON *item);
 /* Render a cJSON entity to text for transfer/storage without any formatting. Free the char* when finished. */
-extern char *cJSON_PrintUnformatted( cJSON *item );
+extern char  *cJSON_PrintUnformatted(cJSON *item);
+/* Render a cJSON entity to text using a buffered strategy. prebuffer is a guess at the final size. guessing well reduces reallocation. fmt=0 gives unformatted, =1 gives formatted */
+extern char *cJSON_PrintBuffered(cJSON *item,int prebuffer,int fmt);
 /* Delete a cJSON entity and all subentities. */
-extern void cJSON_Delete( cJSON *c );
+extern void   cJSON_Delete(cJSON *c);
 
 /* Returns the number of items in an array (or object). */
-extern int cJSON_GetArraySize( cJSON *array );
+extern int	  cJSON_GetArraySize(cJSON *array);
 /* Retrieve item number "item" from array "array". Returns NULL if unsuccessful. */
-extern cJSON *cJSON_GetArrayItem( cJSON *array, int item );
+extern cJSON *cJSON_GetArrayItem(cJSON *array,int item);
 /* Get item "string" from object. Case insensitive. */
-extern cJSON *cJSON_GetObjectItem( cJSON *object, const char *string );
-
+extern cJSON *cJSON_GetObjectItem(cJSON *object,const char *string);
+extern int cJSON_HasObjectItem(cJSON *object,const char *string);
 /* For analysing failed parses. This returns a pointer to the parse error. You'll probably need to look a few chars back to make sense of it. Defined when cJSON_Parse() returns 0. 0 when cJSON_Parse() succeeds. */
-extern const char *cJSON_GetErrorPtr( void );
+extern const char *cJSON_GetErrorPtr(void);
 	
 /* These calls create a cJSON item of the appropriate type. */
-extern cJSON *cJSON_CreateNull( void );
-extern cJSON *cJSON_CreateTrue( void );
-extern cJSON *cJSON_CreateFalse( void );
-extern cJSON *cJSON_CreateBool( int b );
-extern cJSON *cJSON_CreateInt( int64_t num );
-extern cJSON *cJSON_CreateFloat( double num );
-extern cJSON *cJSON_CreateString( const char *string );
-extern cJSON *cJSON_CreateArray( void );
-extern cJSON *cJSON_CreateObject( void );
+extern cJSON *cJSON_CreateNull(void);
+extern cJSON *cJSON_CreateTrue(void);
+extern cJSON *cJSON_CreateFalse(void);
+extern cJSON *cJSON_CreateBool(int b);
+extern cJSON *cJSON_CreateNumber(double num);
+extern cJSON *cJSON_CreateString(const char *string);
+extern cJSON *cJSON_CreateArray(void);
+extern cJSON *cJSON_CreateObject(void);
 
 /* These utilities create an Array of count items. */
-extern cJSON *cJSON_CreateIntArray( int64_t *numbers, int count );
-extern cJSON *cJSON_CreateFloatArray( double *numbers, int count );
-extern cJSON *cJSON_CreateStringArray( const char **strings, int count );
+extern cJSON *cJSON_CreateIntArray(const int *numbers,int count);
+extern cJSON *cJSON_CreateFloatArray(const float *numbers,int count);
+extern cJSON *cJSON_CreateDoubleArray(const double *numbers,int count);
+extern cJSON *cJSON_CreateStringArray(const char **strings,int count);
 
 /* Append item to the specified array/object. */
-extern void cJSON_AddItemToArray( cJSON *array, cJSON *item );
-extern void cJSON_AddItemToObject( cJSON *object, const char *string, cJSON *item );
+extern void cJSON_AddItemToArray(cJSON *array, cJSON *item);
+extern void	cJSON_AddItemToObject(cJSON *object,const char *string,cJSON *item);
+extern void	cJSON_AddItemToObjectCS(cJSON *object,const char *string,cJSON *item);	/* Use this when string is definitely const (i.e. a literal, or as good as), and will definitely survive the cJSON object */
 /* Append reference to item to the specified array/object. Use this when you want to add an existing cJSON to a new cJSON, but don't want to corrupt your existing cJSON. */
-extern void cJSON_AddItemReferenceToArray( cJSON *array, cJSON *item );
-extern void cJSON_AddItemReferenceToObject( cJSON *object, const char *string, cJSON *item );
+extern void cJSON_AddItemReferenceToArray(cJSON *array, cJSON *item);
+extern void	cJSON_AddItemReferenceToObject(cJSON *object,const char *string,cJSON *item);
 
 /* Remove/Detatch items from Arrays/Objects. */
-extern cJSON *cJSON_DetachItemFromArray( cJSON *array, int which );
-extern void cJSON_DeleteItemFromArray( cJSON *array, int which );
-extern cJSON *cJSON_DetachItemFromObject( cJSON *object, const char *string );
-extern void cJSON_DeleteItemFromObject( cJSON *object, const char *string );
+extern cJSON *cJSON_DetachItemFromArray(cJSON *array,int which);
+extern void   cJSON_DeleteItemFromArray(cJSON *array,int which);
+extern cJSON *cJSON_DetachItemFromObject(cJSON *object,const char *string);
+extern void   cJSON_DeleteItemFromObject(cJSON *object,const char *string);
 	
 /* Update array items. */
-extern void cJSON_ReplaceItemInArray( cJSON *array, int which, cJSON *newitem );
-extern void cJSON_ReplaceItemInObject( cJSON *object, const char *string, cJSON *newitem );
+extern void cJSON_InsertItemInArray(cJSON *array,int which,cJSON *newitem);	/* Shifts pre-existing items to the right. */
+extern void cJSON_ReplaceItemInArray(cJSON *array,int which,cJSON *newitem);
+extern void cJSON_ReplaceItemInObject(cJSON *object,const char *string,cJSON *newitem);
 
-#define cJSON_AddNullToObject( object, name )		cJSON_AddItemToObject( object, name, cJSON_CreateNull() )
-#define cJSON_AddTrueToObject( object, name )		cJSON_AddItemToObject( object, name, cJSON_CreateTrue() )
-#define cJSON_AddFalseToObject( object, name )		cJSON_AddItemToObject( object, name, cJSON_CreateFalse() )
-#define cJSON_AddIntToObject( object, name, n )		cJSON_AddItemToObject( object, name, cJSON_CreateInt( n ) )
-#define cJSON_AddFloatToObject( object, name, n )	cJSON_AddItemToObject( object, name, cJSON_CreateFloat( n ) )
-#define cJSON_AddStringToObject( object, name, s )	cJSON_AddItemToObject( object, name, cJSON_CreateString( s ) )
+/* Duplicate a cJSON item */
+extern cJSON *cJSON_Duplicate(cJSON *item,int recurse);
+/* Duplicate will create a new, identical cJSON item to the one you pass, in new memory that will
+need to be released. With recurse!=0, it will duplicate any children connected to the item.
+The item->next and ->prev pointers are always zero on return from Duplicate. */
+
+/* ParseWithOpts allows you to require (and check) that the JSON is null terminated, and to retrieve the pointer to the final byte parsed. */
+/* If you supply a ptr in return_parse_end and parsing fails, then return_parse_end will contain a pointer to the error. If not, then cJSON_GetErrorPtr() does the job. */
+extern cJSON *cJSON_ParseWithOpts(const char *value,const char **return_parse_end,int require_null_terminated);
+
+extern void cJSON_Minify(char *json);
+
+/* Macros for creating things quickly. */
+#define cJSON_AddNullToObject(object,name)		cJSON_AddItemToObject(object, name, cJSON_CreateNull())
+#define cJSON_AddTrueToObject(object,name)		cJSON_AddItemToObject(object, name, cJSON_CreateTrue())
+#define cJSON_AddFalseToObject(object,name)		cJSON_AddItemToObject(object, name, cJSON_CreateFalse())
+#define cJSON_AddBoolToObject(object,name,b)	cJSON_AddItemToObject(object, name, cJSON_CreateBool(b))
+#define cJSON_AddNumberToObject(object,name,n)	cJSON_AddItemToObject(object, name, cJSON_CreateNumber(n))
+#define cJSON_AddStringToObject(object,name,s)	cJSON_AddItemToObject(object, name, cJSON_CreateString(s))
+
+/* When assigning an integer value, it needs to be propagated to valuedouble too. */
+#define cJSON_SetIntValue(object,val)			((object)?(object)->valueint=(object)->valuedouble=(val):(val))
+#define cJSON_SetNumberValue(object,val)		((object)?(object)->valueint=(object)->valuedouble=(val):(val))
+
+/* Macro for iterating over an array */
+#define cJSON_ArrayForEach(pos, head)			for(pos = (head)->child; pos != NULL; pos = pos->next)
 
 #ifdef __cplusplus
 }

src/dscp.c

@@ -0,0 +1,156 @@
+/* dscp lookup routines lifted wholesale from openssh */
+
+/*
+ * Copyright (c) 2000 Markus Friedl.  All rights reserved.
+ * Copyright (c) 2005,2006 Damien Miller.  All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
+ * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
+ * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
+ * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
+ * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
+ * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
+ * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <stdio.h>
+#include <string.h>
+#include <stdlib.h>
+#include <inttypes.h>
+
+#ifdef WIN32
+#define strcasecmp(a,b) _stricmp(a,b)
+#define snprintf _snprintf
+#endif
+
+int parse_qos(const char *cp);
+const char * iptos2str(int iptos);
+
+/*
+ * Definitions for IP type of service (ip_tos)
+ */
+
+#if HAVE_NETINET_IN_SYSTM_H
+#include <netinet/in_systm.h>
+#endif
+#if HAVE_NETINET_IP_H
+#include <netinet/ip.h>
+#endif
+
+#ifndef IPTOS_LOWDELAY
+# define IPTOS_LOWDELAY          0x10
+# define IPTOS_THROUGHPUT        0x08
+# define IPTOS_RELIABILITY       0x04
+# define IPTOS_LOWCOST           0x02
+# define IPTOS_MINCOST           IPTOS_LOWCOST
+#endif /* IPTOS_LOWDELAY */
+
+/*
+ * Definitions for DiffServ Codepoints as per RFC2474
+ */
+#ifndef IPTOS_DSCP_AF11
+# define	IPTOS_DSCP_AF11		0x28
+# define	IPTOS_DSCP_AF12		0x30
+# define	IPTOS_DSCP_AF13		0x38
+# define	IPTOS_DSCP_AF21		0x48
+# define	IPTOS_DSCP_AF22		0x50
+# define	IPTOS_DSCP_AF23		0x58
+# define	IPTOS_DSCP_AF31		0x68
+# define	IPTOS_DSCP_AF32		0x70
+# define	IPTOS_DSCP_AF33		0x78
+# define	IPTOS_DSCP_AF41		0x88
+# define	IPTOS_DSCP_AF42		0x90
+# define	IPTOS_DSCP_AF43		0x98
+# define	IPTOS_DSCP_EF		0xb8
+#endif /* IPTOS_DSCP_AF11 */
+
+#ifndef IPTOS_DSCP_CS0
+# define	IPTOS_DSCP_CS0		0x00
+# define	IPTOS_DSCP_CS1		0x20
+# define	IPTOS_DSCP_CS2		0x40
+# define	IPTOS_DSCP_CS3		0x60
+# define	IPTOS_DSCP_CS4		0x80
+# define	IPTOS_DSCP_CS5		0xa0
+# define	IPTOS_DSCP_CS6		0xc0
+# define	IPTOS_DSCP_CS7		0xe0
+#endif /* IPTOS_DSCP_CS0 */
+#ifndef IPTOS_DSCP_EF
+# define	IPTOS_DSCP_EF		0xb8
+#endif /* IPTOS_DSCP_EF */
+
+static const struct {
+	const char *name;
+	int value;
+} ipqos[] = {
+	{ "af11", IPTOS_DSCP_AF11 },
+	{ "af12", IPTOS_DSCP_AF12 },
+	{ "af13", IPTOS_DSCP_AF13 },
+	{ "af21", IPTOS_DSCP_AF21 },
+	{ "af22", IPTOS_DSCP_AF22 },
+	{ "af23", IPTOS_DSCP_AF23 },
+	{ "af31", IPTOS_DSCP_AF31 },
+	{ "af32", IPTOS_DSCP_AF32 },
+	{ "af33", IPTOS_DSCP_AF33 },
+	{ "af41", IPTOS_DSCP_AF41 },
+	{ "af42", IPTOS_DSCP_AF42 },
+	{ "af43", IPTOS_DSCP_AF43 },
+	{ "cs0", IPTOS_DSCP_CS0 },
+	{ "cs1", IPTOS_DSCP_CS1 },
+	{ "cs2", IPTOS_DSCP_CS2 },
+	{ "cs3", IPTOS_DSCP_CS3 },
+	{ "cs4", IPTOS_DSCP_CS4 },
+	{ "cs5", IPTOS_DSCP_CS5 },
+	{ "cs6", IPTOS_DSCP_CS6 },
+	{ "cs7", IPTOS_DSCP_CS7 },
+	{ "ef", IPTOS_DSCP_EF },
+	{ "lowdelay", IPTOS_LOWDELAY },
+	{ "throughput", IPTOS_THROUGHPUT },
+	{ "reliability", IPTOS_RELIABILITY },
+	{ NULL, -1 }
+};
+
+int
+parse_qos(const char *cp)
+{
+	unsigned int i;
+	char *ep = NULL;
+	long val;
+
+	if (cp == NULL)
+		return -1;
+	for (i = 0; ipqos[i].name != NULL; i++) {
+		if (strcasecmp(cp, ipqos[i].name) == 0)
+			return ipqos[i].value;
+	}
+	/* Try parsing as an integer */
+	val = strtol(cp, &ep, 0);
+	if (*cp == '\0' || *ep != '\0' || val < 0 || val > 255)
+		return -1;
+	return val;
+}
+
+const char *
+iptos2str(int iptos)
+{
+	int i;
+	static char iptos_str[sizeof "0xff"];
+	if (iptos < 0 || iptos > 64) iptos = 0;
+	for (i = 0; ipqos[i].name != NULL; i++) {
+		if (ipqos[i].value == iptos)
+			return ipqos[i].name;
+	}
+	snprintf(iptos_str, sizeof iptos_str, "0x%02x", iptos);
+	return iptos_str;
+}

src/iperf.h

@@ -1,5 +1,5 @@
 /*
- * iperf, Copyright (c) 2014, The Regents of the University of
+ * iperf, Copyright (c) 2014, 2015, 2016, 2017, The Regents of the University of
  * California, through Lawrence Berkeley National Laboratory (subject
  * to receipt of any required approvals from the U.S. Dept. of
  * Energy).  All rights reserved.
@@ -36,6 +36,7 @@
 #endif
 #include <sys/select.h>
 #include <sys/socket.h>
+#define _GNU_SOURCE
 #include <netinet/tcp.h>
 
 #if defined(HAVE_CPUSET_SETAFFINITY)
@@ -79,6 +80,7 @@ struct iperf_interval_results
     TAILQ_ENTRY(iperf_interval_results) irlistentries;
     void     *custom_data;
     int rtt;
+    int rttvar;
 };
 
 struct iperf_stream_result
@@ -87,6 +89,7 @@ struct iperf_stream_result
     iperf_size_t bytes_sent;
     iperf_size_t bytes_received_this_interval;
     iperf_size_t bytes_sent_this_interval;
+    iperf_size_t bytes_sent_omit;
     int stream_prev_total_retrans;
     int stream_retrans;
     int stream_prev_total_sacks;
@@ -98,6 +101,7 @@ struct iperf_stream_result
     int stream_max_snd_cwnd;
     struct timeval start_time;
     struct timeval end_time;
+    struct timeval start_time_fixed;
     TAILQ_HEAD(irlisthead, iperf_interval_results) interval_results;
     void     *data;
 };
@@ -108,7 +112,8 @@ struct iperf_settings
     int       domain;               /* AF_INET or AF_INET6 */
     int       socket_bufsize;       /* window size for TCP */
     int       blksize;              /* size of read/writes (-l) */
-    uint64_t  rate;                 /* target data rate */
+    uint64_t  rate;                 /* target data rate for application pacing*/
+    uint64_t  fqrate;               /* target data rate for FQ pacing*/
     int       burst;                /* packets per burst */
     int       mss;                  /* for TCP MSS */
     int       ttl;                  /* IP TTL option */
@@ -152,7 +157,9 @@ struct iperf_stream
     double    jitter;
     double    prev_transit;
     int       outoforder_packets;
+    int       omitted_outoforder_packets;
     int       cnt_error;
+    int       omitted_cnt_error;
     uint64_t  target;
 
     struct sockaddr_storage local_addr;
@@ -202,7 +209,7 @@ struct iperf_test
     struct protocol *protocol;
     signed char state;
     char     *server_hostname;                  /* -c option */
-    char     *template;                         /* -c option */
+    char     *tmp_template;
     char     *bind_address;                     /* first -B option */
     TAILQ_HEAD(xbind_addrhead, xbind_entry) xbind_addrs; /* all -X opts */
     int       bind_port;                        /* --cport option */
@@ -216,6 +223,8 @@ struct iperf_test
 #endif /* HAVE_CPUSET_SETAFFINITY */
     char     *title;				/* -T option */
     char     *congestion;			/* -C option */
+    char     *congestion_used;			/* what was actually used */
+    char     *remote_congestion_used;		/* what the other side used */
     char     *pidfile;				/* -P option */
 
     char     *logfile;				/* --logfile option */
@@ -225,6 +234,8 @@ struct iperf_test
     int       listener;
     int       prot_listener;
 
+    int	      ctrl_sck_mss;			/* MSS for the control channel */
+
     /* boolean variables for Options */
     int       daemon;                           /* -D option */
     int       one_off;                          /* -1 option */
@@ -236,7 +247,7 @@ struct iperf_test
     int       debug;				/* -d option - enable debug */
     int	      get_server_output;		/* --get-server-output */
     int	      udp_counters_64bit;		/* --use-64-bit-udp-counters */
-
+    int       forceflush; /* --forceflush - flushing output at every interval */
     int	      multisend;
 
     char     *json_output_string; /* rendered JSON output if json_output is set */
@@ -304,10 +315,14 @@ struct iperf_test
 #define SEC_TO_NS 1000000000LL	/* too big for enum/const on some platforms */
 #define MAX_RESULT_STRING 4096
 
+#define UDP_BUFFER_EXTRA 1024
+
 /* constants for command line arg sanity checks */
 #define MB (1024 * 1024)
 #define MAX_TCP_BUFFER (512 * MB)
 #define MAX_BLOCKSIZE MB
+/* Minimum size UDP send is the size of two 32-bit ints followed by a 64-bit int */
+#define MIN_UDP_BLOCKSIZE (4 + 4 + 8)
 /* Maximum size UDP send is (64K - 1) - IP and UDP header sizes */
 #define MAX_UDP_BLOCKSIZE (65535 - 8 - 20)
 #define MIN_INTERVAL 0.1

src/iperf3.1

@@ -1,4 +1,4 @@
-.TH IPERF 1 "October 2015" ESnet "User Manuals"
+.TH IPERF3 1 "April 2017" ESnet "User Manuals"
 .SH NAME
 iperf3 \- perform network throughput tests
 .SH SYNOPSIS
@@ -58,6 +58,10 @@ output in JSON format
 .BR --logfile " \fIfile\fR"
 send output to a log file.
 .TP
+.BR --forceflush " "
+force flushing output at every interval.
+Used to avoid buffering when sending output to pipe.
+.TP
 .BR -d ", " --debug " "
 emit debugging output.
 Primarily (perhaps exclusively) of use to developers.
@@ -85,7 +89,9 @@ handle one client connection, then exit.
 .SH "CLIENT SPECIFIC OPTIONS"
 .TP
 .BR -c ", " --client " \fIhost\fR"
-run in client mode, connecting to the specified server
+run in client mode, connecting to the specified server.
+By default, a test consists of sending data from the client to the
+server, unless the \-R flag is specified.
 .TP
 .BR --sctp
 use SCTP rather than TCP (FreeBSD and Linux)
@@ -95,7 +101,7 @@ use UDP rather than TCP
 .TP
 .BR -b ", " --bandwidth " \fIn\fR[KM]"
 set target bandwidth to \fIn\fR bits/sec (default 1 Mbit/sec for UDP, unlimited for TCP).
-If there are multiple streams (-P flag), the bandwidth limit is applied
+If there are multiple streams (\-P flag), the bandwidth limit is applied
 separately to each stream.
 You can also add a '/' and a number to the bandwidth specifier.
 This is called "burst mode".
@@ -103,18 +109,40 @@ It will send the given number of packets without pausing, even if that
 temporarily exceeds the specified bandwidth limit.
 Setting the target bandwidth to 0 will disable bandwidth limits
 (particularly useful for UDP tests).
+This bandwidth limit is implemented internally inside iperf3, and is
+available on all platforms.
+Compare with the \--fq-rate flag.
+.TP
+.BR --fq-rate " \fIn\fR[KM]"
+Set a rate to be used with fair-queueing based socket-level pacing,
+in bits per second.
+This pacing (if specified) will be in addition to any pacing due to
+iperf3's internal bandwidth pacing (\-b flag), and both can be
+specified for the same test.
+Only available on platforms supporting the
+\fCSO_MAX_PACING_RATE\fR socket option (currently only Linux).
+The default is no fair-queueing based pacing.
+.TP
+.BR --no-fq-socket-pacing
+This option is deprecated and will be removed.
+It is equivalent to specifying --fq-rate=0.
 .TP
 .BR -t ", " --time " \fIn\fR"
 time in seconds to transmit for (default 10 secs)
 .TP
 .BR -n ", " --bytes " \fIn\fR[KM]"
-number of bytes to transmit (instead of -t)
+number of bytes to transmit (instead of \-t)
 .TP
 .BR -k ", " --blockcount " \fIn\fR[KM]"
-number of blocks (packets) to transmit (instead of -t or -n)
+number of blocks (packets) to transmit (instead of \-t or \-n)
 .TP
 .BR -l ", " --length " \fIn\fR[KM]"
-length of buffer to read or write (default 128 KB for TCP, 8KB for UDP)
+length of buffer to read or write.  For TCP tests, the default value
+is 128KB.
+In the case of UDP, iperf3 tries to dynamically determine a reasonable
+sending size based on the path MTU; if that cannot be determined it
+uses 1460 bytes as a sending size.
+For SCTP tests, the default size is 64KB.
 .TP
 .BR --cport " \fIport\fR"
 bind data streams to a specific client port (for TCP and UDP only,
@@ -124,7 +152,8 @@ default is to use an ephemeral port)
 number of parallel client streams to run
 .TP
 .BR -R ", " --reverse
-run in reverse mode (server sends, client receives)
+reverse the direction of a test, so that the server sends data to the
+client
 .TP
 .BR -w ", " --window " \fIn\fR[KM]"
 window size / socket buffer size (this gets sent to the server and used on that side too)
@@ -142,7 +171,10 @@ only use IPv4
 only use IPv6
 .TP
 .BR -S ", " --tos " \fIn\fR"
-set the IP 'type of service'
+set the IP type of service
+.TP
+.BR "--dscp " \fIdscp\fR
+set the IP DSCP bits.  Both numeric and symbolic values are accepted.
 .TP
 .BR -L ", " --flowlabel " \fIn\fR"
 set the IPv6 flow label (currently only supported on Linux)

src/iperf_api.c

@@ -1,5 +1,5 @@
 /*
- * iperf, Copyright (c) 2014, 2015, The Regents of the University of
+ * iperf, Copyright (c) 2014-2017, The Regents of the University of
  * California, through Lawrence Berkeley National Laboratory (subject
  * to receipt of any required approvals from the U.S. Dept. of
  * Energy).  All rights reserved.
@@ -100,9 +100,9 @@ usage()
 
 
 void
-usage_long()
+usage_long(FILE *f)
 {
-    fprintf(stderr, usage_longstr, UDP_RATE / (1024*1024), DURATION, DEFAULT_TCP_BLKSIZE / 1024, DEFAULT_UDP_BLKSIZE / 1024);
+    fprintf(f, usage_longstr, UDP_RATE / (1024*1024), DURATION, DEFAULT_TCP_BLKSIZE / 1024, DEFAULT_UDP_BLKSIZE / 1024);
 }
 
 
@@ -126,6 +126,12 @@ iperf_get_control_socket(struct iperf_test *ipt)
     return ipt->ctrl_sck;
 }
 
+int
+iperf_get_control_socket_mss(struct iperf_test *ipt)
+{
+    return ipt->ctrl_sck_mss;
+}
+
 int
 iperf_get_test_omit(struct iperf_test *ipt)
 {
@@ -144,6 +150,12 @@ iperf_get_test_rate(struct iperf_test *ipt)
     return ipt->settings->rate;
 }
 
+uint64_t
+iperf_get_test_fqrate(struct iperf_test *ipt)
+{
+    return ipt->settings->fqrate;
+}
+
 int
 iperf_get_test_burst(struct iperf_test *ipt)
 {
@@ -213,7 +225,7 @@ iperf_get_test_server_hostname(struct iperf_test *ipt)
 char*
 iperf_get_test_template(struct iperf_test *ipt)
 {
-    return ipt->template;
+    return ipt->tmp_template;
 }
 
 int
@@ -326,6 +338,12 @@ iperf_set_test_rate(struct iperf_test *ipt, uint64_t rate)
     ipt->settings->rate = rate;
 }
 
+void
+iperf_set_test_fqrate(struct iperf_test *ipt, uint64_t fqrate)
+{
+    ipt->settings->fqrate = fqrate;
+}
+
 void
 iperf_set_test_burst(struct iperf_test *ipt, int burst)
 {
@@ -333,9 +351,9 @@ iperf_set_test_burst(struct iperf_test *ipt, int burst)
 }
 
 void
-iperf_set_test_server_port(struct iperf_test *ipt, int server_port)
+iperf_set_test_server_port(struct iperf_test *ipt, int srv_port)
 {
-    ipt->server_port = server_port;
+    ipt->server_port = srv_port;
 }
 
 void
@@ -379,9 +397,9 @@ iperf_set_test_server_hostname(struct iperf_test *ipt, char *server_hostname)
 }
 
 void
-iperf_set_test_template(struct iperf_test *ipt, char *template)
+iperf_set_test_template(struct iperf_test *ipt, char *tmp_template)
 {
-    ipt->template = strdup(template);
+    ipt->tmp_template = strdup(tmp_template);
 }
 
 void
@@ -424,9 +442,9 @@ iperf_set_test_unit_format(struct iperf_test *ipt, char unit_format)
 }
 
 void
-iperf_set_test_bind_address(struct iperf_test *ipt, char *bind_address)
+iperf_set_test_bind_address(struct iperf_test *ipt, char *bnd_address)
 {
-    ipt->bind_address = strdup(bind_address);
+    ipt->bind_address = strdup(bnd_address);
 }
 
 void
@@ -532,7 +550,6 @@ iperf_on_connect(struct iperf_test *test)
     struct sockaddr_in *sa_inP;
     struct sockaddr_in6 *sa_in6P;
     socklen_t len;
-    int opt;
 
     now_secs = time((time_t*) 0);
     (void) strftime(now_str, sizeof(now_str), rfc1123_fmt, gmtime(&now_secs));
@@ -571,11 +588,9 @@ iperf_on_connect(struct iperf_test *test)
 	cJSON_AddStringToObject(test->json_start, "cookie", test->cookie);
         if (test->protocol->id == SOCK_STREAM) {
 	    if (test->settings->mss)
-		cJSON_AddIntToObject(test->json_start, "tcp_mss", test->settings->mss);
+		cJSON_AddNumberToObject(test->json_start, "tcp_mss", test->settings->mss);
 	    else {
-		len = sizeof(opt);
-		getsockopt(test->ctrl_sck, IPPROTO_TCP, TCP_MAXSEG, &opt, &len);
-		cJSON_AddIntToObject(test->json_start, "tcp_mss_default", opt);
+		cJSON_AddNumberToObject(test->json_start, "tcp_mss_default", test->ctrl_sck_mss);
 	    }
 	}
     } else if (test->verbose) {
@@ -584,9 +599,7 @@ iperf_on_connect(struct iperf_test *test)
             if (test->settings->mss)
                 iprintf(test, "      TCP MSS: %d\n", test->settings->mss);
             else {
-                len = sizeof(opt);
-                getsockopt(test->ctrl_sck, IPPROTO_TCP, TCP_MAXSEG, &opt, &len);
-                iprintf(test, "      TCP MSS: %d (default)\n", opt);
+                iprintf(test, "      TCP MSS: %d (default)\n", test->ctrl_sck_mss);
             }
         }
 
@@ -632,6 +645,7 @@ iperf_parse_arguments(struct iperf_test *test, int argc, char **argv)
         {"version4", no_argument, NULL, '4'},
         {"version6", no_argument, NULL, '6'},
         {"tos", required_argument, NULL, 'S'},
+        {"dscp", required_argument, NULL, OPT_DSCP},
 #if defined(HAVE_FLOWLABEL)
         {"flowlabel", required_argument, NULL, 'L'},
 #endif /* HAVE_FLOWLABEL */
@@ -653,8 +667,11 @@ iperf_parse_arguments(struct iperf_test *test, int argc, char **argv)
 #endif
 	{"pidfile", required_argument, NULL, 'I'},
 	{"logfile", required_argument, NULL, OPT_LOGFILE},
+	{"forceflush", no_argument, NULL, OPT_FORCEFLUSH},
 	{"get-server-output", no_argument, NULL, OPT_GET_SERVER_OUTPUT},
 	{"udp-counters-64bit", no_argument, NULL, OPT_UDP_COUNTERS_64BIT},
+ 	{"no-fq-socket-pacing", no_argument, NULL, OPT_NO_FQ_SOCKET_PACING},
+	{"fq-rate", required_argument, NULL, OPT_FQ_RATE},
         {"debug", no_argument, NULL, 'd'},
         {"help", no_argument, NULL, 'h'},
         {NULL, 0, NULL, 0}
@@ -662,6 +679,7 @@ iperf_parse_arguments(struct iperf_test *test, int argc, char **argv)
     int flag;
     int blksize;
     int server_flag, client_flag, rate_flag, duration_flag;
+    char *endptr;
 #if defined(HAVE_CPU_AFFINITY)
     char* comma;
 #endif /* HAVE_CPU_AFFINITY */
@@ -728,11 +746,11 @@ iperf_parse_arguments(struct iperf_test *test, int argc, char **argv)
 #if defined(HAVE_SCTP)
                 set_protocol(test, Psctp);
                 client_flag = 1;
+                break;
 #else /* HAVE_SCTP */
                 i_errno = IEUNIMP;
                 return -1;
 #endif /* HAVE_SCTP */
-            break;
 
             case OPT_NUMSTREAMS:
 #if defined(linux) || defined(__FreeBSD__)
@@ -825,13 +843,28 @@ iperf_parse_arguments(struct iperf_test *test, int argc, char **argv)
                 test->settings->domain = AF_INET6;
                 break;
             case 'S':
-                test->settings->tos = strtol(optarg, NULL, 0);
+                test->settings->tos = strtol(optarg, &endptr, 0);
+		if (endptr == optarg ||
+		    test->settings->tos < 0 ||
+		    test->settings->tos > 255) {
+		    i_errno = IEBADTOS;
+		    return -1;
+		}
+		client_flag = 1;
+                break;
+	    case OPT_DSCP:
+                test->settings->tos = parse_qos(optarg);
+		if(test->settings->tos < 0) {
+			i_errno = IEBADTOS;
+			return -1;
+		}
 		client_flag = 1;
                 break;
             case 'L':
 #if defined(HAVE_FLOWLABEL)
-                test->settings->flowlabel = strtol(optarg, NULL, 0);
-		if (test->settings->flowlabel < 1 || test->settings->flowlabel > 0xfffff) {
+                test->settings->flowlabel = strtol(optarg, &endptr, 0);
+		if (endptr == optarg ||
+		    test->settings->flowlabel < 1 || test->settings->flowlabel > 0xfffff) {
                     i_errno = IESETFLOW;
                     return -1;
 		}
@@ -876,8 +909,9 @@ iperf_parse_arguments(struct iperf_test *test, int argc, char **argv)
                 break;
             case 'A':
 #if defined(HAVE_CPU_AFFINITY)
-                test->affinity = atoi(optarg);
-                if (test->affinity < 0 || test->affinity > 1024) {
+                test->affinity = strtol(optarg, &endptr, 0);
+                if (endptr == optarg || 
+		    test->affinity < 0 || test->affinity > 1024) {
                     i_errno = IEAFFINITY;
                     return -1;
                 }
@@ -918,6 +952,9 @@ iperf_parse_arguments(struct iperf_test *test, int argc, char **argv)
 	    case OPT_LOGFILE:
 		test->logfile = strdup(optarg);
 		break;
+	    case OPT_FORCEFLUSH:
+		test->forceflush = 1;
+		break;
 	    case OPT_GET_SERVER_OUTPUT:
 		test->get_server_output = 1;
 		client_flag = 1;
@@ -925,9 +962,30 @@ iperf_parse_arguments(struct iperf_test *test, int argc, char **argv)
 	    case OPT_UDP_COUNTERS_64BIT:
 		test->udp_counters_64bit = 1;
 		break;
+	    case OPT_NO_FQ_SOCKET_PACING:
+#if defined(HAVE_SO_MAX_PACING_RATE)
+		printf("Warning:  --no-fq-socket-pacing is deprecated\n");
+		test->settings->fqrate = 0;
+		client_flag = 1;
+#else /* HAVE_SO_MAX_PACING_RATE */
+		i_errno = IEUNIMP;
+		return -1;
+#endif
+		break;
+	    case OPT_FQ_RATE:
+#if defined(HAVE_SO_MAX_PACING_RATE)
+		test->settings->fqrate = unit_atof_rate(optarg);
+		client_flag = 1;
+#else /* HAVE_SO_MAX_PACING_RATE */
+		i_errno = IEUNIMP;
+		return -1;
+#endif
+		break;
             case 'h':
+		usage_long(stdout);
+		exit(0);
             default:
-                usage_long();
+                usage_long(stderr);
                 exit(1);
         }
     }
@@ -957,18 +1015,20 @@ iperf_parse_arguments(struct iperf_test *test, int argc, char **argv)
     }
     if (blksize == 0) {
 	if (test->protocol->id == Pudp)
-	    blksize = DEFAULT_UDP_BLKSIZE;
+	    blksize = 0;	/* try to dynamically determine from MSS */
 	else if (test->protocol->id == Psctp)
 	    blksize = DEFAULT_SCTP_BLKSIZE;
 	else
 	    blksize = DEFAULT_TCP_BLKSIZE;
     }
-    if (blksize <= 0 || blksize > MAX_BLOCKSIZE) {
+    if ((test->protocol->id != Pudp && blksize <= 0) 
+	|| blksize > MAX_BLOCKSIZE) {
 	i_errno = IEBLOCKSIZE;
 	return -1;
     }
     if (test->protocol->id == Pudp &&
-	blksize > MAX_UDP_BLOCKSIZE) {
+	(blksize > 0 &&
+	    (blksize < MIN_UDP_BLOCKSIZE || blksize > MAX_UDP_BLOCKSIZE))) {
 	i_errno = IEUDPBLOCKSIZE;
 	return -1;
     }
@@ -1025,7 +1085,7 @@ iperf_check_throttle(struct iperf_stream *sp, struct timeval *nowP)
 
     if (sp->test->done)
         return;
-    seconds = timeval_diff(&sp->result->start_time, nowP);
+    seconds = timeval_diff(&sp->result->start_time_fixed, nowP);
     bits_per_second = sp->result->bytes_sent * 8 / seconds;
     if (bits_per_second < sp->test->settings->rate) {
         sp->green_light = 1;
@@ -1056,8 +1116,8 @@ iperf_send(struct iperf_test *test, fd_set *write_setP)
 	    gettimeofday(&now, NULL);
 	streams_active = 0;
 	SLIST_FOREACH(sp, &test->streams, streams) {
-	    if (sp->green_light &&
-	        (write_setP == NULL || FD_ISSET(sp->socket, write_setP))) {
+	    if ((sp->green_light &&
+		 (write_setP == NULL || FD_ISSET(sp->socket, write_setP)))) {
 		if ((r = sp->snd(sp)) < 0) {
 		    if (r == NET_SOFTERROR)
 			break;
@@ -1129,7 +1189,7 @@ iperf_init_test(struct iperf_test *test)
 	return -1;
     }
     SLIST_FOREACH(sp, &test->streams, streams) {
-	sp->result->start_time = now;
+	sp->result->start_time = sp->result->start_time_fixed = now;
     }
 
     if (test->on_test_start)
@@ -1267,42 +1327,46 @@ send_parameters(struct iperf_test *test)
 	    cJSON_AddTrueToObject(j, "udp");
         else if (test->protocol->id == Psctp)
             cJSON_AddTrueToObject(j, "sctp");
-	cJSON_AddIntToObject(j, "omit", test->omit);
+	cJSON_AddNumberToObject(j, "omit", test->omit);
 	if (test->server_affinity != -1)
-	    cJSON_AddIntToObject(j, "server_affinity", test->server_affinity);
+	    cJSON_AddNumberToObject(j, "server_affinity", test->server_affinity);
 	if (test->duration)
-	    cJSON_AddIntToObject(j, "time", test->duration);
+	    cJSON_AddNumberToObject(j, "time", test->duration);
 	if (test->settings->bytes)
-	    cJSON_AddIntToObject(j, "num", test->settings->bytes);
+	    cJSON_AddNumberToObject(j, "num", test->settings->bytes);
 	if (test->settings->blocks)
-	    cJSON_AddIntToObject(j, "blockcount", test->settings->blocks);
+	    cJSON_AddNumberToObject(j, "blockcount", test->settings->blocks);
 	if (test->settings->mss)
-	    cJSON_AddIntToObject(j, "MSS", test->settings->mss);
+	    cJSON_AddNumberToObject(j, "MSS", test->settings->mss);
 	if (test->no_delay)
 	    cJSON_AddTrueToObject(j, "nodelay");
-	cJSON_AddIntToObject(j, "parallel", test->num_streams);
+	cJSON_AddNumberToObject(j, "parallel", test->num_streams);
 	if (test->reverse)
 	    cJSON_AddTrueToObject(j, "reverse");
 	if (test->settings->socket_bufsize)
-	    cJSON_AddIntToObject(j, "window", test->settings->socket_bufsize);
+	    cJSON_AddNumberToObject(j, "window", test->settings->socket_bufsize);
 	if (test->settings->blksize)
-	    cJSON_AddIntToObject(j, "len", test->settings->blksize);
+	    cJSON_AddNumberToObject(j, "len", test->settings->blksize);
 	if (test->settings->rate)
-	    cJSON_AddIntToObject(j, "bandwidth", test->settings->rate);
+	    cJSON_AddNumberToObject(j, "bandwidth", test->settings->rate);
+	if (test->settings->fqrate)
+	    cJSON_AddNumberToObject(j, "fqrate", test->settings->fqrate);
 	if (test->settings->burst)
-	    cJSON_AddIntToObject(j, "burst", test->settings->burst);
+	    cJSON_AddNumberToObject(j, "burst", test->settings->burst);
 	if (test->settings->tos)
-	    cJSON_AddIntToObject(j, "TOS", test->settings->tos);
+	    cJSON_AddNumberToObject(j, "TOS", test->settings->tos);
 	if (test->settings->flowlabel)
-	    cJSON_AddIntToObject(j, "flowlabel", test->settings->flowlabel);
+	    cJSON_AddNumberToObject(j, "flowlabel", test->settings->flowlabel);
 	if (test->title)
 	    cJSON_AddStringToObject(j, "title", test->title);
 	if (test->congestion)
 	    cJSON_AddStringToObject(j, "congestion", test->congestion);
+	if (test->congestion_used)
+	    cJSON_AddStringToObject(j, "congestion_used", test->congestion_used);
 	if (test->get_server_output)
-	    cJSON_AddIntToObject(j, "get_server_output", iperf_get_test_get_server_output(test));
+	    cJSON_AddNumberToObject(j, "get_server_output", iperf_get_test_get_server_output(test));
 	if (test->udp_counters_64bit)
-	    cJSON_AddIntToObject(j, "udp_counters_64bit", iperf_get_test_udp_counters_64bit(test));
+	    cJSON_AddNumberToObject(j, "udp_counters_64bit", iperf_get_test_udp_counters_64bit(test));
 
 	cJSON_AddStringToObject(j, "client_version", IPERF_VERSION);
 
@@ -1367,6 +1431,8 @@ get_parameters(struct iperf_test *test)
 	    test->settings->blksize = j_p->valueint;
 	if ((j_p = cJSON_GetObjectItem(j, "bandwidth")) != NULL)
 	    test->settings->rate = j_p->valueint;
+	if ((j_p = cJSON_GetObjectItem(j, "fqrate")) != NULL)
+	    test->settings->fqrate = j_p->valueint;
 	if ((j_p = cJSON_GetObjectItem(j, "burst")) != NULL)
 	    test->settings->burst = j_p->valueint;
 	if ((j_p = cJSON_GetObjectItem(j, "TOS")) != NULL)
@@ -1377,10 +1443,13 @@ get_parameters(struct iperf_test *test)
 	    test->title = strdup(j_p->valuestring);
 	if ((j_p = cJSON_GetObjectItem(j, "congestion")) != NULL)
 	    test->congestion = strdup(j_p->valuestring);
+	if ((j_p = cJSON_GetObjectItem(j, "congestion_used")) != NULL)
+	    test->congestion_used = strdup(j_p->valuestring);
 	if ((j_p = cJSON_GetObjectItem(j, "get_server_output")) != NULL)
 	    iperf_set_test_get_server_output(test, 1);
 	if ((j_p = cJSON_GetObjectItem(j, "udp_counters_64bit")) != NULL)
 	    iperf_set_test_udp_counters_64bit(test, 1);
+	
 	if (test->sender && test->protocol->id == Ptcp && has_tcpinfo_retransmits())
 	    test->sender_has_retransmits = 1;
 	cJSON_Delete(j);
@@ -1407,14 +1476,17 @@ send_results(struct iperf_test *test)
 	i_errno = IEPACKAGERESULTS;
 	r = -1;
     } else {
-	cJSON_AddFloatToObject(j, "cpu_util_total", test->cpu_util[0]);
-	cJSON_AddFloatToObject(j, "cpu_util_user", test->cpu_util[1]);
-	cJSON_AddFloatToObject(j, "cpu_util_system", test->cpu_util[2]);
+	cJSON_AddNumberToObject(j, "cpu_util_total", test->cpu_util[0]);
+	cJSON_AddNumberToObject(j, "cpu_util_user", test->cpu_util[1]);
+	cJSON_AddNumberToObject(j, "cpu_util_system", test->cpu_util[2]);
 	if ( ! test->sender )
 	    sender_has_retransmits = -1;
 	else
 	    sender_has_retransmits = test->sender_has_retransmits;
-	cJSON_AddIntToObject(j, "sender_has_retransmits", sender_has_retransmits);
+	cJSON_AddNumberToObject(j, "sender_has_retransmits", sender_has_retransmits);
+	if ( test->congestion_used ) {
+	    cJSON_AddStringToObject(j, "congestion_used", test->congestion_used);
+	}
 
 	/* If on the server and sending server output, then do this */
 	if (test->role == 's' && test->get_server_output) {
@@ -1440,6 +1512,7 @@ send_results(struct iperf_test *test)
 		}
 
 		cJSON_AddStringToObject(j, "server_output_text", output);
+        free(output);
 	    }
 	}
 
@@ -1456,14 +1529,14 @@ send_results(struct iperf_test *test)
 		    r = -1;
 		} else {
 		    cJSON_AddItemToArray(j_streams, j_stream);
-		    bytes_transferred = test->sender ? sp->result->bytes_sent : sp->result->bytes_received;
+		    bytes_transferred = test->sender ? (sp->result->bytes_sent - sp->result->bytes_sent_omit) : sp->result->bytes_received;
 		    retransmits = (test->sender && test->sender_has_retransmits) ? sp->result->stream_retrans : -1;
-		    cJSON_AddIntToObject(j_stream, "id", sp->id);
-		    cJSON_AddIntToObject(j_stream, "bytes", bytes_transferred);
-		    cJSON_AddIntToObject(j_stream, "retransmits", retransmits);
-		    cJSON_AddFloatToObject(j_stream, "jitter", sp->jitter);
-		    cJSON_AddIntToObject(j_stream, "errors", sp->cnt_error);
-		    cJSON_AddIntToObject(j_stream, "packets", sp->packet_count);
+		    cJSON_AddNumberToObject(j_stream, "id", sp->id);
+		    cJSON_AddNumberToObject(j_stream, "bytes", bytes_transferred);
+		    cJSON_AddNumberToObject(j_stream, "retransmits", retransmits);
+		    cJSON_AddNumberToObject(j_stream, "jitter", sp->jitter);
+		    cJSON_AddNumberToObject(j_stream, "errors", sp->cnt_error);
+		    cJSON_AddNumberToObject(j_stream, "packets", sp->packet_count);
 		}
 	    }
 	    if (r == 0 && test->debug) {
@@ -1489,6 +1562,7 @@ get_results(struct iperf_test *test)
     cJSON *j_cpu_util_total;
     cJSON *j_cpu_util_user;
     cJSON *j_cpu_util_system;
+    cJSON *j_remote_congestion_used;
     cJSON *j_sender_has_retransmits;
     int result_has_retransmits;
     cJSON *j_streams;
@@ -1524,9 +1598,9 @@ get_results(struct iperf_test *test)
 		printf("get_results\n%s\n", cJSON_Print(j));
 	    }
 
-	    test->remote_cpu_util[0] = j_cpu_util_total->valuefloat;
-	    test->remote_cpu_util[1] = j_cpu_util_user->valuefloat;
-	    test->remote_cpu_util[2] = j_cpu_util_system->valuefloat;
+	    test->remote_cpu_util[0] = j_cpu_util_total->valuedouble;
+	    test->remote_cpu_util[1] = j_cpu_util_user->valuedouble;
+	    test->remote_cpu_util[2] = j_cpu_util_system->valuedouble;
 	    result_has_retransmits = j_sender_has_retransmits->valueint;
 	    if (! test->sender)
 		test->sender_has_retransmits = result_has_retransmits;
@@ -1555,7 +1629,7 @@ get_results(struct iperf_test *test)
 			    sid = j_id->valueint;
 			    bytes_transferred = j_bytes->valueint;
 			    retransmits = j_retransmits->valueint;
-			    jitter = j_jitter->valuefloat;
+			    jitter = j_jitter->valuedouble;
 			    cerror = j_errors->valueint;
 			    pcount = j_packets->valueint;
 			    SLIST_FOREACH(sp, &test->streams, streams)
@@ -1597,6 +1671,12 @@ get_results(struct iperf_test *test)
 		}
 	    }
 	}
+
+	j_remote_congestion_used = cJSON_GetObjectItem(j, "congestion_used");
+	if (j_remote_congestion_used != NULL) {
+	    test->remote_congestion_used = strdup(j_remote_congestion_used->valuestring);
+	}
+
 	cJSON_Delete(j);
     }
     return r;
@@ -1636,14 +1716,32 @@ JSON_read(int fd)
     uint32_t hsize, nsize;
     char *str;
     cJSON *json = NULL;
+    int rc;
 
+    /*
+     * Read a four-byte integer, which is the length of the JSON to follow.
+     * Then read the JSON into a buffer and parse it.  Return a parsed JSON
+     * structure, NULL if there was an error.
+     */
     if (Nread(fd, (char*) &nsize, sizeof(nsize), Ptcp) >= 0) {
 	hsize = ntohl(nsize);
-	str = (char *) malloc(hsize+1);	/* +1 for EOS */
+	/* Allocate a buffer to hold the JSON */
+	str = (char *) calloc(sizeof(char), hsize+1);	/* +1 for trailing null */
 	if (str != NULL) {
-	    if (Nread(fd, str, hsize, Ptcp) >= 0) {
-		str[hsize] = '\0';	/* add the EOS */
-		json = cJSON_Parse(str);
+	    rc = Nread(fd, str, hsize, Ptcp);
+	    if (rc >= 0) {
+		/*
+		 * We should be reading in the number of bytes corresponding to the
+		 * length in that 4-byte integer.  If we don't the socket might have
+		 * prematurely closed.  Only do the JSON parsing if we got the
+		 * correct number of bytes.
+		 */
+		if (rc == hsize) {
+		    json = cJSON_Parse(str);
+		}
+		else {
+		    printf("WARNING:  Size of data read does not correspond to offered length\n");
+		}
 	    }
 	}
 	free(str);
@@ -1775,6 +1873,8 @@ iperf_defaults(struct iperf_test *testp)
 #endif /* HAVE_CPUSET_SETAFFINITY */
     testp->title = NULL;
     testp->congestion = NULL;
+    testp->congestion_used = NULL;
+    testp->remote_congestion_used = NULL;
     testp->server_port = PORT;
     testp->ctrl_sck = -1;
     testp->prot_listener = -1;
@@ -1790,6 +1890,7 @@ iperf_defaults(struct iperf_test *testp)
     testp->settings->socket_bufsize = 0;    /* use autotuning */
     testp->settings->blksize = DEFAULT_TCP_BLKSIZE;
     testp->settings->rate = 0;
+    testp->settings->fqrate = 0;
     testp->settings->burst = 0;
     testp->settings->mss = 0;
     testp->settings->bytes = 0;
@@ -1881,8 +1982,8 @@ iperf_free_test(struct iperf_test *test)
 
     if (test->server_hostname)
 	free(test->server_hostname);
-    if (test->template)
-    free(test->template);
+    if (test->tmp_template)
+	free(test->tmp_template);
     if (test->bind_address)
 	free(test->bind_address);
     if (!TAILQ_EMPTY(&test->xbind_addrs)) {
@@ -1903,6 +2004,10 @@ iperf_free_test(struct iperf_test *test)
 	free(test->title);
     if (test->congestion)
 	free(test->congestion);
+    if (test->congestion_used)
+	free(test->congestion_used);
+    if (test->remote_congestion_used)
+	free(test->remote_congestion_used);
     if (test->omit_timer != NULL)
 	tmr_cancel(test->omit_timer);
     if (test->timer != NULL)
@@ -2022,6 +2127,10 @@ iperf_reset_test(struct iperf_test *test)
     memset(test->cookie, 0, COOKIE_SIZE);
     test->multisend = 10;	/* arbitrary */
     test->udp_counters_64bit = 0;
+    if (test->title) {
+	free(test->title);
+	test->title = NULL;
+    }
 
     /* Free output line buffers, if any (on the server only) */
     struct iperf_textline *t;
@@ -2049,11 +2158,12 @@ iperf_reset_stats(struct iperf_test *test)
     gettimeofday(&now, NULL);
     SLIST_FOREACH(sp, &test->streams, streams) {
 	sp->omitted_packet_count = sp->packet_count;
+        sp->omitted_cnt_error = sp->cnt_error;
+        sp->omitted_outoforder_packets = sp->outoforder_packets;
 	sp->jitter = 0;
-	sp->outoforder_packets = 0;
-	sp->cnt_error = 0;
 	rp = sp->result;
-        rp->bytes_sent = rp->bytes_received = 0;
+        rp->bytes_sent_omit = rp->bytes_sent;
+        rp->bytes_received = 0;
         rp->bytes_sent_this_interval = rp->bytes_received_this_interval = 0;
 	if (test->sender && test->sender_has_retransmits) {
 	    struct iperf_interval_results ir; /* temporary results structure */
@@ -2120,6 +2230,8 @@ iperf_stats_callback(struct iperf_test *test)
 		    }
 		    rp->stream_sum_rtt += temp.rtt;
 		    rp->stream_count_rtt++;
+
+		    temp.rttvar = get_rttvar(&temp);
 		}
 	    }
 	} else {
@@ -2234,7 +2346,12 @@ iperf_print_intermediate(struct iperf_test *test)
 		    iprintf(test, report_sum_bw_udp_sender_format, start_time, end_time, ubuf, nbuf, total_packets, test->omitting?report_omitted:"");
 	    } else {
 		avg_jitter /= test->num_streams;
-		lost_percent = 100.0 * lost_packets / total_packets;
+		if (total_packets > 0) {
+		    lost_percent = 100.0 * lost_packets / total_packets;
+		}
+		else {
+		    lost_percent = 0.0;
+		}
 		if (test->json_output)
 		    cJSON_AddItemToObject(json_interval, "sum", iperf_json_printf("start: %f  end: %f  seconds: %f  bytes: %d  bits_per_second: %f  jitter_ms: %f  lost_packets: %d  packets: %d  lost_percent: %f  omitted: %b", (double) start_time, (double) end_time, (double) irp->interval_duration, (int64_t) bytes, bandwidth * 8, (double) avg_jitter * 1000.0, (int64_t) lost_packets, (int64_t) total_packets, (double) lost_percent, test->omitting));
 		else
@@ -2263,7 +2380,7 @@ iperf_print_results(struct iperf_test *test)
     struct iperf_stream *sp = NULL;
     iperf_size_t bytes_sent, total_sent = 0;
     iperf_size_t bytes_received, total_received = 0;
-    double start_time, end_time, avg_jitter = 0.0, lost_percent;
+    double start_time, end_time = 0.0, avg_jitter = 0.0, lost_percent;
     double bandwidth;
 
     /* print final summary for all intervals */
@@ -2304,7 +2421,7 @@ iperf_print_results(struct iperf_test *test)
 	    cJSON_AddItemToArray(json_summary_streams, json_summary_stream);
 	}
 
-        bytes_sent = sp->result->bytes_sent;
+        bytes_sent = sp->result->bytes_sent - sp->result->bytes_sent_omit;
         bytes_received = sp->result->bytes_received;
         total_sent += bytes_sent;
         total_received += bytes_received;
@@ -2315,7 +2432,7 @@ iperf_print_results(struct iperf_test *test)
 	    }
 	} else {
             total_packets += (sp->packet_count - sp->omitted_packet_count);
-            lost_packets += sp->cnt_error;
+            lost_packets += (sp->cnt_error - sp->omitted_cnt_error);
             avg_jitter += sp->jitter;
         }
 
@@ -2338,21 +2455,30 @@ iperf_print_results(struct iperf_test *test)
 	    }
 	} else {
 	    /* Summary, UDP. */
-	    lost_percent = 100.0 * sp->cnt_error / (sp->packet_count - sp->omitted_packet_count);
-	    if (test->json_output)
-		cJSON_AddItemToObject(json_summary_stream, "udp", iperf_json_printf("socket: %d  start: %f  end: %f  seconds: %f  bytes: %d  bits_per_second: %f  jitter_ms: %f  lost_packets: %d  packets: %d  lost_percent: %f", (int64_t) sp->socket, (double) start_time, (double) end_time, (double) end_time, (int64_t) bytes_sent, bandwidth * 8, (double) sp->jitter * 1000.0, (int64_t) sp->cnt_error, (int64_t) (sp->packet_count - sp->omitted_packet_count), (double) lost_percent));
+	    if (sp->packet_count - sp->omitted_packet_count > 0) {
+              lost_percent = 100.0 * (sp->cnt_error - sp->omitted_cnt_error) / (sp->packet_count - sp->omitted_packet_count);
+	    }
 	    else {
-		iprintf(test, report_bw_udp_format, sp->socket, start_time, end_time, ubuf, nbuf, sp->jitter * 1000.0, sp->cnt_error, (sp->packet_count - sp->omitted_packet_count), lost_percent, "");
+		lost_percent = 0.0;
+	    }
+	    if (test->json_output)
+              cJSON_AddItemToObject(json_summary_stream, "udp", iperf_json_printf("socket: %d  start: %f  end: %f  seconds: %f  bytes: %d  bits_per_second: %f  jitter_ms: %f  lost_packets: %d  packets: %d  lost_percent: %f  out_of_order: %d", (int64_t) sp->socket, (double) start_time, (double) end_time, (double) end_time, (int64_t) bytes_sent, bandwidth * 8, (double) sp->jitter * 1000.0, (int64_t) (sp->cnt_error - sp->omitted_cnt_error), (int64_t) (sp->packet_count - sp->omitted_packet_count), (double) lost_percent, (int64_t) (sp->outoforder_packets - sp->omitted_outoforder_packets)));
+	    else {
+              iprintf(test, report_bw_udp_format, sp->socket, start_time, end_time, ubuf, nbuf, sp->jitter * 1000.0, (sp->cnt_error - sp->omitted_cnt_error), (sp->packet_count - sp->omitted_packet_count), lost_percent, "");
 		if (test->role == 'c')
 		    iprintf(test, report_datagrams, sp->socket, (sp->packet_count - sp->omitted_packet_count));
-		if (sp->outoforder_packets > 0)
-		    iprintf(test, report_sum_outoforder, start_time, end_time, sp->cnt_error);
+		if ((sp->outoforder_packets - sp->omitted_outoforder_packets) > 0)
+                  iprintf(test, report_sum_outoforder, start_time, end_time, (sp->outoforder_packets - sp->omitted_outoforder_packets));
 	    }
 	}
 
 	if (sp->diskfile_fd >= 0) {
 	    if (fstat(sp->diskfile_fd, &sb) == 0) {
-		int percent = (int) ( ( (double) bytes_sent / (double) sb.st_size ) * 100.0 );
+		/* In the odd case that it's a zero-sized file, say it was all transferred. */
+		int percent = 100;
+		if (sb.st_size > 0) {
+		    percent = (int) ( ( (double) bytes_sent / (double) sb.st_size ) * 100.0 );
+		}
 		unit_snprintf(sbuf, UNIT_LEN, (double) sb.st_size, 'A');
 		if (test->json_output)
 		    cJSON_AddItemToObject(json_summary_stream, "diskfile", iperf_json_printf("sent: %d  size: %d  percent: %d  filename: %s", (int64_t) bytes_sent, (int64_t) sb.st_size, (int64_t) percent, test->diskfile_name));
@@ -2413,12 +2539,12 @@ iperf_print_results(struct iperf_test *test)
         } else {
 	    /* Summary sum, UDP. */
             avg_jitter /= test->num_streams;
-	    /* If no packets were sent, arbitrarily set loss percentage to 100. */
+	    /* If no packets were sent, arbitrarily set loss percentage to 0. */
 	    if (total_packets > 0) {
 		lost_percent = 100.0 * lost_packets / total_packets;
 	    }
 	    else {
-		lost_percent = 100.0;
+		lost_percent = 0.0;
 	    }
 	    if (test->json_output)
 		cJSON_AddItemToObject(test->json_end, "sum", iperf_json_printf("start: %f  end: %f  seconds: %f  bytes: %d  bits_per_second: %f  jitter_ms: %f  lost_packets: %d  packets: %d  lost_percent: %f", (double) start_time, (double) end_time, (double) end_time, (int64_t) total_sent, bandwidth * 8, (double) avg_jitter * 1000.0, (int64_t) lost_packets, (int64_t) total_packets, (double) lost_percent));
@@ -2427,11 +2553,47 @@ iperf_print_results(struct iperf_test *test)
         }
     }
 
-    if (test->json_output)
+    if (test->json_output) {
 	cJSON_AddItemToObject(test->json_end, "cpu_utilization_percent", iperf_json_printf("host_total: %f  host_user: %f  host_system: %f  remote_total: %f  remote_user: %f  remote_system: %f", (double) test->cpu_util[0], (double) test->cpu_util[1], (double) test->cpu_util[2], (double) test->remote_cpu_util[0], (double) test->remote_cpu_util[1], (double) test->remote_cpu_util[2]));
+	if (test->protocol->id == Ptcp) {
+	    char *snd_congestion = NULL, *rcv_congestion = NULL;
+	    if (test->sender) {
+		snd_congestion = test->congestion_used;
+		rcv_congestion = test->remote_congestion_used;
+	    }
+	    else {
+		snd_congestion = test->remote_congestion_used;
+		rcv_congestion = test->congestion_used;
+	    }
+	    if (snd_congestion) {
+		cJSON_AddStringToObject(test->json_end, "sender_tcp_congestion", snd_congestion);
+	    }
+	    if (rcv_congestion) {
+		cJSON_AddStringToObject(test->json_end, "receiver_tcp_congestion", rcv_congestion);
+	    }
+	}
+    }
     else {
 	if (test->verbose) {
 	    iprintf(test, report_cpu, report_local, test->sender?report_sender:report_receiver, test->cpu_util[0], test->cpu_util[1], test->cpu_util[2], report_remote, test->sender?report_receiver:report_sender, test->remote_cpu_util[0], test->remote_cpu_util[1], test->remote_cpu_util[2]);
+
+	    if (test->protocol->id == Ptcp) {
+		char *snd_congestion = NULL, *rcv_congestion = NULL;
+		if (test->sender) {
+		    snd_congestion = test->congestion_used;
+		    rcv_congestion = test->remote_congestion_used;
+		}
+		else {
+		    snd_congestion = test->remote_congestion_used;
+		    rcv_congestion = test->congestion_used;
+		}
+		if (snd_congestion) {
+		    iprintf(test, "snd_tcp_congestion %s\n", snd_congestion);
+		}
+		if (rcv_congestion) {
+		    iprintf(test, "rcv_tcp_congestion %s\n", rcv_congestion);
+		}
+	    }
 	}
 
 	/* Print server output if we're on the client and it was requested/provided */
@@ -2520,7 +2682,12 @@ print_interval_results(struct iperf_test *test, struct iperf_stream *sp, cJSON *
     }
 
     unit_snprintf(ubuf, UNIT_LEN, (double) (irp->bytes_transferred), 'A');
-    bandwidth = (double) irp->bytes_transferred / (double) irp->interval_duration;
+    if (irp->interval_duration > 0.0) {
+	bandwidth = (double) irp->bytes_transferred / (double) irp->interval_duration;
+    }
+    else {
+	bandwidth = 0.0;
+    }
     unit_snprintf(nbuf, UNIT_LEN, bandwidth, test->settings->unit_format);
     
     st = timeval_diff(&sp->result->start_time, &irp->interval_start_time);
@@ -2530,7 +2697,7 @@ print_interval_results(struct iperf_test *test, struct iperf_stream *sp, cJSON *
 	if (test->sender && test->sender_has_retransmits) {
 	    /* Interval, TCP with retransmits. */
 	    if (test->json_output)
-		cJSON_AddItemToArray(json_interval_streams, iperf_json_printf("socket: %d  start: %f  end: %f  seconds: %f  bytes: %d  bits_per_second: %f  retransmits: %d  snd_cwnd:  %d  rtt:  %d  omitted: %b", (int64_t) sp->socket, (double) st, (double) et, (double) irp->interval_duration, (int64_t) irp->bytes_transferred, bandwidth * 8, (int64_t) irp->interval_retrans, (int64_t) irp->snd_cwnd, (int64_t) irp->rtt, irp->omitted));
+		cJSON_AddItemToArray(json_interval_streams, iperf_json_printf("socket: %d  start: %f  end: %f  seconds: %f  bytes: %d  bits_per_second: %f  retransmits: %d  snd_cwnd:  %d  rtt:  %d  rttvar: %d  omitted: %b", (int64_t) sp->socket, (double) st, (double) et, (double) irp->interval_duration, (int64_t) irp->bytes_transferred, bandwidth * 8, (int64_t) irp->interval_retrans, (int64_t) irp->snd_cwnd, (int64_t) irp->rtt, (int64_t) irp->rttvar, irp->omitted));
 	    else {
 		unit_snprintf(cbuf, UNIT_LEN, irp->snd_cwnd, 'A');
 		iprintf(test, report_bw_retrans_cwnd_format, sp->socket, st, et, ubuf, nbuf, irp->interval_retrans, cbuf, irp->omitted?report_omitted:"");
@@ -2550,7 +2717,12 @@ print_interval_results(struct iperf_test *test, struct iperf_stream *sp, cJSON *
 	    else
 		iprintf(test, report_bw_udp_sender_format, sp->socket, st, et, ubuf, nbuf, irp->interval_packet_count, irp->omitted?report_omitted:"");
 	} else {
-	    lost_percent = 100.0 * irp->interval_cnt_error / irp->interval_packet_count;
+	    if (irp->interval_packet_count > 0) {
+		lost_percent = 100.0 * irp->interval_cnt_error / irp->interval_packet_count;
+	    }
+	    else {
+		lost_percent = 0.0;
+	    }
 	    if (test->json_output)
 		cJSON_AddItemToArray(json_interval_streams, iperf_json_printf("socket: %d  start: %f  end: %f  seconds: %f  bytes: %d  bits_per_second: %f  jitter_ms: %f  lost_packets: %d  packets: %d  lost_percent: %f  omitted: %b", (int64_t) sp->socket, (double) st, (double) et, (double) irp->interval_duration, (int64_t) irp->bytes_transferred, bandwidth * 8, (double) irp->jitter * 1000.0, (int64_t) irp->interval_cnt_error, (int64_t) irp->interval_packet_count, (double) lost_percent, irp->omitted));
 	    else
@@ -2558,7 +2730,7 @@ print_interval_results(struct iperf_test *test, struct iperf_stream *sp, cJSON *
 	}
     }
 
-    if (test->logfile)
+    if (test->logfile || test->forceflush)
         iflush(test);
 }
 
@@ -2573,7 +2745,7 @@ iperf_free_stream(struct iperf_stream *sp)
     close(sp->buffer_fd);
     if (sp->diskfile_fd >= 0)
 	close(sp->diskfile_fd);
-    for (irp = TAILQ_FIRST(&sp->result->interval_results); irp != TAILQ_END(sp->result->interval_results); irp = nirp) {
+    for (irp = TAILQ_FIRST(&sp->result->interval_results); irp != NULL; irp = nirp) {
         nirp = TAILQ_NEXT(irp, irlistentries);
         free(irp);
     }
@@ -2591,15 +2763,13 @@ iperf_new_stream(struct iperf_test *test, int s)
     struct iperf_stream *sp;
     
     char template[1024];
-    if (test->template) {
-        snprintf(template, sizeof(template) / sizeof(char), "%s", test->template);
+    if (test->tmp_template) {
+        snprintf(template, sizeof(template) / sizeof(char), "%s", test->tmp_template);
     } else {
         char buf[] = "/tmp/iperf3.XXXXXX";
         snprintf(template, sizeof(template) / sizeof(char), "%s", buf);
     }
 
-    h_errno = 0;
-
     sp = (struct iperf_stream *) malloc(sizeof(struct iperf_stream));
     if (!sp) {
         i_errno = IECREATESTREAM;
@@ -2865,8 +3035,6 @@ iperf_json_start(struct iperf_test *test)
     test->json_top = cJSON_CreateObject();
     if (test->json_top == NULL)
         return -1;
-    if (test->title)
-	cJSON_AddStringToObject(test->json_top, "title", test->title);
     test->json_start = cJSON_CreateObject();
     if (test->json_start == NULL)
         return -1;
@@ -2889,6 +3057,8 @@ iperf_json_start(struct iperf_test *test)
 int
 iperf_json_finish(struct iperf_test *test)
 {
+    if (test->title)
+	cJSON_AddStringToObject(test->json_top, "title", test->title);
     /* Include server output */
     if (test->json_server_output) {
 	cJSON_AddItemToObject(test->json_top, "server_output_json", test->json_server_output);

src/iperf_api.h

@@ -1,5 +1,5 @@
 /*
- * iperf, Copyright (c) 2014, The Regents of the University of
+ * iperf, Copyright (c) 2014-2017, The Regents of the University of
  * California, through Lawrence Berkeley National Laboratory (subject
  * to receipt of any required approvals from the U.S. Dept. of
  * Energy).  All rights reserved.
@@ -39,7 +39,7 @@ struct iperf_stream;
 #define Ptcp SOCK_STREAM
 #define Pudp SOCK_DGRAM
 #define Psctp 12
-#define DEFAULT_UDP_BLKSIZE 8192
+#define DEFAULT_UDP_BLKSIZE 1460 /* default is dynamically set, else this */
 #define DEFAULT_TCP_BLKSIZE (128 * 1024)  /* default read/write block size */
 #define DEFAULT_SCTP_BLKSIZE (64 * 1024)
 
@@ -50,6 +50,10 @@ struct iperf_stream;
 #define OPT_UDP_COUNTERS_64BIT 4
 #define OPT_CLIENT_PORT 5
 #define OPT_NUMSTREAMS 6
+#define OPT_FORCEFLUSH 7
+#define OPT_NO_FQ_SOCKET_PACING 9 /* UNUSED */
+#define OPT_FQ_RATE 10
+#define OPT_DSCP 11
 
 /* states */
 #define TEST_START 1
@@ -114,7 +118,7 @@ void	iperf_set_test_socket_bufsize( struct iperf_test* ipt, int socket_bufsize )
 void	iperf_set_test_num_streams( struct iperf_test* ipt, int num_streams );
 void	iperf_set_test_role( struct iperf_test* ipt, char role );
 void	iperf_set_test_server_hostname( struct iperf_test* ipt, char* server_hostname );
-void    iperf_set_test_template( struct iperf_test *ipt, char *template );
+void    iperf_set_test_template( struct iperf_test *ipt, char *tmp_template );
 void	iperf_set_test_reverse( struct iperf_test* ipt, int reverse );
 void	iperf_set_test_json_output( struct iperf_test* ipt, int json_output );
 int	iperf_has_zerocopy( void );
@@ -161,7 +165,7 @@ void     iperf_reporter_callback(struct iperf_test * test);
  * returns NULL on failure
  *
  */
-struct iperf_test *iperf_new_test();
+struct iperf_test *iperf_new_test(void);
 
 int      iperf_defaults(struct iperf_test * testp);
 
@@ -204,6 +208,7 @@ void save_tcpinfo(struct iperf_stream *sp, struct iperf_interval_results *irp);
 long get_total_retransmits(struct iperf_interval_results *irp);
 long get_snd_cwnd(struct iperf_interval_results *irp);
 long get_rtt(struct iperf_interval_results *irp);
+long get_rttvar(struct iperf_interval_results *irp);
 void print_tcpinfo(struct iperf_test *test);
 void build_tcpinfo_message(struct iperf_interval_results *r, char *message);
 
@@ -213,8 +218,8 @@ int iperf_send(struct iperf_test *, fd_set *) /* __attribute__((hot)) */;
 int iperf_recv(struct iperf_test *, fd_set *);
 void iperf_catch_sigend(void (*handler)(int));
 void iperf_got_sigend(struct iperf_test *test) __attribute__ ((noreturn));
-void usage();
-void usage_long();
+void usage(void);
+void usage_long(FILE * f);
 void warning(char *);
 int iperf_exchange_results(struct iperf_test *);
 int iperf_init_test(struct iperf_test *);
@@ -288,7 +293,8 @@ enum {
     IELOGFILE = 17,	    // Can't open log file
     IENOSCTP = 18,	    // No SCTP support available
     IEBIND = 19,			// Local port specified with no local bind option
-    IEUDPBLOCKSIZE = 20,    // Block size too large. Maximum value = %dMAX_UDP_BLOCKSIZE
+    IEUDPBLOCKSIZE = 20,    // Block size invalid
+    IEBADTOS = 21,	    // Bad TOS value
     /* Test errors */
     IENEWTEST = 100,        // Unable to create a new test (check perror)
     IEINITTEST = 101,       // Test initialization failed (check perror)
@@ -330,6 +336,8 @@ enum {
     IESETSCTPDISABLEFRAG = 137, // Unable to set SCTP Fragmentation (check perror)
     IESETSCTPNSTREAM= 138,  //  Unable to set SCTP number of streams (check perror)
     IESETSCTPBINDX= 139,    // Unable to process sctp_bindx() parameters
+    IESETPACING= 140,       // Unable to set socket pacing rate
+    IESETBUF2= 141,	    // Socket buffer size incorrect (written value != read value)
     /* Stream errors */
     IECREATESTREAM = 200,   // Unable to create a new stream (check herror/perror)
     IEINITSTREAM = 201,     // Unable to initialize stream (check herror/perror)

src/iperf_client_api.c

@@ -1,5 +1,5 @@
 /*
- * iperf, Copyright (c) 2014, 2015, The Regents of the University of
+ * iperf, Copyright (c) 2014-2017, The Regents of the University of
  * California, through Lawrence Berkeley National Laboratory (subject
  * to receipt of any required approvals from the U.S. Dept. of
  * Energy).  All rights reserved.
@@ -32,6 +32,7 @@
 #include <unistd.h>
 #include <signal.h>
 #include <sys/types.h>
+#include <netinet/in.h>
 #include <sys/select.h>
 #include <sys/uio.h>
 #include <arpa/inet.h>
@@ -43,6 +44,11 @@
 #include "net.h"
 #include "timer.h"
 
+#if defined(HAVE_TCP_CONGESTION)
+#if !defined(TCP_CA_NAME_MAX)
+#define TCP_CA_NAME_MAX 16
+#endif /* TCP_CA_NAME_MAX */
+#endif /* HAVE_TCP_CONGESTION */
 
 int
 iperf_create_streams(struct iperf_test *test)
@@ -59,6 +65,31 @@ iperf_create_streams(struct iperf_test *test)
         if ((s = test->protocol->connect(test)) < 0)
             return -1;
 
+#if defined(HAVE_TCP_CONGESTION)
+	if (test->protocol->id == Ptcp) {
+	    if (test->congestion) {
+		if (setsockopt(s, IPPROTO_TCP, TCP_CONGESTION, test->congestion, strlen(test->congestion)) < 0) {
+		    close(s);
+		    i_errno = IESETCONGESTION;
+		    return -1;
+		} 
+	    }
+	    {
+		socklen_t len = TCP_CA_NAME_MAX;
+		char ca[TCP_CA_NAME_MAX + 1];
+		if (getsockopt(s, IPPROTO_TCP, TCP_CONGESTION, ca, &len) < 0) {
+		    close(s);
+		    i_errno = IESETCONGESTION;
+		    return -1;
+		}
+		test->congestion_used = strdup(ca);
+		if (test->debug) {
+		    printf("Congestion algorithm is %s\n", test->congestion_used);
+		}
+	    }
+	}
+#endif /* HAVE_TCP_CONGESTION */
+
 	if (test->sender)
 	    FD_SET(s, &test->write_set);
 	else
@@ -304,6 +335,61 @@ iperf_connect(struct iperf_test *test)
     FD_SET(test->ctrl_sck, &test->read_set);
     if (test->ctrl_sck > test->max_fd) test->max_fd = test->ctrl_sck;
 
+    int opt;
+    socklen_t len;
+
+    len = sizeof(opt);
+    if (getsockopt(test->ctrl_sck, IPPROTO_TCP, TCP_MAXSEG, &opt, &len) < 0) {
+	test->ctrl_sck_mss = 0;
+    }
+    else {
+	test->ctrl_sck_mss = opt;
+    }
+
+    if (test->verbose) {
+	printf("Control connection MSS %d\n", test->ctrl_sck_mss);
+    }
+
+    /*
+     * If we're doing a UDP test and the block size wasn't explicitly
+     * set, then use the known MSS of the control connection to pick
+     * an appropriate default.  If we weren't able to get the
+     * MSS for some reason, then default to something that should
+     * work on non-jumbo-frame Ethernet networks.  The goal is to
+     * pick a reasonable default that is large but should get from
+     * sender to receiver without any IP fragmentation.
+     *
+     * We assume that the control connection is routed the same as the
+     * data packets (thus has the same PMTU).  Also in the case of
+     * --reverse tests, we assume that the MTU is the same in both
+     * directions.  Note that even if the algorithm guesses wrong,
+     * the user always has the option to override.
+     */
+    if (test->protocol->id == Pudp) {
+	if (test->settings->blksize == 0) {
+	    if (test->ctrl_sck_mss) {
+		test->settings->blksize = test->ctrl_sck_mss;
+	    }
+	    else {
+		test->settings->blksize = DEFAULT_UDP_BLKSIZE;
+	    }
+	    if (test->verbose) {
+		printf("Setting UDP block size to %d\n", test->settings->blksize);
+	    }
+	}
+
+	/*
+	 * Regardless of whether explicitly or implicitly set, if the
+	 * block size is larger than the MSS, print a warning.
+	 */
+	if (test->settings->blksize > test->ctrl_sck_mss) {
+	    char str[128];
+	    snprintf(str, sizeof(str),
+		     "Warning:  UDP block size %d exceeds TCP MSS %d, may result in fragmentation / drops", test->settings->blksize, test->ctrl_sck_mss);
+	    warning(str);
+	}
+    }
+
     return 0;
 }
 

src/iperf_config.h.in

@@ -30,6 +30,9 @@
 /* Define to 1 if you have the `sendfile' function. */
 #undef HAVE_SENDFILE
 
+/* Have SO_MAX_PACING_RATE sockopt. */
+#undef HAVE_SO_MAX_PACING_RATE
+
 /* Define to 1 if you have the <stdint.h> header file. */
 #undef HAVE_STDINT_H
 

src/iperf_error.c

@@ -1,5 +1,5 @@
 /*
- * iperf, Copyright (c) 2014, The Regents of the University of
+ * iperf, Copyright (c) 2014, 2015, 2016, 2017, The Regents of the University of
  * California, through Lawrence Berkeley National Laboratory (subject
  * to receipt of any required approvals from the U.S. Dept. of
  * Energy).  All rights reserved.
@@ -74,14 +74,15 @@ iperf_errexit(struct iperf_test *test, const char *format, ...)
 	    fprintf(stderr, "iperf3: %s\n", str);
 	}
     va_end(argp);
-    iperf_delete_pidfile(test);
+    if (test)
+        iperf_delete_pidfile(test);
     exit(1);
 }
 
 int i_errno;
 
 char *
-iperf_strerror(int i_errno)
+iperf_strerror(int int_errno)
 {
     static char errstr[256];
     int len, perr, herr;
@@ -90,7 +91,7 @@ iperf_strerror(int i_errno)
     len = sizeof(errstr);
     memset(errstr, 0, len);
 
-    switch (i_errno) {
+    switch (int_errno) {
         case IENONE:
             snprintf(errstr, len, "no error");
             break;
@@ -125,8 +126,11 @@ iperf_strerror(int i_errno)
             snprintf(errstr, len, "--bind must be specified to use --cport");
             break;
         case IEUDPBLOCKSIZE:
-            snprintf(errstr, len, "block size too large (maximum = %d bytes)", MAX_UDP_BLOCKSIZE);
+            snprintf(errstr, len, "block size invalid (minimum = %d bytes, maximum = %d bytes)", MIN_UDP_BLOCKSIZE, MAX_UDP_BLOCKSIZE);
             break;
+	case IEBADTOS:
+	    snprintf(errstr, len, "bad TOS value (must be between 0 and 255 inclusive)");
+	    break;
         case IEMSS:
             snprintf(errstr, len, "TCP MSS too large (maximum = %d bytes)", MAX_MSS);
             break;
@@ -351,15 +355,20 @@ iperf_strerror(int i_errno)
             snprintf(errstr, len, "unable to set SCTP_INIT num of SCTP streams\n");
             perr = 1;
             break;
+	case IESETPACING:
+	    snprintf(errstr, len, "unable to set socket pacing");
+	    perr = 1;
+	    break;
+	case IESETBUF2:
+	    snprintf(errstr, len, "socket buffer size not set correctly");
+	    break;
+	
     }
 
     if (herr || perr)
         strncat(errstr, ": ", len - strlen(errstr) - 1);
-    if (h_errno && herr) {
-        strncat(errstr, hstrerror(h_errno), len - strlen(errstr) - 1);
-    } else if (errno && perr) {
+    if (errno && perr)
         strncat(errstr, strerror(errno), len - strlen(errstr) - 1);
-    }
 
     return errstr;
 }

src/iperf_locale.c

@@ -1,5 +1,5 @@
 /*--------------------------------------------------------------- 
- * iperf, Copyright (c) 2014, The Regents of the University of
+ * iperf, Copyright (c) 2014, 2016, 2017, The Regents of the University of
  * California, through Lawrence Berkeley National Laboratory (subject
  * to receipt of any required approvals from the U.S. Dept. of
  * Energy).  All rights reserved.
@@ -91,11 +91,11 @@ extern    "C"
  * usage
  * ------------------------------------------------------------------- */
 
-const char usage_shortstr[] = "Usage: iperf [-s|-c host] [options]\n"
-                           "Try `iperf --help' for more information.\n";
+const char usage_shortstr[] = "Usage: iperf3 [-s|-c host] [options]\n"
+                           "Try `iperf3 --help' for more information.\n";
 
-const char usage_longstr[] = "Usage: iperf [-s|-c host] [options]\n"
-                           "       iperf [-h|--help] [-v|--version]\n\n"
+const char usage_longstr[] = "Usage: iperf3 [-s|-c host] [options]\n"
+                           "       iperf3 [-h|--help] [-v|--version]\n\n"
                            "Server or Client:\n"
                            "  -p, --port      #         server port to listen on/connect to\n"
                            "  -f, --format    [kmgKMG]  format to report: Kbits, Mbits, KBytes, MBytes\n"
@@ -108,6 +108,7 @@ const char usage_longstr[] = "Usage: iperf [-s|-c host] [options]\n"
                            "  -V, --verbose             more detailed output\n"
                            "  -J, --json                output in JSON format\n"
                            "  --logfile f               send output to a log file\n"
+                           "  --forceflush              force flushing output at every interval\n"
                            "  -d, --debug               emit debugging output\n"
                            "  -v, --version             show version information and quit\n"
                            "  -h, --help                show this message and quit\n"
@@ -127,11 +128,15 @@ const char usage_longstr[] = "Usage: iperf [-s|-c host] [options]\n"
                            "  -b, --bandwidth #[KMG][/#] target bandwidth in bits/sec (0 for unlimited)\n"
                            "                            (default %d Mbit/sec for UDP, unlimited for TCP)\n"
                            "                            (optional slash and packet count for burst mode)\n"
+#if defined(HAVE_SO_MAX_PACING_RATE)
+                           "  --fq-rate #[KMG]          enable fair-queuing based socket pacing in\n"
+			   "                            bits/sec (Linux only)\n"
+#endif
                            "  -t, --time      #         time in seconds to transmit for (default %d secs)\n"
                            "  -n, --bytes     #[KMG]    number of bytes to transmit (instead of -t)\n"
                            "  -k, --blockcount #[KMG]   number of blocks (packets) to transmit (instead of -t or -n)\n"
                            "  -l, --len       #[KMG]    length of buffer to read or write\n"
-			   "                            (default %d KB for TCP, %d KB for UDP)\n"
+			   "                            (default %d KB for TCP, dynamic or %d for UDP)\n"
                            "  --cport         <port>    bind to a specific client port (TCP and UDP, default: ephemeral port)\n"
                            "  -P, --parallel  #         number of parallel client streams to run\n"
                            "  -R, --reverse             run in reverse mode (server sends, client receives)\n"
@@ -143,7 +148,8 @@ const char usage_longstr[] = "Usage: iperf [-s|-c host] [options]\n"
                            "  -N, --no-delay            set TCP/SCTP no delay, disabling Nagle's Algorithm\n"
                            "  -4, --version4            only use IPv4\n"
                            "  -6, --version6            only use IPv6\n"
-                           "  -S, --tos N               set the IP 'type of service'\n"
+                           "  -S, --tos N               set the IP type of service, 0-255\n"
+                           "  --dscp N or --dscp val    set the IP dscp value, either 0-63 or symbolic\n"
 #if defined(HAVE_FLOWLABEL)
                            "  -L, --flowlabel N         set the IPv6 flow label (only supported on Linux)\n"
 #endif /* HAVE_FLOWLABEL */

src/iperf_sctp.c

@@ -35,7 +35,6 @@
 #include <sys/types.h>
 #include <netinet/in.h>
 #include <netdb.h>
-#include <netinet/tcp.h>
 #include <sys/time.h>
 #include <sys/select.h>
 

src/iperf_server_api.c

@@ -1,5 +1,5 @@
 /*
- * iperf, Copyright (c) 2014, 2015, The Regents of the University of
+ * iperf, Copyright (c) 2014, 2015, 2016, 2017, The Regents of the University of
  * California, through Lawrence Berkeley National Laboratory (subject
  * to receipt of any required approvals from the U.S. Dept. of
  * Energy).  All rights reserved.
@@ -44,7 +44,6 @@
 #ifdef HAVE_STDINT_H
 #include <stdint.h>
 #endif
-#include <netinet/tcp.h>
 #include <sys/time.h>
 #include <sys/resource.h>
 #include <sched.h>
@@ -62,6 +61,11 @@
 #include "iperf_util.h"
 #include "iperf_locale.h"
 
+#if defined(HAVE_TCP_CONGESTION)
+#if !defined(TCP_CA_NAME_MAX)
+#define TCP_CA_NAME_MAX 16
+#endif /* TCP_CA_NAME_MAX */
+#endif /* HAVE_TCP_CONGESTION */
 
 int
 iperf_server_listen(struct iperf_test *test)
@@ -311,6 +315,26 @@ iperf_test_reset(struct iperf_test *test)
     memset(test->cookie, 0, COOKIE_SIZE); 
 }
 
+static void
+server_timer_proc(TimerClientData client_data, struct timeval *nowP)
+{
+    struct iperf_test *test = client_data.p;
+    struct iperf_stream *sp;
+
+    test->timer = NULL;
+    if (test->done)
+        return;
+    test->done = 1;
+    /* Free streams */
+    while (!SLIST_EMPTY(&test->streams)) {
+        sp = SLIST_FIRST(&test->streams);
+        SLIST_REMOVE_HEAD(&test->streams, streams);
+        close(sp->socket);
+        iperf_free_stream(sp);
+    }
+    close(test->ctrl_sck);
+}
+
 static void
 server_stats_timer_proc(TimerClientData client_data, struct timeval *nowP)
 {
@@ -344,6 +368,16 @@ create_server_timers(struct iperf_test * test)
 	return -1;
     }
     cd.p = test;
+    test->timer = test->stats_timer = test->reporter_timer = NULL;
+    if (test->duration != 0 ) {
+        test->done = 0;
+        test->timer = tmr_create(&now, server_timer_proc, cd, (test->duration + test->omit + 5) * SEC_TO_US, 0);
+        if (test->timer == NULL) {
+            i_errno = IEINITTEST;
+            return -1;
+        }
+    }
+
     test->stats_timer = test->reporter_timer = NULL;
     if (test->stats_interval != 0) {
         test->stats_timer = tmr_create(&now, server_stats_timer_proc, cd, test->stats_interval * SEC_TO_US, 1);
@@ -410,8 +444,12 @@ static void
 cleanup_server(struct iperf_test *test)
 {
     /* Close open test sockets */
-    close(test->ctrl_sck);
-    close(test->listener);
+    if (test->ctrl_sck) {
+	close(test->ctrl_sck);
+    }
+    if (test->listener) {
+	close(test->listener);
+    }
 
     /* Cancel any remaining timers. */
     if (test->stats_timer != NULL) {
@@ -426,6 +464,14 @@ cleanup_server(struct iperf_test *test)
 	tmr_cancel(test->omit_timer);
 	test->omit_timer = NULL;
     }
+    if (test->congestion_used != NULL) {
+        free(test->congestion_used);
+	test->congestion_used = NULL;
+    }
+    if (test->timer != NULL) {
+        tmr_cancel(test->timer);
+        test->timer = NULL;
+    }
 }
 
 
@@ -440,11 +486,11 @@ iperf_run_server(struct iperf_test *test)
 
     if (test->affinity != -1) 
 	if (iperf_setaffinity(test, test->affinity) != 0)
-	    return -1;
+	    return -2;
 
     if (test->json_output)
 	if (iperf_json_start(test) < 0)
-	    return -1;
+	    return -2;
 
     if (test->json_output) {
 	cJSON_AddItemToObject(test->json_start, "version", cJSON_CreateString(version));
@@ -458,7 +504,7 @@ iperf_run_server(struct iperf_test *test)
 
     // Open socket and listen
     if (iperf_server_listen(test) < 0) {
-        return -1;
+        return -2;
     }
 
     // Begin calculating CPU utilization
@@ -506,6 +552,47 @@ iperf_run_server(struct iperf_test *test)
                         return -1;
 		    }
 
+#if defined(HAVE_TCP_CONGESTION)
+		    if (test->protocol->id == Ptcp) {
+			if (test->congestion) {
+			    if (setsockopt(s, IPPROTO_TCP, TCP_CONGESTION, test->congestion, strlen(test->congestion)) < 0) {
+				/*
+				 * ENOENT means we tried to set the
+				 * congestion algorithm but the algorithm
+				 * specified doesn't exist.  This can happen
+				 * if the client and server have different
+				 * congestion algorithms available.  In this
+				 * case, print a warning, but otherwise
+				 * continue.
+				 */
+				if (errno == ENOENT) {
+				    warning("TCP congestion control algorithm not supported");
+				}
+				else {
+				    close(s);
+				    cleanup_server(test);
+				    i_errno = IESETCONGESTION;
+				    return -1;
+				}
+			    } 
+			}
+			{
+			    socklen_t len = TCP_CA_NAME_MAX;
+			    char ca[TCP_CA_NAME_MAX + 1];
+			    if (getsockopt(s, IPPROTO_TCP, TCP_CONGESTION, ca, &len) < 0) {
+				close(s);
+				cleanup_server(test);
+				i_errno = IESETCONGESTION;
+				return -1;
+			    }
+			    test->congestion_used = strdup(ca);
+			    if (test->debug) {
+				printf("Congestion algorithm is %s\n", test->congestion_used);
+			    }
+			}
+		    }
+#endif /* HAVE_TCP_CONGESTION */
+
                     if (!is_closed(s)) {
                         sp = iperf_new_stream(test, s);
                         if (!sp) {
@@ -545,6 +632,7 @@ iperf_run_server(struct iperf_test *test)
                         if (test->no_delay || test->settings->mss || test->settings->socket_bufsize) {
                             FD_CLR(test->listener, &test->read_set);
                             close(test->listener);
+			    test->listener = 0;
                             if ((s = netannounce(test->settings->domain, Ptcp, test->bind_address, test->server_port)) < 0) {
 				cleanup_server(test);
                                 i_errno = IELISTEN;

src/iperf_tcp.c

@@ -1,5 +1,5 @@
 /*
- * iperf, Copyright (c) 2014, The Regents of the University of
+ * iperf, Copyright (c) 2014, 2016, The Regents of the University of
  * California, through Lawrence Berkeley National Laboratory (subject
  * to receipt of any required approvals from the U.S. Dept. of
  * Energy).  All rights reserved.
@@ -35,7 +35,6 @@
 #include <sys/types.h>
 #include <netinet/in.h>
 #include <netdb.h>
-#include <netinet/tcp.h>
 #include <sys/time.h>
 #include <sys/select.h>
 
@@ -235,16 +234,29 @@ iperf_tcp_listen(struct iperf_test *test)
 	    }
 	    printf("SO_SNDBUF is %u\n", opt);
 	}
-#if defined(HAVE_TCP_CONGESTION)
-	if (test->congestion) {
-	    if (setsockopt(s, IPPROTO_TCP, TCP_CONGESTION, test->congestion, strlen(test->congestion)) < 0) {
-		close(s);
-		freeaddrinfo(res);
-		i_errno = IESETCONGESTION;
-		return -1;
-	    } 
+#if defined(HAVE_SO_MAX_PACING_RATE)
+    /* If fq socket pacing is specified, enable it. */
+    if (test->settings->fqrate) {
+	/* Convert bits per second to bytes per second */
+	unsigned int fqrate = test->settings->fqrate / 8;
+	if (fqrate > 0) {
+	    if (test->debug) {
+		printf("Setting fair-queue socket pacing to %u\n", fqrate);
+	    }
+	    if (setsockopt(s, SOL_SOCKET, SO_MAX_PACING_RATE, &fqrate, sizeof(fqrate)) < 0) {
+		warning("Unable to set socket pacing");
+	    }
 	}
-#endif /* HAVE_TCP_CONGESTION */
+    }
+#endif /* HAVE_SO_MAX_PACING_RATE */
+    {
+	unsigned int rate = test->settings->rate / 8;
+	if (rate > 0) {
+	    if (test->debug) {
+		printf("Setting application pacing to %u\n", rate);
+	    }
+	}
+    }
         opt = 1;
         if (setsockopt(s, SOL_SOCKET, SO_REUSEADDR, &opt, sizeof(opt)) < 0) {
 	    saved_errno = errno;
@@ -311,6 +323,7 @@ iperf_tcp_connect(struct iperf_test *test)
     struct addrinfo hints, *local_res, *server_res;
     char portstr[6];
     int s, opt;
+    socklen_t optlen;
     int saved_errno;
 
     if (test->bind_address) {
@@ -400,18 +413,43 @@ iperf_tcp_connect(struct iperf_test *test)
             return -1;
         }
     }
-    if (test->debug) {
-	socklen_t optlen = sizeof(opt);
-	if (getsockopt(s, SOL_SOCKET, SO_SNDBUF, &opt, &optlen) < 0) {
-	    saved_errno = errno;
-	    close(s);
-	    freeaddrinfo(server_res);
-	    errno = saved_errno;
-	    i_errno = IESETBUF;
-	    return -1;
-	}
-	printf("SO_SNDBUF is %u\n", opt);
+
+    /* Read back and verify the sender socket buffer size */
+    optlen = sizeof(opt);
+    if (getsockopt(s, SOL_SOCKET, SO_SNDBUF, &opt, &optlen) < 0) {
+	saved_errno = errno;
+	close(s);
+	freeaddrinfo(server_res);
+	errno = saved_errno;
+	i_errno = IESETBUF;
+	return -1;
     }
+    if (test->debug) {
+	printf("SNDBUF is %u, expecting %u\n", opt, test->settings->socket_bufsize);
+    }
+    if (test->settings->socket_bufsize && test->settings->socket_bufsize > opt) {
+	i_errno = IESETBUF2;
+	return -1;
+    }
+
+    /* Read back and verify the receiver socket buffer size */
+    optlen = sizeof(opt);
+    if (getsockopt(s, SOL_SOCKET, SO_RCVBUF, &opt, &optlen) < 0) {
+	saved_errno = errno;
+	close(s);
+	freeaddrinfo(server_res);
+	errno = saved_errno;
+	i_errno = IESETBUF;
+	return -1;
+    }
+    if (test->debug) {
+	printf("RCVBUF is %u, expecting %u\n", opt, test->settings->socket_bufsize);
+    }
+    if (test->settings->socket_bufsize && test->settings->socket_bufsize > opt) {
+	i_errno = IESETBUF2;
+	return -1;
+    }
+
 #if defined(HAVE_FLOWLABEL)
     if (test->settings->flowlabel) {
         if (server_res->ai_addr->sa_family != AF_INET6) {
@@ -457,16 +495,29 @@ iperf_tcp_connect(struct iperf_test *test)
     }
 #endif /* HAVE_FLOWLABEL */
 
-#if defined(HAVE_TCP_CONGESTION)
-    if (test->congestion) {
-	if (setsockopt(s, IPPROTO_TCP, TCP_CONGESTION, test->congestion, strlen(test->congestion)) < 0) {
-	    close(s);
-	    freeaddrinfo(server_res);
-	    i_errno = IESETCONGESTION;
-	    return -1;
+#if defined(HAVE_SO_MAX_PACING_RATE)
+    /* If socket pacing is specified try to enable it. */
+    if (test->settings->fqrate) {
+	/* Convert bits per second to bytes per second */
+	unsigned int fqrate = test->settings->fqrate / 8;
+	if (fqrate > 0) {
+	    if (test->debug) {
+		printf("Setting fair-queue socket pacing to %u\n", fqrate);
+	    }
+	    if (setsockopt(s, SOL_SOCKET, SO_MAX_PACING_RATE, &fqrate, sizeof(fqrate)) < 0) {
+		warning("Unable to set socket pacing");
+	    }
+	}
+    }
+#endif /* HAVE_SO_MAX_PACING_RATE */
+    {
+	unsigned int rate = test->settings->rate / 8;
+	if (rate > 0) {
+	    if (test->debug) {
+		printf("Setting application pacing to %u\n", rate);
+	    }
 	}
     }
-#endif /* HAVE_TCP_CONGESTION */
 
     if (connect(s, (struct sockaddr *) server_res->ai_addr, server_res->ai_addrlen) < 0 && errno != EINPROGRESS) {
 	saved_errno = errno;

src/iperf_udp.c

@@ -1,5 +1,5 @@
 /*
- * iperf, Copyright (c) 2014, The Regents of the University of
+ * iperf, Copyright (c) 2014, 2016, 2017, The Regents of the University of
  * California, through Lawrence Berkeley National Laboratory (subject
  * to receipt of any required approvals from the U.S. Dept. of
  * Energy).  All rights reserved.
@@ -193,6 +193,80 @@ iperf_udp_send(struct iperf_stream *sp)
  * connection knows about each other before the real data transfers begin.
  */
 
+/*
+ * Set and verify socket buffer sizes.
+ * Return 0 if no error, -1 if an error, +1 if socket buffers are
+ * potentially too small to hold a message.
+ */
+int
+iperf_udp_buffercheck(struct iperf_test *test, int s)
+{
+    int rc = 0;
+
+    /*
+     * Set socket buffer size if requested.  Do this for both sending and
+     * receiving so that we can cover both normal and --reverse operation.
+     */
+    int opt;
+    socklen_t optlen;
+    
+    if ((opt = test->settings->socket_bufsize)) {
+        if (setsockopt(s, SOL_SOCKET, SO_RCVBUF, &opt, sizeof(opt)) < 0) {
+            i_errno = IESETBUF;
+            return -1;
+        }
+        if (setsockopt(s, SOL_SOCKET, SO_SNDBUF, &opt, sizeof(opt)) < 0) {
+            i_errno = IESETBUF;
+            return -1;
+        }
+    }
+
+    /* Read back and verify the sender socket buffer size */
+    optlen = sizeof(opt);
+    if (getsockopt(s, SOL_SOCKET, SO_SNDBUF, &opt, &optlen) < 0) {
+	i_errno = IESETBUF;
+	return -1;
+    }
+    if (test->debug) {
+	printf("SNDBUF is %u, expecting %u\n", opt, test->settings->socket_bufsize);
+    }
+    if (test->settings->socket_bufsize && test->settings->socket_bufsize > opt) {
+	i_errno = IESETBUF2;
+	return -1;
+    }
+    if (test->settings->blksize > opt) {
+	char str[80];
+	snprintf(str, sizeof(str),
+		 "Block size %d > sending socket buffer size %d",
+		 test->settings->blksize, opt);
+	warning(str);
+	rc = 1;
+    }
+
+    /* Read back and verify the receiver socket buffer size */
+    optlen = sizeof(opt);
+    if (getsockopt(s, SOL_SOCKET, SO_RCVBUF, &opt, &optlen) < 0) {
+	i_errno = IESETBUF;
+	return -1;
+    }
+    if (test->debug) {
+	printf("RCVBUF is %u, expecting %u\n", opt, test->settings->socket_bufsize);
+    }
+    if (test->settings->socket_bufsize && test->settings->socket_bufsize > opt) {
+	i_errno = IESETBUF2;
+	return -1;
+    }
+    if (test->settings->blksize > opt) {
+	char str[80];
+	snprintf(str, sizeof(str),
+		 "Block size %d > receiving socket buffer size %d",
+		 test->settings->blksize, opt);
+	warning(str);
+	rc = 1;
+    }
+    return rc;
+}
+
 /*
  * iperf_udp_accept
  *
@@ -205,6 +279,7 @@ iperf_udp_accept(struct iperf_test *test)
     int       buf;
     socklen_t len;
     int       sz, s;
+    int	      rc;
 
     /*
      * Get the current outstanding socket.  This socket will be used to handle
@@ -228,20 +303,49 @@ iperf_udp_accept(struct iperf_test *test)
         return -1;
     }
 
+    /* Check and set socket buffer sizes */
+    rc = iperf_udp_buffercheck(test, s);
+    if (rc < 0)
+	/* error */
+	return rc;
     /*
-     * Set socket buffer size if requested.  Do this for both sending and
-     * receiving so that we can cover both normal and --reverse operation.
+     * If the socket buffer was too small, but it was the default
+     * size, then try explicitly setting it to something larger.
      */
-    int opt;
-    if ((opt = test->settings->socket_bufsize)) {
-        if (setsockopt(s, SOL_SOCKET, SO_RCVBUF, &opt, sizeof(opt)) < 0) {
-            i_errno = IESETBUF;
-            return -1;
-        }
-        if (setsockopt(s, SOL_SOCKET, SO_SNDBUF, &opt, sizeof(opt)) < 0) {
-            i_errno = IESETBUF;
-            return -1;
-        }
+    if (rc > 0) {
+	if (test->settings->socket_bufsize == 0) {
+	    int bufsize = test->settings->blksize + UDP_BUFFER_EXTRA;
+	    printf("Increasing socket buffer size to %d\n",
+		bufsize);
+	    test->settings->socket_bufsize = bufsize;
+	    rc = iperf_udp_buffercheck(test, s);
+	    if (rc < 0)
+		return rc;
+	}
+    }
+	
+#if defined(HAVE_SO_MAX_PACING_RATE)
+    /* If socket pacing is specified, try it. */
+    if (test->settings->fqrate) {
+	/* Convert bits per second to bytes per second */
+	unsigned int fqrate = test->settings->fqrate / 8;
+	if (fqrate > 0) {
+	    if (test->debug) {
+		printf("Setting fair-queue socket pacing to %u\n", fqrate);
+	    }
+	    if (setsockopt(s, SOL_SOCKET, SO_MAX_PACING_RATE, &fqrate, sizeof(fqrate)) < 0) {
+		warning("Unable to set socket pacing");
+	    }
+	}
+    }
+#endif /* HAVE_SO_MAX_PACING_RATE */
+    {
+	unsigned int rate = test->settings->rate / 8;
+	if (rate > 0) {
+	    if (test->debug) {
+		printf("Setting application pacing to %u\n", rate);
+	    }
+	}
     }
 
     /*
@@ -303,6 +407,7 @@ iperf_udp_connect(struct iperf_test *test)
 #ifdef SO_RCVTIMEO
     struct timeval tv;
 #endif
+    int rc;
 
     /* Create and bind our local socket. */
     if ((s = netdial(test->settings->domain, Pudp, test->bind_address, test->bind_port, test->server_hostname, test->server_port)) < 0) {
@@ -310,20 +415,49 @@ iperf_udp_connect(struct iperf_test *test)
         return -1;
     }
 
+    /* Check and set socket buffer sizes */
+    rc = iperf_udp_buffercheck(test, s);
+    if (rc < 0)
+	/* error */
+	return rc;
     /*
-     * Set socket buffer size if requested.  Do this for both sending and
-     * receiving so that we can cover both normal and --reverse operation.
+     * If the socket buffer was too small, but it was the default
+     * size, then try explicitly setting it to something larger.
      */
-    int opt;
-    if ((opt = test->settings->socket_bufsize)) {
-        if (setsockopt(s, SOL_SOCKET, SO_RCVBUF, &opt, sizeof(opt)) < 0) {
-            i_errno = IESETBUF;
-            return -1;
-        }
-        if (setsockopt(s, SOL_SOCKET, SO_SNDBUF, &opt, sizeof(opt)) < 0) {
-            i_errno = IESETBUF;
-            return -1;
-        }
+    if (rc > 0) {
+	if (test->settings->socket_bufsize == 0) {
+	    int bufsize = test->settings->blksize + UDP_BUFFER_EXTRA;
+	    printf("Increasing socket buffer size to %d\n",
+		bufsize);
+	    test->settings->socket_bufsize = bufsize;
+	    rc = iperf_udp_buffercheck(test, s);
+	    if (rc < 0)
+		return rc;
+	}
+    }
+	
+#if defined(HAVE_SO_MAX_PACING_RATE)
+    /* If socket pacing is available and not disabled, try it. */
+    if (test->settings->fqrate) {
+	/* Convert bits per second to bytes per second */
+	unsigned int fqrate = test->settings->fqrate / 8;
+	if (fqrate > 0) {
+	    if (test->debug) {
+		printf("Setting fair-queue socket pacing to %u\n", fqrate);
+	    }
+	    if (setsockopt(s, SOL_SOCKET, SO_MAX_PACING_RATE, &fqrate, sizeof(fqrate)) < 0) {
+		warning("Unable to set socket pacing");
+	    }
+	}
+    }
+#endif /* HAVE_SO_MAX_PACING_RATE */
+    {
+	unsigned int rate = test->settings->rate / 8;
+	if (rate > 0) {
+	    if (test->debug) {
+		printf("Setting application pacing to %u\n", rate);
+	    }
+	}
     }
 
 #ifdef SO_RCVTIMEO

src/iperf_util.c

@@ -1,5 +1,5 @@
 /*
- * iperf, Copyright (c) 2014, The Regents of the University of
+ * iperf, Copyright (c) 2014, 2016, The Regents of the University of
  * California, through Lawrence Berkeley National Laboratory (subject
  * to receipt of any required approvals from the U.S. Dept. of
  * Energy).  All rights reserved.
@@ -285,6 +285,16 @@ get_optional_features(void)
     numfeatures++;
 #endif /* HAVE_SENDFILE */
 
+#if defined(HAVE_SO_MAX_PACING_RATE)
+    if (numfeatures > 0) {
+	strncat(features, ", ",
+		sizeof(features) - strlen(features) - 1);
+    }
+    strncat(features, "socket pacing",
+	sizeof(features) - strlen(features) - 1);
+    numfeatures++;
+#endif /* HAVE_SO_MAX_PACING_RATE */
+
     if (numfeatures == 0) {
 	strncat(features, "None", 
 		sizeof(features) - strlen(features) - 1);
@@ -340,19 +350,22 @@ iperf_json_printf(const char *format, ...)
 		j = cJSON_CreateBool(va_arg(argp, int));
 		break;
 		case 'd':
-		j = cJSON_CreateInt(va_arg(argp, int64_t));
+		j = cJSON_CreateNumber(va_arg(argp, int64_t));
 		break;
 		case 'f':
-		j = cJSON_CreateFloat(va_arg(argp, double));
+		j = cJSON_CreateNumber(va_arg(argp, double));
 		break;
 		case 's':
 		j = cJSON_CreateString(va_arg(argp, char *));
 		break;
 		default:
+		va_end(argp);
 		return NULL;
 	    }
-	    if (j == NULL)
-		return NULL;
+	    if (j == NULL) {
+	    	va_end(argp);
+	    	return NULL;
+	    }
 	    cJSON_AddItemToObject(o, name, j);
 	    np = name;
 	    break;

src/libiperf.3

@@ -5,7 +5,7 @@ libiperf \- API for iperf3 network throughput tester
 .SH SYNOPSIS
 #include <iperf_api.h>
 .br
--liperf
+\-liperf
 
 .SH DESCRIPTION
 .PP

src/main.c

@@ -1,5 +1,5 @@
 /*
- * iperf, Copyright (c) 2014, The Regents of the University of
+ * iperf, Copyright (c) 2014, 2015, 2017, The Regents of the University of
  * California, through Lawrence Berkeley National Laboratory (subject
  * to receipt of any required approvals from the U.S. Dept. of
  * Energy).  All rights reserved.
@@ -41,10 +41,6 @@
 #include <netinet/in.h>
 #include <arpa/inet.h>
 #include <netdb.h>
-#ifdef HAVE_STDINT_H
-#include <stdint.h>
-#endif
-#include <netinet/tcp.h>
 
 #include "iperf.h"
 #include "iperf_api.h"
@@ -104,7 +100,7 @@ main(int argc, char **argv)
     if (iperf_parse_arguments(test, argc, argv) < 0) {
         iperf_err(test, "parameter error - %s", iperf_strerror(i_errno));
         fprintf(stderr, "\n");
-        usage_long();
+        usage_long(stdout);
         exit(1);
     }
 
@@ -119,7 +115,7 @@ main(int argc, char **argv)
 
 static jmp_buf sigend_jmp_buf;
 
-static void
+static void __attribute__ ((noreturn))
 sigend_handler(int sig)
 {
     longjmp(sigend_jmp_buf, 1);
@@ -129,13 +125,14 @@ sigend_handler(int sig)
 static int
 run(struct iperf_test *test)
 {
-    int consecutive_errors;
-
     /* Termination signals. */
     iperf_catch_sigend(sigend_handler);
     if (setjmp(sigend_jmp_buf))
 	iperf_got_sigend(test);
 
+    /* Ignore SIGPIPE to simplify error handling */
+    signal(SIGPIPE, SIG_IGN);
+
     switch (test->role) {
         case 's':
 	    if (test->daemon) {
@@ -145,22 +142,19 @@ run(struct iperf_test *test)
 		    iperf_errexit(test, "error - %s", iperf_strerror(i_errno));
 		}
 	    }
-	    consecutive_errors = 0;
 	    if (iperf_create_pidfile(test) < 0) {
 		i_errno = IEPIDFILE;
 		iperf_errexit(test, "error - %s", iperf_strerror(i_errno));
 	    }
             for (;;) {
-		if (iperf_run_server(test) < 0) {
+		int rc;
+		rc = iperf_run_server(test);
+		if (rc < 0) {
 		    iperf_err(test, "error - %s", iperf_strerror(i_errno));
-                    fprintf(stderr, "\n");
-		    ++consecutive_errors;
-		    if (consecutive_errors >= 5) {
-		        fprintf(stderr, "too many errors, exiting\n");
-			break;
+		    if (rc < -1) {
+		        iperf_errexit(test, "exiting");
 		    }
-                } else
-		    consecutive_errors = 0;
+                }
                 iperf_reset_test(test);
                 if (iperf_get_test_one_off(test))
                     break;
@@ -177,6 +171,7 @@ run(struct iperf_test *test)
     }
 
     iperf_catch_sigend(SIG_DFL);
+    signal(SIGPIPE, SIG_DFL);
 
     return 0;
 }

src/net.h

@@ -37,6 +37,7 @@ int getsock_tcp_mss(int inSock);
 int set_tcp_options(int sock, int no_delay, int mss);
 int setnonblocking(int fd, int nonblocking);
 int getsockdomain(int sock);
+int parse_qos(const char *tos);
 
 #define NET_SOFTERROR -1
 #define NET_HARDERROR -2

src/t_timer.c

@@ -24,6 +24,11 @@
  * This code is distributed under a BSD style license, see the LICENSE
  * file for complete information.
  */
+#include "iperf_config.h"
+
+#ifdef HAVE_STDINT_H
+#include <stdint.h>
+#endif
 #include <stdio.h>
 #include <stdlib.h>
 #include <unistd.h>

src/t_units.c

@@ -1,5 +1,5 @@
 /*
- * iperf, Copyright (c) 2014, The Regents of the University of
+ * iperf, Copyright (c) 2014, 2017, The Regents of the University of
  * California, through Lawrence Berkeley National Laboratory (subject
  * to receipt of any required approvals from the U.S. Dept. of
  * Energy).  All rights reserved.
@@ -45,20 +45,13 @@ main(int argc, char **argv)
     assert(1024.0 == unit_atof("1K"));
     assert(1024.0 * 1024.0 == unit_atof("1M"));
     assert(4.0 * 1024.0 * 1024.0 * 1024.0 == unit_atof("4G"));
+    assert(3.0 * 1024.0 * 1024.0 * 1024.0 * 1024.0 == unit_atof("3T"));
 
-#ifdef notdef
-    /* Obsolete - we no longer make a distinction between upper and lower
-    ** case.
-    */
-    assert(1000.0 * 0.5 == unit_atof("0.5k"));
-    assert(1000.0 == unit_atof("1k"));
-    assert(1000.0 * 1000.0 == unit_atof("1m"));
-    assert(4.0 * 1000.0 * 1000.0 * 1000.0 == unit_atof("4g"));
-#endif
     assert(1024.0 * 0.5 == unit_atof("0.5k"));
     assert(1024.0 == unit_atof("1k"));
     assert(1024.0 * 1024.0 == unit_atof("1m"));
     assert(4.0 * 1024.0 * 1024.0 * 1024.0 == unit_atof("4g"));
+    assert(3.0 * 1024.0 * 1024.0 * 1024.0 * 1024.0 == unit_atof("3t"));
 
     assert(1024 * 0.5 == unit_atoi("0.5K"));
     assert(1024 == unit_atoi("1K"));
@@ -66,22 +59,19 @@ main(int argc, char **argv)
     d = 4.0 * 1024 * 1024 * 1024;
     llu = (iperf_size_t) d;
     assert(llu == unit_atoi("4G"));
-
-#ifdef notdef
-    /* Also obsolete. */
-    assert(1000 * 0.5 == unit_atoi("0.5k"));
-    assert(1000 == unit_atoi("1k"));
-    assert(1000 * 1000 == unit_atoi("1m"));
-    d = 4.0 * 1000 * 1000 * 1000;
+    d = 3.0 * 1024 * 1024 * 1024 * 1024;
     llu = (iperf_size_t) d;
-    assert(llu == unit_atoi("4g"));
-#endif
+    assert(llu == unit_atoi("3T"));
+
     assert(1024 * 0.5 == unit_atoi("0.5k"));
     assert(1024 == unit_atoi("1k"));
     assert(1024 * 1024 == unit_atoi("1m"));
     d = 4.0 * 1024 * 1024 * 1024;
     llu = (iperf_size_t) d;
     assert(llu == unit_atoi("4g"));
+    d = 3.0 * 1024 * 1024 * 1024 * 1024;
+    llu = (iperf_size_t) d;
+    assert(llu == unit_atoi("3t"));
 
     unit_snprintf(s, 11, 1024.0, 'A');
     assert(strncmp(s, "1.00 KByte", 11) == 0);
@@ -102,5 +92,12 @@ main(int argc, char **argv)
     unit_snprintf(s, 11, d, 'a');
     assert(strncmp(s, "34.4 Gbit", 11) == 0);
 
+    d = 4.0 * 1024 * 1024 * 1024 * 1024;
+    unit_snprintf(s, 11, d, 'A');
+    assert(strncmp(s, "4.00 TByte", 11) == 0);
+
+    unit_snprintf(s, 11, d, 'a');
+    assert(strncmp(s, "35.2 Tbit", 11) == 0);
+
     return 0;
 }

src/tcp_info.c

@@ -48,7 +48,6 @@
 #include <sys/param.h>
 #include <sys/types.h>
 #include <sys/socket.h>
-#include <netinet/tcp.h>
 #include <string.h>
 #include <netinet/in.h>
 #include <errno.h>
@@ -136,7 +135,7 @@ get_snd_cwnd(struct iperf_interval_results *irp)
 #if defined(linux) && defined(TCP_MD5SIG)
     return irp->tcpInfo.tcpi_snd_cwnd * irp->tcpInfo.tcpi_snd_mss;
 #elif defined(__FreeBSD__) && __FreeBSD_version >= 600000
-    return irp->tcpInfo.tcpi_snd_cwnd * irp->tcpInfo.tcpi_snd_mss;
+    return irp->tcpInfo.tcpi_snd_cwnd;
 #elif defined(__NetBSD__) && defined(TCP_INFO)
     return irp->tcpInfo.tcpi_snd_cwnd * irp->tcpInfo.tcpi_snd_mss;
 #else
@@ -162,6 +161,24 @@ get_rtt(struct iperf_interval_results *irp)
 #endif
 }
 
+/*************************************************************/
+/*
+ * Return rttvar in usec.
+ */
+long
+get_rttvar(struct iperf_interval_results *irp)
+{
+#if defined(linux) && defined(TCP_MD5SIG)
+    return irp->tcpInfo.tcpi_rttvar;
+#elif defined(__FreeBSD__) && __FreeBSD_version >= 600000
+    return irp->tcpInfo.tcpi_rttvar;
+#elif defined(__NetBSD__) && defined(TCP_INFO)
+    return irp->tcpInfo.tcpi_rttvar;
+#else
+    return -1;
+#endif
+}
+
 /*************************************************************/
 void
 build_tcpinfo_message(struct iperf_interval_results *r, char *message)

src/units.c

@@ -48,7 +48,7 @@
  * by Mark Gates <mgates@nlanr.net>
  * and Ajay Tirumalla <tirumala@ncsa.uiuc.edu>
  * -------------------------------------------------------------------
- * input and output numbers, converting with kilo, mega, giga
+ * input and output numbers, converting with kilo, mega, giga, tera
  * ------------------------------------------------------------------- */
 
 #include <stdio.h>
@@ -60,7 +60,6 @@
 #include <sys/socket.h>
 #include <sys/types.h>
 #include <sys/time.h>
-#include <netinet/tcp.h>
 
 
 #include "iperf.h"
@@ -70,13 +69,15 @@ extern    "C"
 {
 #endif
 
-    const long KILO_UNIT = 1024;
-    const long MEGA_UNIT = 1024 * 1024;
-    const long GIGA_UNIT = 1024 * 1024 * 1024;
+    const double KILO_UNIT = 1024.0;
+    const double MEGA_UNIT = 1024.0 * 1024.0;
+    const double GIGA_UNIT = 1024.0 * 1024.0 * 1024.0;
+    const double TERA_UNIT = 1024.0 * 1024.0 * 1024.0 * 1024.0;
 
-    const long KILO_RATE_UNIT = 1000;
-    const long MEGA_RATE_UNIT = 1000 * 1000;
-    const long GIGA_RATE_UNIT = 1000 * 1000 * 1000;
+    const double KILO_RATE_UNIT = 1000.0;
+    const double MEGA_RATE_UNIT = 1000.0 * 1000.0;
+    const double GIGA_RATE_UNIT = 1000.0 * 1000.0 * 1000.0;
+    const double TERA_RATE_UNIT = 1000.0 * 1000.0 * 1000.0 * 1000.0;
 
 /* -------------------------------------------------------------------
  * unit_atof
@@ -96,9 +97,12 @@ extern    "C"
 	/* scan the number and any suffices */
 	          sscanf(s, "%lf%c", &n, &suffix);
 
-	/* convert according to [Gg Mm Kk] */
+	/* convert according to [Tt Gg Mm Kk] */
 	switch    (suffix)
 	{
+	case 't': case 'T':
+	    n *= TERA_UNIT;
+	    break;
 	case 'g': case 'G':
 	    n *= GIGA_UNIT;
 	    break;
@@ -132,9 +136,12 @@ extern    "C"
 	/* scan the number and any suffices */
 	          sscanf(s, "%lf%c", &n, &suffix);
 
-	/* convert according to [Gg Mm Kk] */
+	/* convert according to [Tt Gg Mm Kk] */
 	switch    (suffix)
 	{
+	case 't': case 'T':
+	    n *= TERA_RATE_UNIT;
+	    break;
 	case 'g': case 'G':
 	    n *= GIGA_RATE_UNIT;
 	    break;
@@ -157,7 +164,7 @@ extern    "C"
  *
  * Given a string of form #x where # is a number and x is a format
  * character listed below, this returns the interpreted integer.
- * Gg, Mm, Kk are giga, mega, kilo respectively
+ * Tt, Gg, Mm, Kk are tera, giga, mega, kilo respectively
  * ------------------------------------------------------------------- */
 
     iperf_size_t unit_atoi(const char *s)
@@ -170,9 +177,12 @@ extern    "C"
 	/* scan the number and any suffices */
 	          sscanf(s, "%lf%c", &n, &suffix);
 
-	/* convert according to [Gg Mm Kk] */
+	/* convert according to [Tt Gg Mm Kk] */
 	switch    (suffix)
 	{
+	case 't': case 'T':
+	    n *= TERA_UNIT;
+	    break;
 	case 'g': case 'G':
 	    n *= GIGA_UNIT;
 	    break;
@@ -198,7 +208,8 @@ extern    "C"
 	UNIT_CONV,
 	KILO_CONV,
 	MEGA_CONV,
-	GIGA_CONV
+	GIGA_CONV,
+	TERA_CONV
     };
 
 /* factor to multiply the number by */
@@ -207,7 +218,8 @@ extern    "C"
 	1.0,			/* unit */
 	1.0 / 1024,		/* kilo */
 	1.0 / 1024 / 1024,	/* mega */
-	1.0 / 1024 / 1024 / 1024/* giga */
+	1.0 / 1024 / 1024 / 1024, /* giga */
+	1.0 / 1024 / 1024 / 1024 / 1024 /* tera */
     };
 
 /* factor to multiply the number by for bits*/
@@ -216,26 +228,29 @@ extern    "C"
 	1.0,			/* unit */
 	1.0 / 1000,		/* kilo */
 	1.0 / 1000 / 1000,	/* mega */
-	1.0 / 1000 / 1000 / 1000/* giga */
+	1.0 / 1000 / 1000 / 1000, /* giga */
+	1.0 / 1000 / 1000 / 1000 / 1000 /* tera */
     };
 
 
-/* labels for Byte formats [KMG] */
+/* labels for Byte formats [KMGT] */
     const char *label_byte[] =
     {
 	"Byte",
 	"KByte",
 	"MByte",
-	"GByte"
+	"GByte",
+	"TByte"
     };
 
-/* labels for bit formats [kmg] */
+/* labels for bit formats [kmgt] */
     const char *label_bit[] =
     {
 	"bit",
 	"Kbit",
 	"Mbit",
-	"Gbit"
+	"Gbit",
+	"Tbit"
     };
 
 /* -------------------------------------------------------------------
@@ -276,6 +291,9 @@ extern    "C"
 	case 'G':
 	    conv = GIGA_CONV;
 	    break;
+	case 'T':
+	    conv = TERA_CONV;
+	    break;
 
 	default:
 	case 'A':
@@ -285,14 +303,14 @@ extern    "C"
 
 		if (isupper((int) inFormat))
 		{
-		    while (tmpNum >= 1024.0 && conv <= GIGA_CONV)
+		    while (tmpNum >= 1024.0 && conv <= TERA_CONV)
 		    {
 			tmpNum /= 1024.0;
 			conv++;
 		    }
 		} else
 		{
-		    while (tmpNum >= 1000.0 && conv <= GIGA_CONV)
+		    while (tmpNum >= 1000.0 && conv <= TERA_CONV)
 		    {
 			tmpNum /= 1000.0;
 			conv++;