1.. _faq: 2 3iperf3 FAQ 4========== 5 6What is the history of iperf3, and what is the difference between iperf2 and iperf3? 7 iperf2 was orphaned in the late 2000s at version 2.0.5, despite some 8 known bugs and issues. After spending some time trying to fix 9 iperf2's problems, ESnet decided by 2010 that a new, simpler tool 10 was needed, and began development of iperf3. The goal was make the 11 tool as simple as possible, so others could contribute to the code 12 base. For this reason, it was decided to make the tool single 13 threaded, and not worry about backwards compatibility with 14 iperf2. Many of the feature requests for iperf3 came from the 15 perfSONAR project (http://www.perfsonar.net). 16 17 Then in 2014, Bob (Robert) McMahon from Broadcom restarted 18 development of iperf2 (See 19 https://sourceforge.net/projects/iperf2/). He fixed many of the 20 problems with iperf2, and added a number of new features similar to 21 iperf3. iperf2.0.8, released in 2015, made iperf2 a useful tool. iperf2's 22 current development is focused is on using UDP for latency testing, as well 23 as broad platform support. 24 25 As of this writing (2017), both iperf2 and iperf3 are being actively 26 (although independently) developed. We recommend being familiar with 27 both tools, and use whichever tool’s features best match your needs. 28 29 A feature comparison of iperf2, iperf3, and nuttcp is available at: 30 https://fasterdata.es.net/performance-testing/network-troubleshooting-tools/throughput-tool-comparision/ 31 32iperf3 parallel stream performance is much less than iperf2. Why? 33 iperf3 is single threaded, and iperf2 is multi-threaded. We 34 recommend using iperf2 for parallel streams. 35 If you want to use multiple iperf3 streams use the method described `here <https://fasterdata.es.net/performance-testing/network-troubleshooting-tools/iperf/multi-stream-iperf3/>`_. 36 37I’m trying to use iperf3 on Windows, but having trouble. What should I do? 38 iperf3 is not officially supported on Windows, but iperf2 is. We 39 recommend you use iperf2. 40 41 Some people are using Cygwin to run iperf3 in Windows, but not all 42 options will work. Some community-provided binaries of iperf3 for 43 Windows exist. 44 45How can I build a statically-linked executable of iperf3? 46 There are a number of reasons for building an iperf3 executable with 47 no dependencies on any shared libraries. Unfortunately this isn't 48 quite a straight-forward process. 49 50 The steps below have nominally been tested on CentOS 7.4, but 51 can probably be adapted for use with other Linux distributions: 52 53 #. If necessary, install the static C libraries; for CentOS this is 54 the ``glibc-static`` package. 55 56 #. If OpenSSL is installed, be sure that its static libraries are 57 also installed, from the ``openssl-static`` package. 58 59 #. Be sure that ``lksctp-*`` packages are not installed, because 60 as of this writing, there do not appear to be any static 61 libraries available for SCTP. 62 63 #. Configure iperf3 thusly: ``configure "LDFLAGS=--static" 64 --disable-shared`` These options are necessary to disable the 65 generation of shared libraries and link the executable 66 statically. For iperf-3.8 or later, configuring as ``configure 67 --enable-static-bin`` is another, shorter way to accomplish 68 this. If SCTP is installed on the system it might also be 69 necessary to pass the ``--without-sctp`` flag at configure 70 time. 71 72 #. Compile as normal. 73 74 It appears that for FreeBSD (tested on FreeBSD 11.1-RELEASE), only 75 the last two steps are needed to produce a static executable. 76 77How can I build on a system that doesn't support profiled executables? 78 This problem has been noted by users attempting to build iperf3 for 79 Android systems, as well as some recent versions of macOS. 80 There are several workarounds. In order from least 81 effort to most effort: 82 83 #. Beginning with iperf-3.8, profiled executables are actually not 84 built by default, so this question becomes somewhat moot. Pass 85 the ``--enable-profiling`` flag to ``configure`` to build 86 profiled executables. 87 88 #. In iperf-3.6 and iperf-3.7, the ``--disable-profiling`` flag can be 89 passed to ``configure`` to disable the building of profiled 90 object files and the profiled executable. 91 92 #. At the time the linking of the iperf3 profiled executable fails, 93 the "normal" iperf3 executable is probably already created. So if 94 you are willing to accept the error exit from the make process 95 (and a little bit of wasted work on the build host), you might 96 not need to do anything. 97 98 #. After the configure step, there will be a definition in 99 ``src/Makefile`` that looks like this:: 100 101 noinst_PROGRAMS = t_timer$(EXEEXT) t_units$(EXEEXT) t_uuid$(EXEEXT) \ 102 iperf3_profile$(EXEEXT) 103 104 If you edit it to look like this, it will disable the build of the profiled iperf3:: 105 106 noinst_PROGRAMS = t_timer$(EXEEXT) t_units$(EXEEXT) t_uuid$(EXEEXT) 107 108 #. Similar to item 2 above, but more permanent...if you edit 109 ``src/Makefile.am`` and change the line reading like this:: 110 111 noinst_PROGRAMS = t_timer t_units t_uuid iperf3_profile 112 113 To look like this:: 114 115 noinst_PROGRAMS = t_timer t_units t_uuid 116 117 And then run ``./bootstrap.sh``, that will regenerate the project 118 Makefiles to make the exclusion of the profiled iperf3 executable 119 permanant (within that source tree). 120 121I'm seeing quite a bit of unexpected UDP loss. Why? 122 First, confirm you are using iperf 3.1.5 or higher. There was an 123 issue with the default UDP send size that was fixed in 124 3.1.5. Second, try adding the flag ``-w2M`` to increase the socket 125 buffer sizes. That seems to make a big difference on some hosts. 126 127iperf3 UDP does not seem to work at bandwidths less than 100Kbps. Why? 128 You'll need to reduce the default packet length to get UDP rates of less that 100Kbps. Try ``-l100``. 129 130TCP throughput drops to (almost) zero during a test, what's going on? 131 A drop in throughput to almost zero, except maybe for the first 132 reported interval(s), may be related to problems in NIC TCP Offload, 133 which is used to offload TCP functionality to the NIC (see 134 https://en.wikipedia.org/wiki/TCP_offload_engine). The goal of TCP 135 Offload is to save main CPU performance, mainly in the areas of 136 segmentation and reassembly of large packets and checksum 137 computation. 138 139 When TCP packets are sent with the "Don't Fragment" flag set, which 140 is the recommended setting, segmentation is done by the TCP stack 141 based on the reported next hop MSS in the ICMP Fragmentation Needed 142 message. With TCP Offload, active segmentation is done by the NIC on 143 the sending side, which is known as TCP Segmentation offload (TSO) 144 or in Windows as Large Send Offload (LSO). It seems that there are 145 TSO/LSO implementations which for some reason ignore the reported 146 MSS and therefore don’t perform segmentation. In these cases, when 147 large packets are sent, e.g. the default iperf3 128KB (131,072 148 bytes), iperf3 will show that data was sent in the first interval, 149 but since the packets don’t get to the server, no ack is received 150 and therefore no data is sent in the following intervals. It may 151 happen that after certain timeout the main CPU will re-send the 152 packet by re-segmenting it, and in these cases data will get to the 153 server after a while. However, it seems that segmentation is not 154 automatically continued with the next packet, so the data transfer 155 rate be very low. 156 157 The recommended solution in such a case is to disable TSO/LSO, at 158 least on the relevant port. See for example: 159 https://atomicit.ca/kb/articles/slow-network-speed-windows-10/. If 160 that doesn’t help then "Don't Fragment" TCP flag may be 161 disabled. See for example: 162 https://support.microsoft.com/en-us/help/900926/recommended-tcp-ip-settings-for-wan-links-with-a-mtu-size-of-less-than. However, 163 note that disabling the “Don’t Fragment” flag may cause other 164 issues. 165 166 To test whether TSO/LSO may be the problem, do the following: 167 168 * If different machine configurations are used for the client and 169 server, try the iperf3 reverse mode (``-R``). If TSO/LSO is only 170 enabled on the client machine, this test should succeed. 171 * Reduce the sending length to a small value that should not require 172 segmentation, using the iperf3 ``-l`` option, e.g. ``-l 512``. It 173 may also help to reduce the MTU by using the iperf3 ``-M`` option, 174 e.g. ``-M 1460``. 175 * Using tools like Wireshark, identify the required MSS in the ICMP 176 Fragmentation Needed messages (if reported). Run tests with the 177 ``-l`` value set to 2 times the MSS and then 4 times, 6 times, 178 etc. With TSO/LSO issue in each test the throughput should be 179 reduced more. It may help to increase the testing time beyond the 180 default 10 seconds to better see the behavior (iperf3 ``-t`` 181 option). 182 183What congestion control algorithms are supported? 184 On Linux, run this command to see the available congestion control 185 algorithms (note that some algorithms are packaged as kernel 186 modules, which must be loaded before they can be used):: 187 188 /sbin/sysctl net.ipv4.tcp_available_congestion_control 189 190 On FreeBSD, the equivalent command is:: 191 192 /sbin/sysctl net.inet.tcp.cc.available 193 194I’m using the ``--logfile`` option. How do I see file output in real time? 195 Use the ``--forceflush`` flag. 196 197I'm using the --fq-rate flag, but it does not seem to be working. Why? 198 You need to add 'net.core.default_qdisc = fq' to /etc/sysctl.conf for that option to work. 199 200I'm having trouble getting iperf3 to work on Windows, Android, etc. Where can I get help? 201 iperf3 only supports Linux, FreeBSD, and OSX. For other platforms we recommend using iperf2. 202 203I managed to get a Windows executable built, but why do I get a BSOD on Windows 7? 204 There seems to be a bug in Windows 7 where running iperf3 from a 205 network filesystem can cause a system crash (in other words Blue 206 Screen of Death, or BSOD). This is a Windows bug addressed in kb2839149: 207 208 https://support.microsoft.com/en-us/help/2839149/stop-error-0x00000027-in-the-rdbss-sys-process-in-windows-7-or-windows 209 210 A hotfix is available under kb2732673: 211 212 https://support.microsoft.com/en-us/help/2732673/-delayed-write-failed-error-message-when--pst-files-are-stored-on-a-ne 213 214Why can’t I run a UDP client with no server? 215 This is potentially dangerous, and an attacker could use this for a 216 denial of service attack. We don't want iperf3 to be an attack tool. 217 218I'm trying to use iperf3 to test a 40G/100G link...What do I need to know? 219 See the following pages on fasterdata.es.net: 220 221 - https://fasterdata.es.net/host-tuning/100g-tuning/ 222 - https://fasterdata.es.net/performance-testing/network-troubleshooting-tools/iperf/multi-stream-iperf3/ 223 224My receiver didn't get all the bytes that got sent but there was no loss. Huh? 225 iperf3 uses a control connection between the client and server to 226 manage the start and end of each test. Sometimes the commands on 227 the control connection can be received and acted upon before all of 228 the test data has been processed. Thus the test ends with data 229 still in flight. This effect can be significant for short (a few 230 seconds) tests, but is probably negligible for longer tests. 231 232A file sent using the ``-F`` option got corrupted...what happened? 233 The ``-F`` option to iperf3 is not a file transfer utility. It's a 234 way of testing the end-to-end performance of a file transfer, 235 including filesystem and disk overheads. So while the test will 236 mimic an actual file transfer, the data stored to disk may not be 237 the same as what was sent. In particular, the file size will be 238 rounded up to the next larger multiple of the transfer block size, 239 and for UDP tests, iperf's metadata (containing timestamps and 240 sequence numbers) will overwrite the start of every UDP packet 241 payload. 242 243I have a question regarding iperf3...what's the best way to get help? 244 Searching on the Internet is a good first step. 245 http://stackoverflow.com/ has a number of iperf3-related questions 246 and answers, but a simple query into your favorite search engine can 247 also yield some results. 248 249 There is a mailing list nominally used for iperf3 development, 250 iperf-dev@googlegroups.com. 251 252 We discourage the use of the iperf3 issue tracker on GitHub for 253 support questions. Actual bug reports, enhancement requests, or 254 pull requests are encouraged, however. 255 256 257