1.. SPDX-License-Identifier: GPL-2.0 2 3========================================== 4EQL Driver: Serial IP Load Balancing HOWTO 5========================================== 6 7 Simon "Guru Aleph-Null" Janes, simon@ncm.com 8 9 v1.1, February 27, 1995 10 11 This is the manual for the EQL device driver. EQL is a software device 12 that lets you load-balance IP serial links (SLIP or uncompressed PPP) 13 to increase your bandwidth. It will not reduce your latency (i.e. ping 14 times) except in the case where you already have lots of traffic on 15 your link, in which it will help them out. This driver has been tested 16 with the 1.1.75 kernel, and is known to have patched cleanly with 17 1.1.86. Some testing with 1.1.92 has been done with the v1.1 patch 18 which was only created to patch cleanly in the very latest kernel 19 source trees. (Yes, it worked fine.) 20 211. Introduction 22=============== 23 24 Which is worse? A huge fee for a 56K leased line or two phone lines? 25 It's probably the former. If you find yourself craving more bandwidth, 26 and have a ISP that is flexible, it is now possible to bind modems 27 together to work as one point-to-point link to increase your 28 bandwidth. All without having to have a special black box on either 29 side. 30 31 32 The eql driver has only been tested with the Livingston PortMaster-2e 33 terminal server. I do not know if other terminal servers support load- 34 balancing, but I do know that the PortMaster does it, and does it 35 almost as well as the eql driver seems to do it (-- Unfortunately, in 36 my testing so far, the Livingston PortMaster 2e's load-balancing is a 37 good 1 to 2 KB/s slower than the test machine working with a 28.8 Kbps 38 and 14.4 Kbps connection. However, I am not sure that it really is 39 the PortMaster, or if it's Linux's TCP drivers. I'm told that Linux's 40 TCP implementation is pretty fast though.--) 41 42 43 I suggest to ISPs out there that it would probably be fair to charge 44 a load-balancing client 75% of the cost of the second line and 50% of 45 the cost of the third line etc... 46 47 48 Hey, we can all dream you know... 49 50 512. Kernel Configuration 52======================= 53 54 Here I describe the general steps of getting a kernel up and working 55 with the eql driver. From patching, building, to installing. 56 57 582.1. Patching The Kernel 59------------------------ 60 61 If you do not have or cannot get a copy of the kernel with the eql 62 driver folded into it, get your copy of the driver from 63 ftp://slaughter.ncm.com/pub/Linux/LOAD_BALANCING/eql-1.1.tar.gz. 64 Unpack this archive someplace obvious like /usr/local/src/. It will 65 create the following files:: 66 67 -rw-r--r-- guru/ncm 198 Jan 19 18:53 1995 eql-1.1/NO-WARRANTY 68 -rw-r--r-- guru/ncm 30620 Feb 27 21:40 1995 eql-1.1/eql-1.1.patch 69 -rwxr-xr-x guru/ncm 16111 Jan 12 22:29 1995 eql-1.1/eql_enslave 70 -rw-r--r-- guru/ncm 2195 Jan 10 21:48 1995 eql-1.1/eql_enslave.c 71 72 Unpack a recent kernel (something after 1.1.92) someplace convenient 73 like say /usr/src/linux-1.1.92.eql. Use symbolic links to point 74 /usr/src/linux to this development directory. 75 76 77 Apply the patch by running the commands:: 78 79 cd /usr/src 80 patch </usr/local/src/eql-1.1/eql-1.1.patch 81 82 832.2. Building The Kernel 84------------------------ 85 86 After patching the kernel, run make config and configure the kernel 87 for your hardware. 88 89 90 After configuration, make and install according to your habit. 91 92 933. Network Configuration 94======================== 95 96 So far, I have only used the eql device with the DSLIP SLIP connection 97 manager by Matt Dillon (-- "The man who sold his soul to code so much 98 so quickly."--) . How you configure it for other "connection" 99 managers is up to you. Most other connection managers that I've seen 100 don't do a very good job when it comes to handling more than one 101 connection. 102 103 1043.1. /etc/rc.d/rc.inet1 105----------------------- 106 107 In rc.inet1, ifconfig the eql device to the IP address you usually use 108 for your machine, and the MTU you prefer for your SLIP lines. One 109 could argue that MTU should be roughly half the usual size for two 110 modems, one-third for three, one-fourth for four, etc... But going 111 too far below 296 is probably overkill. Here is an example ifconfig 112 command that sets up the eql device:: 113 114 ifconfig eql 198.67.33.239 mtu 1006 115 116 Once the eql device is up and running, add a static default route to 117 it in the routing table using the cool new route syntax that makes 118 life so much easier:: 119 120 route add default eql 121 122 1233.2. Enslaving Devices By Hand 124------------------------------ 125 126 Enslaving devices by hand requires two utility programs: eql_enslave 127 and eql_emancipate (-- eql_emancipate hasn't been written because when 128 an enslaved device "dies", it is automatically taken out of the queue. 129 I haven't found a good reason to write it yet... other than for 130 completeness, but that isn't a good motivator is it?--) 131 132 133 The syntax for enslaving a device is "eql_enslave <master-name> 134 <slave-name> <estimated-bps>". Here are some example enslavings:: 135 136 eql_enslave eql sl0 28800 137 eql_enslave eql ppp0 14400 138 eql_enslave eql sl1 57600 139 140 When you want to free a device from its life of slavery, you can 141 either down the device with ifconfig (eql will automatically bury the 142 dead slave and remove it from its queue) or use eql_emancipate to free 143 it. (-- Or just ifconfig it down, and the eql driver will take it out 144 for you.--):: 145 146 eql_emancipate eql sl0 147 eql_emancipate eql ppp0 148 eql_emancipate eql sl1 149 150 1513.3. DSLIP Configuration for the eql Device 152------------------------------------------- 153 154 The general idea is to bring up and keep up as many SLIP connections 155 as you need, automatically. 156 157 1583.3.1. /etc/slip/runslip.conf 159^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 160 161 Here is an example runslip.conf:: 162 163 name sl-line-1 164 enabled 165 baud 38400 166 mtu 576 167 ducmd -e /etc/slip/dialout/cua2-288.xp -t 9 168 command eql_enslave eql $interface 28800 169 address 198.67.33.239 170 line /dev/cua2 171 172 name sl-line-2 173 enabled 174 baud 38400 175 mtu 576 176 ducmd -e /etc/slip/dialout/cua3-288.xp -t 9 177 command eql_enslave eql $interface 28800 178 address 198.67.33.239 179 line /dev/cua3 180 181 1823.4. Using PPP and the eql Device 183--------------------------------- 184 185 I have not yet done any load-balancing testing for PPP devices, mainly 186 because I don't have a PPP-connection manager like SLIP has with 187 DSLIP. I did find a good tip from LinuxNET:Billy for PPP performance: 188 make sure you have asyncmap set to something so that control 189 characters are not escaped. 190 191 192 I tried to fix up a PPP script/system for redialing lost PPP 193 connections for use with the eql driver the weekend of Feb 25-26 '95 194 (Hereafter known as the 8-hour PPP Hate Festival). Perhaps later this 195 year. 196 197 1984. About the Slave Scheduler Algorithm 199====================================== 200 201 The slave scheduler probably could be replaced with a dozen other 202 things and push traffic much faster. The formula in the current set 203 up of the driver was tuned to handle slaves with wildly different 204 bits-per-second "priorities". 205 206 207 All testing I have done was with two 28.8 V.FC modems, one connecting 208 at 28800 bps or slower, and the other connecting at 14400 bps all the 209 time. 210 211 212 One version of the scheduler was able to push 5.3 K/s through the 213 28800 and 14400 connections, but when the priorities on the links were 214 very wide apart (57600 vs. 14400) the "faster" modem received all 215 traffic and the "slower" modem starved. 216 217 2185. Testers' Reports 219=================== 220 221 Some people have experimented with the eql device with newer 222 kernels (than 1.1.75). I have since updated the driver to patch 223 cleanly in newer kernels because of the removal of the old "slave- 224 balancing" driver config option. 225 226 227 - icee from LinuxNET patched 1.1.86 without any rejects and was able 228 to boot the kernel and enslave a couple of ISDN PPP links. 229 2305.1. Randolph Bentson's Test Report 231----------------------------------- 232 233 :: 234 235 From bentson@grieg.seaslug.org Wed Feb 8 19:08:09 1995 236 Date: Tue, 7 Feb 95 22:57 PST 237 From: Randolph Bentson <bentson@grieg.seaslug.org> 238 To: guru@ncm.com 239 Subject: EQL driver tests 240 241 242 I have been checking out your eql driver. (Nice work, that!) 243 Although you may already done this performance testing, here 244 are some data I've discovered. 245 246 Randolph Bentson 247 bentson@grieg.seaslug.org 248 249------------------------------------------------------------------ 250 251 252 A pseudo-device driver, EQL, written by Simon Janes, can be used 253 to bundle multiple SLIP connections into what appears to be a 254 single connection. This allows one to improve dial-up network 255 connectivity gradually, without having to buy expensive DSU/CSU 256 hardware and services. 257 258 I have done some testing of this software, with two goals in 259 mind: first, to ensure it actually works as described and 260 second, as a method of exercising my device driver. 261 262 The following performance measurements were derived from a set 263 of SLIP connections run between two Linux systems (1.1.84) using 264 a 486DX2/66 with a Cyclom-8Ys and a 486SLC/40 with a Cyclom-16Y. 265 (Ports 0,1,2,3 were used. A later configuration will distribute 266 port selection across the different Cirrus chips on the boards.) 267 Once a link was established, I timed a binary ftp transfer of 268 289284 bytes of data. If there were no overhead (packet headers, 269 inter-character and inter-packet delays, etc.) the transfers 270 would take the following times:: 271 272 bits/sec seconds 273 345600 8.3 274 234600 12.3 275 172800 16.7 276 153600 18.8 277 76800 37.6 278 57600 50.2 279 38400 75.3 280 28800 100.4 281 19200 150.6 282 9600 301.3 283 284 A single line running at the lower speeds and with large packets 285 comes to within 2% of this. Performance is limited for the higher 286 speeds (as predicted by the Cirrus databook) to an aggregate of 287 about 160 kbits/sec. The next round of testing will distribute 288 the load across two or more Cirrus chips. 289 290 The good news is that one gets nearly the full advantage of the 291 second, third, and fourth line's bandwidth. (The bad news is 292 that the connection establishment seemed fragile for the higher 293 speeds. Once established, the connection seemed robust enough.) 294 295 ====== ======== === ======== ======= ======= === 296 #lines speed mtu seconds theory actual %of 297 kbit/sec duration speed speed max 298 ====== ======== === ======== ======= ======= === 299 3 115200 900 _ 345600 300 3 115200 400 18.1 345600 159825 46 301 2 115200 900 _ 230400 302 2 115200 600 18.1 230400 159825 69 303 2 115200 400 19.3 230400 149888 65 304 4 57600 900 _ 234600 305 4 57600 600 _ 234600 306 4 57600 400 _ 234600 307 3 57600 600 20.9 172800 138413 80 308 3 57600 900 21.2 172800 136455 78 309 3 115200 600 21.7 345600 133311 38 310 3 57600 400 22.5 172800 128571 74 311 4 38400 900 25.2 153600 114795 74 312 4 38400 600 26.4 153600 109577 71 313 4 38400 400 27.3 153600 105965 68 314 2 57600 900 29.1 115200 99410.3 86 315 1 115200 900 30.7 115200 94229.3 81 316 2 57600 600 30.2 115200 95789.4 83 317 3 38400 900 30.3 115200 95473.3 82 318 3 38400 600 31.2 115200 92719.2 80 319 1 115200 600 31.3 115200 92423 80 320 2 57600 400 32.3 115200 89561.6 77 321 1 115200 400 32.8 115200 88196.3 76 322 3 38400 400 33.5 115200 86353.4 74 323 2 38400 900 43.7 76800 66197.7 86 324 2 38400 600 44 76800 65746.4 85 325 2 38400 400 47.2 76800 61289 79 326 4 19200 900 50.8 76800 56945.7 74 327 4 19200 400 53.2 76800 54376.7 70 328 4 19200 600 53.7 76800 53870.4 70 329 1 57600 900 54.6 57600 52982.4 91 330 1 57600 600 56.2 57600 51474 89 331 3 19200 900 60.5 57600 47815.5 83 332 1 57600 400 60.2 57600 48053.8 83 333 3 19200 600 62 57600 46658.7 81 334 3 19200 400 64.7 57600 44711.6 77 335 1 38400 900 79.4 38400 36433.8 94 336 1 38400 600 82.4 38400 35107.3 91 337 2 19200 900 84.4 38400 34275.4 89 338 1 38400 400 86.8 38400 33327.6 86 339 2 19200 600 87.6 38400 33023.3 85 340 2 19200 400 91.2 38400 31719.7 82 341 4 9600 900 94.7 38400 30547.4 79 342 4 9600 400 106 38400 27290.9 71 343 4 9600 600 110 38400 26298.5 68 344 3 9600 900 118 28800 24515.6 85 345 3 9600 600 120 28800 24107 83 346 3 9600 400 131 28800 22082.7 76 347 1 19200 900 155 19200 18663.5 97 348 1 19200 600 161 19200 17968 93 349 1 19200 400 170 19200 17016.7 88 350 2 9600 600 176 19200 16436.6 85 351 2 9600 900 180 19200 16071.3 83 352 2 9600 400 181 19200 15982.5 83 353 1 9600 900 305 9600 9484.72 98 354 1 9600 600 314 9600 9212.87 95 355 1 9600 400 332 9600 8713.37 90 356 ====== ======== === ======== ======= ======= === 357 3585.2. Anthony Healy's Report 359--------------------------- 360 361 :: 362 363 Date: Mon, 13 Feb 1995 16:17:29 +1100 (EST) 364 From: Antony Healey <ahealey@st.nepean.uws.edu.au> 365 To: Simon Janes <guru@ncm.com> 366 Subject: Re: Load Balancing 367 368 Hi Simon, 369 I've installed your patch and it works great. I have trialed 370 it over twin SL/IP lines, just over null modems, but I was 371 able to data at over 48Kb/s [ISDN link -Simon]. I managed a 372 transfer of up to 7.5 Kbyte/s on one go, but averaged around 373 6.4 Kbyte/s, which I think is pretty cool. :) 374