Lines Matching +full:switch +full:- +full:mode
7 Corrections, HA extensions : 2000/10/03-15 :
8 - Willy Tarreau <willy at meta-x.org>
9 - Constantine Gavrilov <const-g at xpert.com>
10 - Chad N. Tindel <ctindel at ieee dot org>
11 - Janice Girouard <girouard at us dot ibm dot com>
12 - Jay Vosburgh <fubar at us dot ibm dot com>
16 - Mitch Williams <mitch.a.williams at intel.com>
23 The behavior of the bonded interfaces depends upon the mode; generally
29 the original tools from extreme-linux and beowulf sites will not work
54 3.7 Configuring LACP for 802.3ad mode in a more secure way
60 5. Switch Configuration
76 10. Promiscuous mode
79 11.1 High Availability in a Single Switch Topology
80 11.2 High Availability in a Multiple Switch Topology
81 11.2.1 HA Bonding Mode Selection for Multiple Switch Topology
82 11.2.2 HA Link Monitoring for Multiple Switch Topology
85 12.1 Maximum Throughput in a Single Switch Topology
86 12.1.1 MT Bonding Mode Selection for Single Switch Topology
87 12.1.2 MT Link Monitoring for Single Switch Topology
88 12.2 Maximum Throughput in a Multiple Switch Topology
89 12.2.1 MT Bonding Mode Selection for Multiple Switch Topology
90 12.2.2 MT Link Monitoring for Multiple Switch Topology
92 13. Switch Behavior Issues
114 -----------------------------------------------
130 -------------------------------------
143 /etc/modprobe.d/*.conf configuration files, or in a distro-specific
151 configuring a bond, it is recommended "tail -f /var/log/messages" be
161 "mode=802.3ad" and "mode=4" set the same mode.
168 (active-backup, balance-alb and balance-tlb). Possible values
180 the current mode does not use an active slave.
185 is 1 - 65535. If the value is not specified, it takes 65535 as the
188 This parameter has effect only in 802.3ad mode and is available through
193 In an AD system, this specifies the mac-address for the actor in
195 multicast. It is preferred to have the local-admin bit set for this
200 This parameter has effect only in 802.3ad mode and is available through
224 - A slave is added to or removed from the bond
226 - Any slave's link state changes
228 - Any slave's 802.3ad association state changes
230 - The bond's administrative state changes to up
247 In an AD system, the port-key has three parts as shown below -
251 01-05 Speed
252 06-15 User-defined
255 from 0 - 1023. If not given, the system defaults to 0.
257 This parameter has effect only in 802.3ad mode and is available through
279 bonding mode, and the state of the slave). Regular traffic is
286 If ARP monitoring is used in an etherchannel compatible mode
287 (modes 0 and 2), the switch should be configured in a mode
289 switch is configured to distribute the packets in an XOR
310 validated in any mode that supports arp monitoring, or whether
311 non-ARP traffic should be filtered (disregarded) for link
357 active slave. It is possible that some switch or network
369 beyond a common switch. Should the link between the switch and
370 target fail (but not the switch itself), the probe traffic
401 This option affects only active-backup mode for slaves with
427 Specifies whether active-backup mode should set all slaves to
437 bonding to set all slaves of an active-backup bond to
456 vs. just updating a switch or set of switches (which
458 traffic, if the switch snoops incoming traffic to
499 to transmit LACPDU packets in 802.3ad mode. Possible values
531 asserting carrier. It is similar to the Cisco EtherChannel min-links
533 must be up (link-up state) before marking the bond device as up
537 mode.
540 802.3ad mode) whenever there is an active aggregator, regardless of the
545 mode
548 balance-rr (round robin). Possible values are:
550 balance-rr or 0
552 Round-robin policy: Transmit packets in sequential
554 last. This mode provides load balancing and fault
557 active-backup or 1
559 Active-backup policy: Only one slave in the bond is
563 to avoid confusing the switch.
566 occurs in active-backup mode, bonding will issue one
574 This mode provides fault tolerance. The primary
576 mode.
578 balance-xor or 2
587 This mode provides load balancing and fault tolerance.
592 interfaces. This mode provides fault tolerance.
606 regards to the packet mis-ordering requirements of
616 2. A switch that supports IEEE 802.3ad Dynamic link
620 to enable 802.3ad mode.
622 balance-tlb or 5
625 does not require any special switch support.
627 In tlb_dynamic_lb=1 mode; the outgoing traffic is
631 In tlb_dynamic_lb=0 mode; the load balancing based on
644 balance-alb or 6
646 Adaptive load balancing: includes balance-tlb plus
648 does not require any special switch support. The
674 and when an inactive slave is re-activated. The
683 be set to a value equal or greater than the switch's
685 peers will not be blocked by the switch.
709 bonding device and each VLAN sub-device. This is repeated at
713 The valid range is 0 - 255; the default value is 1. These options
714 affect only the active-backup mode. These options were added for
727 The valid range is 0 - 65535; the default value is 1. This option
728 has effect only in balance-rr mode.
735 off-line will alternate devices be used. This is useful when
739 The primary option is only valid for active-backup(1),
740 balance-tlb (5) and balance-alb (6) mode.
747 occurs. This option is designed to prevent flip-flopping between
785 mode. The value has no effect on any other modes.
787 The default behavior of tlb mode is to shuffle active flows across
789 characteristics but can cause packet reordering. If re-ordering is
792 xmit-hash-policy can be used to select the appropriate hashing for
837 balance-xor, 802.3ad, and tlb modes. Possible values are:
870 network peer on the same slave. For non-IP traffic,
902 information is omitted. For non-IP traffic, the
947 The valid range is 0 - 255; the default value is 1. A value of 0
951 This option is useful for bonding modes balance-rr (0), active-backup
952 (1), balance-tlb (5) and balance-alb (6), in which a failover can
953 switch the IGMP traffic from one slave to another. Therefore a fresh
954 IGMP report must be issued to cause the switch to forward the incoming
962 driver sends learning packets to each slaves peer switch.
964 The valid range is 1 - 0x7fffffff; the default value is 1. This Option
965 has effect only in balance-tlb and balance-alb modes.
993 $ rpm -qf /sbin/ifup
1008 ----------------------------------------
1021 ifcfg-id file for each slave device. The simplest way to accomplish
1023 file ifcfg-id file created; see below for some issues with DHCP). The
1026 ifcfg-id-xx:xx:xx:xx:xx:xx
1031 Once the set of ifcfg-id-xx:xx:xx:xx:xx:xx files has been
1041 _nm_name='bus-pci-0001:61:01.0'
1051 Once the ifcfg-id-xx:xx:xx:xx:xx:xx files have been modified,
1053 itself. This file is named ifcfg-bondX, where X is the number of the
1055 ifcfg-bond0, the second is ifcfg-bond1, and so on. The sysconfig
1059 The contents of the ifcfg-bondX file is as follows:
1069 BONDING_MODULE_OPTS="mode=active-backup miimon=100"
1071 BONDING_SLAVE1="bus-pci-0000:06:08.1"
1097 for the bonding mode, link monitoring, and so on here. Do not include
1107 (bus-pci-0000:06:08.1 in the example above) specify the physical
1138 -------------------------------
1148 -----------------------------------------------
1152 bonding instance to have an appropriately configured ifcfg-bondX file
1156 ifcfg-bondX files.
1159 options in the ifcfg-bondX file, it is not necessary to add them to
1163 ------------------------------------------
1176 network-script file for all physical adapters that will be members of
1179 /etc/sysconfig/network-scripts
1181 The file name must be prefixed with "ifcfg-eth" and suffixed
1183 for eth0 would be named /etc/sysconfig/network-scripts/ifcfg-eth0.
1194 must correspond with the name of the file, i.e., ifcfg-eth1 must have
1202 script will be /etc/sysconfig/network-scripts/ifcfg-bondX where X is
1203 the number of the bond. For bond0 the file is named "ifcfg-bond0",
1204 for bond1 it is named "ifcfg-bond1", and so on. Within that file,
1221 and, indeed, preferable, to specify the bonding options in the ifcfg-bond0
1224 BONDING_OPTS="mode=active-backup arp_interval=60 arp_ip_target=192.168.1.254"
1246 options bond0 mode=balance-alb miimon=100
1256 ---------------------------------
1269 -------------------------------------------------
1273 specifying the appropriate BONDING_OPTS= in ifcfg-bondX where X is the
1281 -----------------------------------------------
1300 modprobe bonding mode=balance-alb miimon=100
1323 enabled without re-running the entire global init script.
1339 -----------------------------------------
1368 options bond0 -o bond0 mode=balance-rr miimon=100
1371 options bond1 -o bond1 mode=balance-alb miimon=50
1374 named "bond0" and creates the bond0 device in balance-rr mode with an
1376 bond1 device in balance-alb mode with an miimon of 50.
1383 install bond1 /sbin/modprobe --ignore-install bonding -o bond1 \
1384 mode=balance-alb miimon=50
1390 to rename modules at load time (the "-o bond1" part). Attempts to pass
1398 ------------------------------------------
1418 -----------------------------
1423 # echo -bar > /sys/class/net/bonding_masters
1433 --------------------------
1443 # echo -eth0 > /sys/class/net/bond0/bonding/slaves
1452 # echo -eth0 > /sys/class/net/eth0/master/bonding/slaves
1457 -------------------------------
1461 The names of these files correspond directly with the command-
1470 To configure bond0 for balance-alb mode:
1472 # echo 6 > /sys/class/net/bond0/bonding/mode
1473 - or -
1474 # echo balance-alb > /sys/class/net/bond0/bonding/mode
1475 NOTE: The bond interface must be down before the mode can be
1481 monitoring is enabled, and vice-versa.
1489 # echo -192.168.0.100 > /sys/class/net/bond0/bonding/arp_ip_target
1494 the bonding driver sends learning packets to each slaves peer switch. The
1498 ---------------------
1509 echo balance-alb > /sys/class/net/bond0/bonding/mode
1516 active-backup mode, using ARP monitoring, add the following lines to
1521 echo active-backup > /sys/class/net/bond1/bonding/mode
1529 -----------------------------------------
1536 the box. The ifenslave-2.6 package should be installed to provide bonding
1537 support. Once installed, this package will provide bond-* options to be used
1540 Note that ifenslave-2.6 package will load the bonding module and use
1544 ----------------------
1547 active-backup mode, with eth0 and eth1 as slaves.
1551 bond-slaves eth0 eth1
1552 bond-mode active-backup
1553 bond-miimon 100
1554 bond-primary eth0 eth1
1563 bond-slaves none
1564 bond-mode active-backup
1565 bond-miimon 100
1569 bond-master bond0
1570 bond-primary eth0 eth1
1574 bond-master bond0
1575 bond-primary eth0 eth1
1577 For a full list of bond-* supported options in /etc/network/interfaces and some
1579 /usr/share/doc/ifenslave-2.6.
1582 ----------------------------------------------
1587 the selected bonding mode. On occasion however, it is helpful to direct certain
1605 Bonding Mode: fault-tolerance (active-backup)
1656 a pass-through for selecting output queues on the slave device rather than
1660 output slave selection was limited to round-robin and active-backup modes.
1662 3.7 Configuring LACP for 802.3ad mode in a more secure way
1663 ----------------------------------------------------------
1665 When using 802.3ad bonding mode, the Actor (host) and Partner (switch)
1671 domain can spoof LACPDU packets from other hosts to the switch and potentially
1672 cause mayhem by joining (from the point of view of the switch) another
1679 (a) ad_actor_system : You can set a random mac-address that can be used for
1681 Also it's preferable to set the local-admin bit. Following shell code
1682 generates a random mac-address as described above.
1694 is 65535, but system can take the value from 1 - 65535. Following shell
1700 (c) ad_user_port_key : Use the user portion of the port-key. The default
1701 keeps this empty. These are the upper 10 bits of the port-key and value
1702 ranges from 0 - 1023. Following shell code generates these 10 bits and
1713 -------------------------
1715 Each bonding device has a read-only file residing in the
1720 driver is loaded with parameters of mode=0 and miimon=1000 is
1724 Bonding Mode: load balancing (round-robin)
1743 -------------------------
1777 5. Switch Configuration
1780 For this section, "switch" refers to whatever system the
1782 the cable plugs into). This may be an actual dedicated switch device,
1786 The active-backup, balance-tlb and balance-alb modes do not
1787 require any specific configuration of the switch.
1789 The 802.3ad mode requires that the switch have the appropriate
1791 to configure this varies from switch to switch, but, for example, a
1792 Cisco 3550 series switch requires that the appropriate ports first be
1794 etherchannel is set to mode "lacp" to enable 802.3ad (instead of
1797 The balance-rr, balance-xor and broadcast modes generally
1798 require that the switch have the appropriate ports grouped together.
1801 group" or some other similar variation. For these modes, each switch
1802 will also have its own configuration options for the switch's transmit
1805 match. For these three modes, the bonding mode really selects a
1817 packets generated by either ALB mode or the ARP monitor mechanism, are
1829 "un-accelerated" by the bonding driver so the VLAN tag sits in the
1836 would pick up the all-zeroes hardware address. Once the first slave
1857 underlying device -- i.e. the bonding interface -- to promiscuous
1858 mode, which might not be what you want.
1873 -------------------------
1883 date the last receive time, dev->last_rx. Drivers that use NETIF_F_LLTX
1884 flag must also update netdev_queue->trans_start. If they do not, then the
1891 ------------------------------------
1914 -------------------------
1943 -------------------------
1978 ----------------------------
1989 options bond0 mode=some-mode miimon=50
2022 ---------------------------------------------------------
2046 beyond other ports of a switch, or if a switch is refusing to pass
2095 10. Promiscuous mode
2099 common to enable promiscuous mode on the device, so that all traffic
2101 The bonding driver handles promiscuous mode changes to the bonding
2105 For the balance-rr, balance-xor, broadcast, and 802.3ad modes,
2106 the promiscuous mode setting is propagated to all slaves.
2108 For the active-backup, balance-tlb and balance-alb modes, the
2109 promiscuous mode setting is propagated only to the active slave.
2111 For balance-tlb mode, the active slave is the slave currently
2114 For balance-alb mode, the active slave is the slave used as a
2115 "primary." This slave is used for mode-specific control traffic, for
2118 For the active-backup, balance-tlb and balance-alb modes, when
2132 11.1 High Availability in a Single Switch Topology
2133 --------------------------------------------------
2135 If two hosts (or a host and a single switch) are directly
2138 only one switch (or peer), so if it fails, there is no alternative
2146 11.2 High Availability in a Multiple Switch Topology
2147 ----------------------------------------------------
2150 network changes dramatically. In multiple switch topologies, there is
2158 +-----+----+ +-----+----+
2160 | switch A +--------------------------+ switch B |
2162 +-----+----+ +-----++---+
2164 | +-------+ |
2165 +-------------+ host1 +---------------+
2166 eth0 +-------+ eth1
2169 switches (ISL, or inter switch link), and multiple ports connecting to
2170 the outside world ("port3" on each switch). There is no technical
2171 reason that this could not be extended to a third switch.
2173 11.2.1 HA Bonding Mode Selection for Multiple Switch Topology
2174 -------------------------------------------------------------
2176 In a topology such as the example above, the active-backup and
2181 active-backup: This is generally the preferred mode, particularly if
2183 network configuration is such that one switch is specifically
2184 a backup switch (e.g., has lower capacity, higher cost, etc),
2188 broadcast: This mode is really a special purpose mode, and is suitable
2192 necessary for some specific one-way traffic to reach both
2193 independent networks, then the broadcast mode may be suitable.
2195 11.2.2 HA Link Monitoring Selection for Multiple Switch Topology
2196 ----------------------------------------------------------------
2199 switch. If the switch can reliably fail ports in response to other
2204 thus detecting that failure without switch support.
2206 In general, however, in a multiple switch topology, the ARP
2211 one for each switch in the network). This will insure that,
2212 regardless of which switch is active, the ARP monitor has a suitable
2217 switch that causes the link state of a particular switch port to be set
2218 down (or up) when the state of another switch port goes down (or up).
2222 switch, but this can be a viable alternative to the ARP monitor when using
2228 12.1 Maximizing Throughput in a Single Switch Topology
2229 ------------------------------------------------------
2231 In a single switch configuration, the best method to maximize
2240 In a gatewayed configuration, the "switch" is acting primarily
2245 +----------+ +----------+
2247 | Host A +---------------------+ router +------------------->
2248 | +---------------------+ | Hosts B and C are out
2250 +----------+ +----------+
2267 In a local configuration, the "switch" is acting primarily as
2268 a switch, and the majority of traffic passes through this switch to
2272 +----------+ +----------+ +--------+
2273 | |eth0 port1| +-------+ Host B |
2274 | Host A +------------+ switch |port3 +--------+
2275 | +------------+ | +--------+
2276 | |eth1 port2| +------------------+ Host C |
2277 +----------+ +----------+port4 +--------+
2280 Again, the switch may be a dedicated switch device, or another
2296 mode is described below.
2299 12.1.1 MT Bonding Mode Selection for Single Switch Topology
2300 -----------------------------------------------------------
2303 although you will have to decide which bonding mode best suits your
2304 needs. The trade offs for each mode are detailed below:
2306 balance-rr: This mode is the only mode that will permit a single
2308 interfaces. It is therefore the only mode that will allow a
2323 networking interfaces, the switch, and the topology of the
2332 through the switch to a balance-rr bond will not utilize greater
2337 delivery, then this mode can allow for single stream datagram
2341 This mode requires the switch to have the appropriate ports
2344 active-backup: There is not much advantage in this network topology to
2345 the active-backup mode, as the inactive backup devices are all
2347 load balancing mode (with link monitoring) will provide the
2349 available bandwidth. On the plus side, active-backup mode
2350 does not require any configuration of the switch, so it may
2354 balance-xor: This mode will limit traffic such that packets destined
2357 addresses involved, this mode works best in a "local" network
2359 the same local network. This mode is likely to be suboptimal
2363 As with balance-rr, the switch ports need to be configured for
2366 broadcast: Like active-backup, there is not much advantage to this
2367 mode in this type of network topology.
2369 802.3ad: This mode can be a good choice for this type of network
2370 topology. The 802.3ad mode is an IEEE standard, so all peers
2373 so minimal manual configuration of the switch is needed
2378 packets. The 802.3ad mode does have some drawbacks: the
2381 balance modes other than balance-rr, no single connection will
2394 Finally, the 802.3ad mode mandates the use of the MII monitor,
2395 therefore, the ARP monitor is not available in this mode.
2397 balance-tlb: The balance-tlb mode balances outgoing traffic by peer.
2399 "gatewayed" configuration (as described above), this mode will
2401 "local" network configuration, this mode balances multiple
2403 manner (not a simple XOR as in balance-xor or 802.3ad mode),
2409 special switch configuration is required. On the down side,
2410 in this mode all incoming traffic arrives over a single
2411 interface, this mode requires certain ethtool support in the
2415 balance-alb: This mode is everything that balance-tlb is, and more.
2416 It has all of the features (and restrictions) of balance-tlb,
2421 The only additional down side to this mode is that the network
2425 12.1.2 MT Link Monitoring for Single Switch Topology
2426 ----------------------------------------------------
2429 mode you choose to use. The more advanced load balancing modes do not
2434 12.2 Maximum Throughput in a Multiple Switch Topology
2435 -----------------------------------------------------
2441 +-----------+
2443 +-+---+---+-+
2445 +--------+ | +---------+
2447 +------+---+ +-----+----+ +-----+----+
2448 | Switch A | | Switch B | | Switch C |
2449 +------+---+ +-----+----+ +-----+----+
2451 +--------+ | +---------+
2453 +-+---+---+-+
2455 +-----------+
2461 cost effective than a single larger switch, e.g., on a network with 24
2463 a single 72 port switch.
2469 12.2.1 MT Bonding Mode Selection for Multiple Switch Topology
2470 -------------------------------------------------------------
2472 In actual practice, the bonding mode typically employed in
2473 configurations of this type is balance-rr. Historically, in this
2478 packets has arrived). When employed in this fashion, the balance-rr
2479 mode allows individual connections between two hosts to effectively
2482 12.2.2 MT Link Monitoring for Multiple Switch Topology
2483 ------------------------------------------------------
2492 13. Switch Behavior Issues
2496 -------------------------------------------
2499 timing of link up and down reporting by the switch.
2505 during switch initialization (e.g., during recovery after a switch
2512 the switch is initializing. Again, an appropriate updelay value may
2525 In addition to the concerns about switch timings, if your
2526 switches take a long time to go into backup mode, it may be desirable
2531 --------------------------------
2543 For example, on a bond in active-backup mode with five slaves
2544 all connected to one switch, the output may appear as follows:
2546 # ping -n 10.0.4.2
2559 tables. Initially, the switch does not associate the MAC address in
2560 the packet with a particular switch port, and so it may send the
2563 single switch, when the switch (temporarily) floods the traffic to all
2567 The duplicated packet behavior is switch dependent, some
2570 most Cisco switches, the privileged command "clear mac address-table
2581 --------------------
2586 balance-rr, active-backup, balance-tlb and balance-alb modes. This is
2591 --------------------------------
2594 integrated on the planar (that's "motherboard" in IBM-speak). In the
2597 An add-on Broadcom daughter card can be installed on a JS20 to provide
2601 Each I/O Module may contain either a switch or a passthrough
2603 switch). Some bonding modes require a specific BladeCenter internal
2606 Additional BladeCenter-specific networking information can be
2610 "IBM eServer BladeCenter Layer 2-7 Network Switching"
2613 ------------------------------------
2619 Normally, Ethernet Switch Modules (ESMs) are used in I/O
2626 switch. By using PMs in I/O module #1 and #2, the eth0 and eth1
2628 connected to a common external switch.
2631 appear to bonding as either a single switch topology (all PMs) or as a
2632 multiple switch topology (one or more ESMs, zero or more PMs). It is
2634 much like the example in "High Availability in a Multiple Switch
2638 -------------------------------
2640 The balance-rr mode requires the use of passthrough modules
2641 for devices in the bond, all connected to an common external switch.
2642 That switch must be configured for "etherchannel" or "trunking" on the
2643 appropriate ports, as is usual for balance-rr.
2645 The balance-alb and balance-tlb modes will function with
2646 either switch modules or passthrough modules (or a mix). The only
2652 The active-backup mode has no additional requirements.
2655 ----------------------
2657 When an Ethernet Switch Module is in place, only the ARP
2658 monitor will reliably detect link loss to an external switch. This is
2661 the system, when it fact there is a switch between these "external"
2670 --------------
2678 It may be desirable to disable spanning tree on the switch
2679 (either the internal Ethernet Switch Module, or an external switch) to
2680 avoid fail-over delay issues when using bonding.
2693 Any Ethernet type cards (you can even mix cards - a Intel
2698 slaves in active-backup mode.
2713 disabled. The active-backup mode will fail over to a backup link, and
2716 manner is appropriate for the mode). See the sections on High
2717 Availability and the documentation for each mode for additional
2730 depends upon the bonding mode and network configuration.
2738 The full answer to this depends upon the desired mode.
2740 In the basic balance modes (balance-rr and balance-xor), it
2745 The advanced balance modes (balance-tlb and balance-alb) do
2746 not have special switch requirements, but do need device drivers that
2750 In 802.3ad mode, it works with systems that support IEEE
2754 The active-backup mode should work with any Layer-II switch.
2778 # ifconfig bond0 down ; modprobe -r bonding
2786 from the bond (`ifenslave -d bond0 eth0'). The bonding driver will
2800 bonding-devel mailing list, hosted at sourceforge.net. If you have questions or
2803 bonding-devel@lists.sourceforge.net
2808 https://lists.sourceforge.net/lists/listinfo/bonding-devel
2819 http://vger.kernel.org/vger-lists.html#netdev
2822 - http://web.archive.org/web/*/http://www.scyld.com/network/
2827 -- END --