Loopback Aliasing Alternatives when using MAC Forwarding for WebSphere® Edge Load Balancer on Linux™

Stand-alone server configuration

A stand-alone server is a server that is not installed on the same machine as the Load Balancer software. The following configuration steps are necessary on a stand-alone server.

1. Ensure that ARP suppression and reverse path filtering settings are set correctly.

   sysctl -w net.ipv4.conf.all.arp_ignore=3
   sysctl -w net.ipv4.conf.all.arp_announce=2
   sysctl -w net.ipv4.conf.all.rp_filter=2

2. The cluster address must be aliased to the loopback interface for stand-alone servers. A mask of 255.255.255.255 or a prefix of 32 must be used for ipv4 addresses. A prefix of 128 must be used for ipv6 addresses. The 'ifconfig' or the 'ip' operating system commands can be used to alias the cluster.

   The 'ip' command (recommended):

   ip addr add $CLUSTER_ADDRESS/32 scope host dev lo

   Or

   The 'ifconfig' command (alternative):

   ifconfig lo:1 $CLUSTER_ADDRESS netmask 255.255.255.255 up

   Note: The alias number (:1) shown after the interface must be unique. If multiple clusters must be configured, ensure unique alias numbers (:1) are used for each alias.

   Avoid trouble: Ensure that the alias persists when the system is rebooted. Consult your operating system documentation for necessary procedures. A common method is to add the cluster alias to the /etc/sysconfig/network-scripts/ifcfg-lo file.

   DEVICE=lo
   IPADDR=127.0.0.1
   NETMASK=255.0.0.0
   NETWORK=127.0.0.0
   IPADDR2=$CLUSTER_ADDRESS
   NETMASK2=255.255.255.255

Collocated server configuration

A collocated server is a server that is installed on the same machine as the Load Balancer software. Collocated servers are not recommended but can be configured.

When the server definition is set as collocated, the Load Balancer creates a virtual interface named ibmulb. Packets the Load Balancer forwarded to this server are not transmitted on the ethernet interface. Instead, the packets are directed to the ibmulb interface. When the Load Balancer's executor is stopped, the ibmulb interface is deleted.

The cluster address must be aliased on the ibmulb interface not the loopback interface.

Since the cluster address is aliased on an operating system interface, the operating system receives a copy of all inbound cluster packets. A filter must be added to block the operating system from receiving copies of cluster packets. If a filter is not added, connections forwarded to stand-alone servers fail because the clients two different responses, one from the collocated server and one from the stand-alone server. To create the necessary filter, firewall software must present.

Steps to configure a collocated server:

1. Ensure correct ARP suppression settings. Define loose routing for the reverse path filtering setting.

   sysctl -w net.ipv4.conf.all.arp_ignore=3
   sysctl -w net.ipv4.conf.all.arp_announce=2
   sysctl -w net.ipv4.conf.all.rp_filter=2

2. Install and load firewall software (if necessary):

   Installation (if necessary):
   Ubuntu:

   sudo apt-get install iptables 

   Red Hat:

   dnf install iptables-services 

   
   Load the module:

   modprobe ip_tables

3. Create a filter to block cluster traffic on the ethernet interface. If cluster traffic can be received on multiple interfaces, all interfaces that can receive cluster traffic must filter the traffic. Existing filters can affect the placement of the new filter. The examples provided append the rule to filter cluster traffic to the end of existing filters. Examine your current filters to determine correct placement. Filter management can be performed by using operating system tools such as iptables, iptables6, firewalld, nft, netfilter, or other firewall software. Examples are provided for the iptables and nft software.

   iptables:
   Display the current filters:

   # iptables -L INPUT
   Chain INPUT (policy ACCEPT)
   prot opt source          destination

   Append the cluster filter:

   # iptables -A INPUT -i $INTERFACE -d $CLUSTER/32 -j DROP
   
   # iptables -L INPUT
   Chain INPUT (policy ACCEPT)
   target     prot opt source               destination
   DROP       all  --  anywhere             

   nft:
   Display the current filters:

   # nft list table inet filter
   table inet filter {
     chain input {
        type filter hook input priority 0; policy accept;
     }
   
     chain forward {
        type filter hook forward priority 0; policy accept;
     }
   
     chain output {
        type filter hook output priority 0; policy accept;
     }
   } 

   Append the cluster filter:

   # nft add rule inet filter input iifname "$INTERFACE" ip daddr $CLUSTER counter drop
   
   # nft list table inet filter
   table inet filter {
     chain input {
        type filter hook input priority 0; policy accept;
        iifname "$INTERFACE" ip daddr $CLUSTER counter packets 0 bytes 0 drop
     }
   
     chain forward {
        type filter hook forward priority 0; policy accept;
     }
   
     chain output {
       type filter hook output priority 0; policy accept;
     }
   }

4. Define the collocated server to the Load Balancer and set the collocation flag:

   dscontrol server add $CLUSTER@$PORT@$SERVER
   dscontrol server set $CLUSTER@$PORT@$SERVER collocated yes 

5. Alias the cluster address on the ibmulb interface and set the interface UP:

   The ip command (recommended):

   ip addr add $CLUSTER/32 dev ibmulb scope host
   ip link set dev ibmulb up

   Or

   The ifconfig command (alternative):

   ifconfig ibmulb:1 $CLUSTER netmask 255.255.255.255 up

   Note: The alias number (:1) shown after the interface must be unique. If multiple clusters must be configured, ensure that each cluster alias has a unique alias number.

6. If the reverse path filtering setting was disabled instead of set to loose routing, you must disable the reverse path filtering setting on the new ibmulb interface:

   sysctl -w net.ipv4.conf.ibmulb.rp_filter=0

The collocated server configuration changes based on the Load Balancer high availability state. When the Load Balancer is actively forwarding traffic, the server must be configured as a collocated server. However, when the Load Balancer is in the backup state, the collocated server must be configured as a stand-alone server. The Load Balancer provides user exits that are invoked during high availability state changes to make the necessary configuration changes.

Servers must be defined before high availability is defined. The Load Balancer 'go' scripts execute after high availability state changes. If high availability is defined before the server is defined, the ibmulb tunnel does not exist when the state changes. After the tunnel is created, necessary configuration is not performed.

The user exits must be named goActive, goStandby, and goInOp.  Samples are provided in the /opt/IBM/WebSphere/Edge/ULB/servers/samples directory. These files must be edited to use the correct interface and cluster addresses. The user exits are invoked in the /opt/IBM/WebSphere/Edge/ULB/servers/bin directory with execute permission set for the Load Balancer user.

1. Ensure that ARP suppression settings are correct and reverse path filtering is set for loose routing.

   sysctl -w net.ipv4.conf.all.arp_ignore=3
   sysctl -w net.ipv4.conf.all.arp_announce=2
   sysctl -w net.ipv4.conf.all.rp_filter=2

2. Customize the goActive, goStandby, and goInOp scripts. Place your customized scripts in the /opt/IBM/WebSphere/Edge/ULB/servers/bin directory and set execute permission on the files.  

   Avoid trouble: The file names must use the exact uppercase and lowercase lettering in the file name. The scripts have no file extension in the name. Cluster traffic must be filtered and dropped by the ethernet interface.

   goActive:
   When the Load Balancer enters the active forwarding state, the goActive script is invoked. The server on the Load Balancer machine must be configured as a collocated server when the Load Balancer is actively forwarding traffic. Remove the cluster address from the loopback interface and alias the cluster to the ibmulb tunnel. Add a filter to block the ethernet interface from receiving cluster traffic.

   Sample goActive script:

   # Modify CLUSTER and INTERFACE to match your environment
   
   CLUSTER=4.3.2.1
   INTERFACE=eth0
   
   # Add filter to block OS from receiving cluster traffic
   iptables -A INPUT -i $INTERFACE -d $CLUSTER/32 -j DROP
   
   # Remove cluster alias from loopback
   ip addr del $CLUSTER/32 scope host dev lo
   
   # Add cluster alias to tunnel
   ip addr add $CLUSTER/32 dev ibmulb scope host
   
   # Set tunnel up
   ip link set ibmulb up

   goStandby:
   When the Load Balancer enters the standby state, the goStandby script is invoked. The Load Balancer does not forward traffic in the standby state. The partner Load Balancer is forwarding traffic. As such, the server must be configured as a stand-alone server when the collocated Load Balancer is in the standby state. Remove the filter to block cluster traffic on the ethernet interface. Remove the cluster alias from the ibmulb interface and add the cluster alias to the ethernet interface.

   Note: Do not modify the server definition on the Load Balancer machine. The definition on the standby Load Balancer does not change and the server remains marked as collocated.

   Sample goStandby script:

   # Modify CLUSTER and INTERFACE to match your environment
   CLUSTER=4.3.2.1
   INTERFACE=eth0
   
   # Remove filter which blocks cluster traffic to OS
   iptables -D INPUT -i $INTERFACE -d $CLUSTER/32 -j DROP
   
   # Set collocated tunnel down
   ip link set ibmulb down
   
   # Remove cluster alias from tunnel
   ip addr del $CLUSTER/32 dev ibmulb scope host
   
   # Add cluster alias to loopback
   ip addr add $CLUSTER/32 scope host dev lo

   goInOp:
   When the Load Balancer's executor stops, the goInOp script is invoked. The Load Balancer is no longer forwarding traffic. The only traffic that the server receives is from the partner Load Balancer. The server must be configured as a stand-alone server.

   Sample goInOp script:

   # Modify CLUSTER and INTERFACE to match your environment
   CLUSTER=4.3.2.1
   INTERFACE=eth0
   
   # Remove filter which blocks cluster traffic to OS
   iptables -D INPUT -i $INTERFACE -d $CLUSTER/32 -j DROP
   
   # Set collocated tunnel down
   ip link set ibmulb down
   
   # Remove cluster alias from tunnel
   ip addr del $CLUSTER/32 dev ibmulb scope host
   
   # Add cluster alias to loopback
   ip addr add $CLUSTER/32 scope host dev lo 

3. Add the collocated server before high availability is defined. Set the collocated flag on the servers.

   dscontrol server add <cluster>@<port>@<server>
   dscontrol server set <cluster>@<port>@<server> collocated yes
   . . .
   dscontrol high availability add...