Setting up an external load balancer

Learn how to configure an external load balancer for your master or proxy nodes in a high availability environment.

Master and proxy nodes in an IBM® Cloud Private high availability environment uses both ucarp and etcd as a load-balancer. With this setup, the virtual IP address (VIP) binds to one master or one proxy node.

You might also want to use an external load balancer as an alternative or replacement for the VIP.

To enable an external load balancer mode in an IBM Cloud Private high availability environment, you must prepare a load balancer node and install HAProxy. Then, configure the load balancer by setting the cluster_lb_address and the proxy_lb_address parameters in the config.yaml file.

The cluster external load balancer is used to load balance the IBM Cloud Private management services. The proxy external load balancer is used to load balance the IBM Cloud Private workload services.

To set up a cluster external load balancer, ensure that the following ports are added to the load balancer node and are open: 8001, 8443, 8500, 8600.
To set up a proxy external load balancer, ensure that the following ports are added to the load balancer node and are open: 80 and 443.

Set up the load balancer node. This load balancer node must not be shared with other cluster nodes such as master, worker, or proxy nodes. A dedicated node is needed to prevent port conflicts.

Install HAproxy on the load balancer node.

For Ubuntu:
```
apt-get install haproxy
```
Red Hat Enterprise Linux (RHEL):
```
yum install haproxy
```

Configure HAproxy. Configure HAproxy in the /etc/haproxy/haproxy.cfg file on the load balancer node. For example:

  # Example configuration for a possible web application.  See the
  # full configuration options online.
  #
  #   http://haproxy.1wt.eu/download/1.4/doc/configuration.txt
  #
  # Global settings
  global
  # To view messages in the /var/log/haproxy.log you need to:
  #
  # 1) Configure syslog to accept network log events.  This is done
  #    by adding the '-r' option to the SYSLOGD_OPTIONS in
  #    /etc/sysconfig/syslog.
  #
  # 2) Configure local2 events to go to the /var/log/haproxy.log
  #   file. A line similar to the following can be added to
  #   /etc/sysconfig/syslog.
  #
  #    local2.*                       /var/log/haproxy.log
  #
  log         127.0.0.1 local2

  chroot      /var/lib/haproxy
  pidfile     /var/run/haproxy.pid
  maxconn     4000
  user        haproxy
  group       haproxy
  daemon

  # 3) Turn on stats unix socket
  stats socket /var/lib/haproxy/stats
  # Common defaults that all the 'listen' and 'backend' sections
  # use, if not designated in their block.
  defaults
  mode                    http
  log                     global
  option                  httplog
  option                  dontlognull
  option http-server-close
  option                  redispatch
  retries                 3
  timeout http-request    10s
  timeout queue           1m
  timeout connect         10s
  timeout client          2m
  timeout server          2m
  timeout http-keep-alive 10s
  timeout check           10s
  maxconn                 3000

  frontend k8s-api
  bind *:8001
  mode tcp
  option tcplog
  use_backend k8s-api

  backend k8s-api
  mode tcp
  balance roundrobin
  server server1 <master_node_1_IP_address>:8001 check
  server server2 <master_node_2_IP_address>:8001 check
  server server3 <master_node_3_IP_address>:8001 check

  frontend dashboard
  bind *:8443
  mode tcp
  option tcplog
  use_backend dashboard

  backend dashboard
  mode tcp
  balance roundrobin
  server server1 <master_node_1_IP_address>:8443 check
  server server2 <master_node_2_IP_address>:8443 check
  server server3 <master_node_3_IP_address>:8443 check

  frontend registry
  bind *:8500
  mode tcp
  option tcplog
  use_backend registry

  frontend image-manager
  bind *:8600
  mode tcp
  option tcplog
  use_backend image-manager

  backend image-manager
  mode tcp
  balance roundrobin
  server server1 <master_node_1_IP_address>:8600 check
  server server2 <master_node_2_IP_address>:8600 check
  server server3 <master_node_3_IP_address>:8600 check

  backend registry
  mode tcp
  balance roundrobin
  server server1 <master_node_1_IP_address>:8500 check
  server server2 <master_node_2_IP_address>:8500 check
  server server3 <master_node_3_IP_address>:8500 check

  frontend proxy-http
  bind *:80
  mode tcp
  option tcplog
  use_backend proxy-http

  backend proxy-http
  mode tcp
  balance roundrobin
  server server1 <proxy_node_1_IP_address>:80 check
  server server2 <proxy_node_2_IP_address>:80 check
  server server3 <proxy_node_3_IP_address>:80 check

  frontend proxy-https
  bind *:443
  mode tcp
  option tcplog
  use_backend proxy-https

  backend proxy-https
  mode tcp
  balance roundrobin
  server server1 <proxy_node_1_IP_address>:443 check
  server server2 <proxy_node_2_IP_address>:443 check
  server server3 <proxy_node_3_IP_address>:443 check

  # OPTIONAL: Enable the following Kubernetes NodePorts for applications that require them:
  frontend proxy-nodeport
  bind *:30000-32767
  mode tcp
  option tcplog
  use_backend proxy-nodeport

  backend proxy-nodeport
  mode tcp
  balance roundrobin
  server server1 <proxy_node_1_IP_address>
  server server2 <proxy_node_2_IP_address>
  server server3 <proxy_node_3_IP_address>_

To set up a cluster load balancer, replace <master_node_1_IP_address>, <master_node_2_IP_address>, and <master_node_3_IP_address> with the IP addresses for your HA master nodes.
To set up a proxy load balancer, replace <proxy_node_1_IP_address>, <proxy_node_2_IP_address>, and <proxy_node_3_IP_address> with the IP addresses for your HA proxy nodes.

Start the haproxy service by running the following command on the load balancer node:
```
  systemctl start haproxy
```

Update the config.yaml file. Replace the cluster_lb_address or proxy_lb_address parameter with the IP address for your external load balancer node. For example:

 ## External loadbalancer IP or domain
 ## Or floating IP in OpenStack environment
 cluster_lb_address: none

 ## External loadbalancer IP or domain
 ## Or floating IP in OpenStack environment
 proxy_lb_address: none