IBM Support

QRadar: Unable to SSH to High Availability Appliance

Troubleshooting


Problem

I cannot SSH from primary to secondary appliances in High Availability (HA).
 

Symptom

  1. When adding High Availability (HA) to the Data Node, the HA wizard showed a failure to SSH, suggesting invalid password may have been used.
  2. When trying to test the Crossover by typing the command
    /opt/qradar/ha/bin/qradar_nettune.pl crossover status
    You see this message
    Crossover status: configured/disabled
        Role: [primary/secondary]
        Admin status: disabled
        Operative status: stopped
        Interface: ens4
        Interface status: UP
        Interface MTU: 9000
        Firewall status: enabled
        Routing status: disabled
    This message indicates that the crossover is configured but has stopped and it is not sending data.

Cause

The cause of this issue is that the MTU may be set too high on both appliances.
Note: The default is 9000 MTU.
 
image-20190502133745-2

Diagnosing The Problem

Typing the command /opt/qradar/ha/bin/qradar_nettune.pl crossover test on each HA node might sometimes indicate what to set the MTU value for your network.
/opt/qradar/ha/bin/qradar_nettune.pl crossover test
Crossover IPs: 192.168.0.81 -> 192.168.0.83
Testing crossover (default): 192.168.0.81 -> 192.168.0.83
ping -c 20 -s 8900 -M do 192.168.0.83
PING 192.168.0.83 (192.168.0.83) 8900(8928) bytes of data.
ping: local error: Message too long, mtu=1500
ping: local error: Message too long, mtu=1500
ping: local error: Message too long, mtu=1500
In this example qradar_nettune.pl crossover test is sending a signal over the management interface and is showing the MTU of the LAN cards installed. In cases where you are using your hardware, or Virtual Machines (VM)'s the result may be different. Under these conditions, it would be advisable to lower the MTU values to 1500 and raise it till the crossover fails to connect using an SSH session.

Resolving The Problem

Before you begin:

Make sure you have a remote management interface configured such as IMM or a VM Console. Do not modify the Management Interface configuration file. From the screenshot in the Cause section of this article, we are using the interface ens4. As part of this solution, you will need to restart network services on each HA node. Restarting might briefly interfere with the HA pair logging. Please schedule maintenance period before restarting network services.
To resolve this issue.
  1. Log in to the Console using an SSH session as the root user.
    Note: If the appliance with the HA crossover issues is not the Console, use an SSH session to connect to the appliance that is having issues.
  2. Verify which connection is the Management Interface by running the command:
    grep "MGMT_INTERFACE" /opt/qradar/conf/nva.hostcontext.conf

    Example:
    grep  "MGMT_INTERFACE" /opt/qradar/conf/nva.hostcontext.conf
    MGMT_INTERFACE=ens3
    Note: Do not modify this interface connection.
  3. Verify the backup directory exists
    mkdir -p /store/IBM_Support
  4. Change the directory to /etc/sysconfig/network-scripts
    cd /etc/sysconfig/network-scripts
  5. Make a backup of the interface configuration file you are modifying. In this example its ifcfg-ens4.
    cp ifcfg-ens4 /store/IBM_Support
  6. Using an editor such as vi to edit the configuration file by typing:
    vi ifcfg-ens4
  7. Reduce the MTU from 9000 to the lower value. For our example, we want to change MTU from 9000 to 1500.
    image-20190502141415-2
    If the setting MTU = is not in the configuration file, then add the variable.
  8. Save the changes by typing esc :wq
  9. Restart the network connection typing the command.
    Systemctl restart network
    Note:     This may cause the HA nodes to fail-over.
  10. Use an SSH session from the management interface of Primary Node to the Secondary Node repeat steps #3 through #10.
  11. Test the Crossover by trying to connect over them using an SSH session.
Results:
Your Crossover connections are now working.

Document Location

Worldwide

[{"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSBQAC","label":"IBM Security QRadar SIEM"},"Component":"HA;Networking","Platform":[{"code":"PF016","label":"Linux"}],"Version":"All Versions","Edition":"","Line of Business":{"code":"LOB24","label":"Security Software"}}]

Document Information

Modified date:
26 January 2021

UID

ibm10882644

Page Feedback