IBM Support

QRadar: Recovering Appliances in High-Availability (HA) Pairs when the Secondary failed

Troubleshooting


Problem

What is the best way to recover a High-Availability Secondary appliance that has failed due to disk corruption or a catastrophic failure, and the Primary is Active and healthy.

Resolving The Problem

 

Step 1: Required prerequisites before you begin

  1. Obtain the DVD iso for the QRadar build closest to the primary build from first installation from fix central.
    Note: Installing an ISO that was not the same as what was used when the primary was installed will result in different sized partitions, and can cause issues.
    For 7.3.x ISOs please use this link.
    QRadar 7.3.x ISO.
    For 7.2.x ISOs please use this link.
    QRadar 7.2.x ISO.
  2. Obtain the Patch Fix pack that corresponds to the QRadar Primary HA version.
    To obtain other builds of QRadar Fix packs, log in to Fix Central .
  3. Please ensure that you have access to either a remote management console, such as an IMM, iDRAC, or a KVM connection to the Secondary HA appliance before continuing this procedure.
  4. Retain the Host Name, IP Address, Subnet, Gateway, Name Servers, and Email Servers for this Appliance.
  5. Ensure you have the Password for this Appliance.



 

Step 2: Validate that one of the HA pairs is Active

  1. Click Admin tab > System and License Management.
  2. Click Display Systems.
  3. Expand the HA pair in question and validate that one of the hosts has failed as seen below:

 

Step 3: Mounting the ISO when using an IMM

To mount the iso using the IMM please follow the guide that is appropriate for your appliance.

  1. Mapping and Un-Mapping drives with IMM and M3 Appliances
  2. Mapping and Un-Mapping drives with IMM2 and M4 or M5 Appliances
 

Step 4: Rebuilding the secondary HA appliance

 
  1. When the splash menu is displayed with the option <F12> Select Boot device, press the F12 key on your keyboard.
  2. At the Boot Devices Manager window scroll down to the option CD/DVD.
  3. Press the enter key to select the entry.
    Note: for M3 and M5 Appliances select the option CD/DVD. For M4 Appliances select the option USB4:CD/DVD - USB Port 4 Remote.
  4. After the system boots, a splash screen will be displayed with these options.
    Choose flatten option for this step since we need to rebuild all partitions.
  5. When the system completes the rebuild.
    1. For QRadar 7.3.x versions. From the Appliance install menu, check High Availability Appliance > check HA Appliance (All Models) 500 from the setup menu.
      For QRadar 7.2.x versions. Enter the activation key for a 500 Appliance.
    2. Enter if this is an HA for a Console or Non-Console.
    3. Enter all information for Host Name, IP Address, Subnet, Gateway, Name Servers, and Email Servers for this Appliance.
    4. Enter the password for this Appliance.
  6. Upon completing this installation umount the ISO for this Appliance from the IMM.
  7. Using SCP move the QRadar Fixpack that is the same QRadar version as the Primary Node to the rebuilt Secondary Node to /tmp
    If space is limited, copy the fix pack to another location.
  8. If the ISO build was not the same version as the Primary Node, mount the fixpack patch file on the secondary node using the command:
    mount -o loop -t squashfs 7xx_QRadar_patchupdate-7.x.x.201xxxxxxxxxx.sfs /media/updates
  9. To run the patch installer, type the following command:
    /media/updates/installer
    Note: The first time you run the fix pack there might be a delay before the installation menu is displayed.
  10. Using the patch installer, select the appliance to install the patch on.
  11. After the patch completes, should the appliance not require an automatic restart, un-mount the fix pack using this command
    umount /media/updates
  12. If you are using Offboard Storage for any partitions, refer to the offboard storage guide IBM Security Offboard Storage Guide to re-configure your storage.
 

Step 5: Restore the Secondary HA appliance

  1. Log in to the QRadar User Interface.
  2. Click Admin tab > System and License Management.
  3. Click Highlight the failed Secondary.
  4. Click High Availability and then Restore System.

  5. Upon completion your HA secondary appliance will Synchronize with the Primary and you will not have any interruption on the Primary node.
 

Results: Once Synchronization is complete your Secondary will go into standby mode and its now available for Fail Over.

[{"Product":{"code":"SSBQAC","label":"IBM Security QRadar SIEM"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Component":"High Availability","Platform":[{"code":"PF016","label":"Linux"}],"Version":"7.2;7.3","Edition":"","Line of Business":{"code":"LOB24","label":"Security Software"}}]

Document Information

Modified date:
10 November 2020

UID

swg22010619