IBM Support

QRadar: Replacement hard disk drive cannot rebuild and firmware state displays "JBOD"

Troubleshooting


Problem

When you replace a failed drive, the 930/530 RAID controller can set the drive into JBOD mode, which prevents a rebuild of the existing RAID virtual drive. This issue is due to a firmware problem. Administrators must set the status to unconfigured (good) state to ensure the drive can rebuild successfully.

Symptom

The replacement drive is not usable due to the current state of the drive in the enclosure. A review of the physical drive state can show JBOD, which prevents the RAID controller from successfully rebuilding the drive for the array.

Cause

QRadar M6 hardware appliances with the Raid firmware from the 5.0.0 or 6.0.0 M6 firmware package might be affected by this JBOD state issue for replacement drives.

Environment

While this primarily impacts RAID adapters in QRadar M6 hardware appliances, the M5200 series RAID controllers in the M4 and M5 appliances could potentially be impacted as well.

Diagnosing The Problem

To diagnose the replacement drive, confirm the current state of the physical and virtual drive with the StorCLI show command.
  1. Use a keyboard console or IMM to log in to the QRadar appliance with the replacement drive.
  2. To display the virtual drive state, type:
    /opt/MegaRAID/storcli/storcli64 /c0/dall Show
    If the virtual drive shows as 'Degraded' or 'Partially Degraded' and the state of the physical drive displays "JBOD", you must correct the state of the drive.  In this example we can see the virtual drive is partially degraded (RAID 6 Pdgd). The drive in slot 1 is showing at missing (Msng), which is consistent for a drive in a JBOD state.  
    /opt/MegaRAID/storcli/storcli64 /c0/dall Show
    ...
    
    TOPOLOGY :
    ========
    
    ----------------------------------------------------------------------------
    DG Arr Row EID:Slot DID Type  State BT      Size PDC  PI SED DS3  FSpace TR
    ----------------------------------------------------------------------------
     0 -   -   -        -   RAID6 Pdgd  N  72.768 TB dsbl N  N   dflt N      N
     0 0   -   -        -   RAID6 Dgrd  N  72.768 TB dsbl N  N   dflt N      N
     0 0   0   134:0    1   DRIVE Onln  N   7.276 TB dsbl N  N   dflt -      N
     0 0   1   -        -   DRIVE Msng  -   7.276 TB -    -  -   -    -      N
     0 0   2   134:2    5   DRIVE Onln  N   7.276 TB dsbl N  N   dflt -      N
    
    ...
  3. To confirm the physical drive state, type:
    /opt/MegaRAID/storcli/storcli64 /c0/eall/sall Show
    The output displays the state and the enclosure and slot number.
  4. Identify any drives list the state as JBOD.
    Drive Information :
    =================
    
    ----------------------------------------------------------------------------
    EID:Slt DID State DG     Size Intf Med SED PI SeSz Model            Sp Type
    ----------------------------------------------------------------------------
    134:0     1 Onln   0 7.276 TB SAS  HDD N   N  512B HUS728T8TAL5200  U  -
    134:1     3 JBOD   - 7.277 TB SAS  HDD N   N  512B HUS728T8TAL5200  U  -
    134:2     5 Onln   0 7.276 TB SAS  HDD N   N  512B HUS728T8TAL5200  U  -
    134:3     7 Onln   0 7.276 TB SAS  HDD N   N  512B HUS728T8TAL5200  U  -
    
     ...
    Results
    If the state displays JBOD, you are affected by the issue and must set the replacement drive to an unconfigured state.  It is important to record the enclosure ID (EID) and the slot number (Slt) as this information is required to resolve the issue. In the example, we can see the EID:Slt shows the JBOD disk is in Slot 1, on Enclosure Device ID 134.

Resolving The Problem

Before you begin
Ensure you use the correct Enclosure Device and Slot number for the drive in a JBOD state. If you force a drive that is already online to the unconfigured good state, it might further degrade the virtual drive or take the virtual drive offline.

Procedure
To resolve the problem, change the drive state to unconfigured good. As the drive is configured as JBOD, it is likely assigned as a device in the OS and you might need to use the -Force option to set the state.
  1. Type the following command to change the state for the drive to 'unconfigured good':
    Important: The examples in this section use a drive in Enclosure 134, Slot 1 for adapter 0. The drive enclosure and slot values are likely different on your appliance. It is important you use the correct values in this procedure for the replacement drive in your QRadar appliance.
    /opt/MegaRAID/storcli/storcli64 /c0/e134/s1 Set Good Force
  2. Configure the replacement drive as a hot spare with the following command:
    /opt/MegaRAID/storcli/storcli64 /c0/e134/s1 Add HotSpareDrive

    Results
    The replacement drive is recognized as a hot spare and the RAID controller can rebuild the disk for the array. It might take a few minutes for the rebuild process to begin. The view the rebuild status type the following command:
    /opt/MegaRAID/storcli/storcli64 /c0/e134/s1 Show Rebuild

Preventing JBOD state issues for replacement drives

Update RAID Controller Firmware (preferred)

The QRadar M6 7.1.0 Firmware Package contains the Lenovo RAID 530/930 firmware version: 51.20.0-4374-0, which prevents this issue from occurring.  If you are unable to update the QRadar host firmware, the JBOD property can be disabled on the controller as an alternative workaround to prevent this issue.
 

Manually disable JBOD status for the RAID Controller (alternate)

After you resolve the problem to allow replacement drives to be rebuilt, administrators can use the StorCLI utility to turn off the RAID controller's JBOD property. Disabling JBOD ensures the drive state is not incorrectly set by the RAID controller on future drive replacements.
Use the following command to show the JBOD state on controller 0. If the controller properties shows JBOD is on, we can then use the Set command to turn it off.
  1. To confirm the controller properties, type:
    /opt/MegaRAID/storcli/storcli64 /c0 Show JBOD
    The output confirms the JBOD property status.
    Controller Properties:
    =====================
    ----------------
    Ctrl_Prop Value
    ----------------
    JBOD      ON
  2. Configure the controller properties to disable JBOD on the appliance:
    /opt/MegaRAID/storcli/storcli64 /c0 Set JBOD=off
    or
    /opt/MegaRAID/storcli/storcli64 /c0 Set JBOD=off Force
  3. Verify the status of JBOD is set to off:
    /opt/MegaRAID/storcli/storcli64 /c0 Show JBOD
    The output displays the current properties.
    Controller Properties:
    =====================
    ----------------
    Ctrl_Prop Value
    ----------------
    JBOD      OFF
    Results
    After you confirmed JBOD is disabled, the procedure is complete. If you continue to experience issues or problems with this technical note, contact QRadar Support.

     

Document Location

Worldwide

[{"Type":"MASTER","Line of Business":{"code":"LOB24","label":"Security Software"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSQKYT","label":"IBM QRadar xx48 Appliance"},"ARM Category":[{"code":"a8m0z000000cwtcAAA","label":"Hardware"}],"ARM Case Number":"TS008497375","Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Versions"},{"Type":"MASTER","Line of Business":{"code":"LOB24","label":"Security Software"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSBQAC","label":"IBM Security QRadar SIEM"},"ARM Category":[{"code":"a8m0z000000cwtcAAA","label":"Hardware"}],"ARM Case Number":"","Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Versions"}]

Document Information

Modified date:
06 March 2024

UID

ibm16573915