Troubleshooting BMC issues

Troubleshooting

Problem

Troubleshooting BMC issues on IBM System x and BladeCenter servers

Resolving The Problem

To begin troubleshooting the Baseboard Management Controller (BMC), check the following top issues. If your issue is listed, select the link, otherwise proceed to step 2.

	BMC error log overview
	BMC error log access
	BMC flash update
	BMC Intelligent Platform Management Interface (IPMI) data loss
	BMC log shows voltage or temperature errors
	BMC log shows Non Maskable Interrupt (NMI)
	BMC network does not work when AC power first applied
	BMC busy and not responding to initial requests from MM
	Error "Dr Watson has created an error log for the following program, SMBridge.exe"
	Errors running the command line interface through the serial port
	Ethernet connections to the BMC fail

Check the operator information panel on the front of the server. If the information LED is lit, it indicates that information about a suboptimal condition in the server is available in the BMC log or in the system-error log. Proceed to step 3.

Note: If the system-error LED is lit, refer to your system's Hardware Maintenance Manual (HMM) for light path diagnostics.

Access the BMC log.

	Viewing error logs from the Configuration/Setup Utility program
	Viewing the BMC log from the diagnostic programs

Check the system-error log and the BMC log for messages indicating the cause of the error. Take the action indicated before replacing a CRU or FRU. When troubleshooting an error, make sure to clear the BMC log so that you can more easily find current errors.
If these steps have not solved your problem, refer to your system's Hardware Maintenance Manual (HMM),
or refer to "Need more help?"

Error log overview

Newer IBM eServer xSeries systems (e.g. xSeries 236, 306, 346, 366) have a BMC log which is the default "service processor" if the optional RSA2 is not installed. The BMC log and the system-error log contain messages that were generated during POST and all system status messages from the service processor. The BMC log contains a large number of information, error, and warning messages. The POST error log contains the three most recent error codes and messages that were generated during POST.

Notes:

The BMC log is limited in size and is designed so that when the log is full, new entries will not overwrite existing entries; therefore, you must periodically clear the BMC log from the Configuration/Setup Utility program (the menu choices are described in the User's Guide).
When troubleshooting an error, make sure to clear the BMC log so that you can find current errors more easily.
Entries written to the BMC log early in the POST procedure will show an incorrect date as the default timestamp; however, the date and time will correct itself as POST continues.
Each BMC log entry appears on its own page; to display all the data for an entry, use the up arrow and down arrow or the Page Up and Page Down keys. To move from one entry to the next, move the cursor to the Get Next Entry or Get Previous Entry line; then, press Enter.
The log indicates an Assertion Event when an event has occurred. It indicates a Deassertion Event when the event is no longer occurring.
Some of the error codes and messages in the BMC log are abbreviated.
Viewing the BMC log through the web interface of the optional Remote Supervisor Adapter II SlimLine allows all messages to be translated.

Example BMC log entry:

BMC system event log

	Back to top

Accessing the BMC log

You can view the contents of the POST error log, the BMC log, and the system-error log from the Configuration/Setup Utility program. You can also view the contents of the BMC log from the diagnostic programs. The BMC log contains the same information whether it is viewed from the Configuration/Setup Utility program or from the diagnostic programs.

Note: When troubleshooting PCI-X slots, note that the error logs report the PCI-X buses numerically. The numerical assignments vary depending on the configuration. You can check the assignments by running the Configuration/Setup Utility program.

Viewing error logs from the Configuration/Setup Utility program

To view the error logs from the Configuration/Setup Utility:

Turn on the server.
When the prompt "Press F1 for Configuration/Setup" appears, press F1. If you have set both a power-on password and an administrator password, you must type the administrator password to view the error logs.
Use one of the following procedures:
1. To view the POST error log, select Error Logs, and then select POST Error Log.
2. To view the BMC log, select Advanced Settings, select Baseboard Management Controller (BMC) settings, and then select BMC System Event Log.
3. To view the system-error log, select Event/Error Logs, and then select System Event/Error Log
  Note: This option is only available if an optional Remote Supervisor Adapter II SlimLine is installed.

	Back to top

Viewing the BMC log from the diagnostic programs

To view the BMC log from diagnostics:

If the server is running, turn off the server and all attached devices.
Turn on all attached devices; then, turn on the server.
When the prompt "F2 for Diagnostics" appears, press F2.

Note: If you have set both a power-on password and an administrator password, you must type the administrator password to run the diagnostic programs.
From the top of the screen, select Hardware Info.
From the list, select BMC Log.

	Back to top

Need more help?

Please select one of the the following options for further assistance:

	Support forums

	Submit a technical question

	Before you call IBM Service

Document Location

Worldwide

Operating System

BladeCenter:Operating system independent / None

System x:Operating system independent / None

[{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"HW19V","label":"BladeCenter->BladeCenter HS20"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"HW20D","label":"BladeCenter JS20 Blade"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"LOB57","label":"Power"}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"HW20G","label":"BladeCenter->BladeCenter HS40"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"LOB18","label":"Miscellaneous LOB"}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"HW20M","label":"BladeCenter->BladeCenter T Chassis"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"HW21G","label":"BladeCenter->BladeCenter LS20"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"HW21X","label":"BladeCenter JS21 Blade"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"LOB57","label":"Power"}},{"Type":"HW","Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"HW21Y","label":"BladeCenter H Chassis"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"LOB57","label":"Power"}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"HW22E","label":"BladeCenter->BladeCenter HS21"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"HW22F","label":"BladeCenter->BladeCenter LS21"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"HW22G","label":"BladeCenter->BladeCenter LS41"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"HW22H","label":"BladeCenter QS20 Blade"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"LOB57","label":"Power"}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"HW22I","label":"BladeCenter->BladeCenter HS21 XM"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"HW22Q","label":"BladeCenter->BladeCenter HT Chassis"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"LOB57","label":"Power"}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"HW312","label":"System x->System x3800"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"HW313","label":"System x->System x3650 T"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"HW314","label":"System x->System x3850"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"HW315","label":"System x->System x3950"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU054","label":"Systems w\/TPS"},"Product":{"code":"HW316","label":"System x->System x3950 E"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"HW317","label":"System x->System x3500"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"HW318","label":"System x->System x3550"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"HW319","label":"System x->System x3650"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"HW321","label":"System x->System x3400"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"HW322","label":"System x->System x3455"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"HW323","label":"System x->System x3655"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"HW324","label":"System x->System x3755"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"HW328","label":"System x->System x3105"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"LOB26","label":"Storage"}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"HW329","label":"System x->System x3200"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"HW331","label":"System x->System x3250"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"HWXA1","label":"System x->System x3650 NAS"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"HW20T","label":"BladeCenter E Chassis"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"HW20T","label":"BladeCenter E Chassis"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}}]

Tips

Troubleshooting BMC issues