Troubleshooting the Outdated VM Manager Data and No VM Manager Data statuses

Edit online

Understand why the VM manager data cannot be retrieved by License Metric Tool and troubleshoot the issues. Apply the solutions for computers that have the Outdated VM Manager Data or No VM Manager Data status.

Background

Basic virtual environment is configured with one VM Manager Tool. Location of the VM Manager Tool depends on the version of License Metric Tool that you are using. For more information, see: VM Manager Tool deployment types and flow of data.

You can also use a more complex environment, with many instances of the VM Manager Tool. Each VM Manager Tool must be able to connect to all configured VM managers. Results from every VM manager must reach License Metric Tool.

Reasons behind failure of data retrieval

VM manager data cannot be retrieved by License Metric Tool due to one of the following reasons.
  • The hypervisor connection is not defined.
  • The VM Manager Tool cannot connect to the hypervisor to collect the data.
  • The VM Manager Tool does not generate results, or does not properly communicate them to License Metric Tool.
  • The VM Manager Tool collects invalid or incomplete results.

Information required for troubleshooting

Before you start troubleshooting the issue, prepare the following information.

  • The number of VM Manager Tool instances that are installed in your environment.
  • The number of hypervisors to be connected.
  • Valid credentials with sufficient rights to log in to the hypervisors.

9.2.12 Starting from application update 9.2.12, you usually do not need to go to the computer where the VM Manager Tool is installed, to get data that is needed for troubleshooting purposes. When License Metric Tool cannot retrieve VM manager data from a computer, go to the Computer Support Data panel and download the logs from the affected computer. For more information about how to download the log package, see: Collecting logs for troubleshooting purposes.

Troubleshooting

I. If the computer runs on a public cloud, identify it as running on a public cloud.
For detailed instructions, see the following links.
II. Check whether all VM manager connections are defined in License Metric Tool.
  1. Log in to License Metric Tool.
  2. Go to Management > VM Managers, and check whether all VM manager connections are defined. If some VM manager connections are not defined, add them. For more information, see: Adding VM managers in local and central mode or Adding and editing VM managers in distributed and disconnected mode.
    Note: Make sure that the VM manager URL is in the correct format. The format differs between technologies, and versions.
III. Check whether all VM manager connections are in the OK status icon OK status.
If any of the VM manager connections is in a status other than OK status icon OK, troubleshoot the problem. For more information, see: VM manager statuses.
IV. Check whether the VM Manager Tool is properly configured.
  1. In case of the central and distributed VM Manager Tool, check whether the VM Manager Tool was properly installed. By default, the VM Manager Tool is installed in the following directory.
    • Linux /var/opt/BESClient/LMT/VMMAN
    • Windows C:\Program Files (x86)\BigFix Enterprise\BES Client\LMT\VMMAN
    1. Log in to the BigFix console.
    2. In the navigation tree, click Actions, and select Install VM Manager Tool.
    3. Open the Computers tab, and check whether the status of the action is Completed. If the installation failed, check which line of the action script caused the failure. Double-click the listed computer, and analyze the View Action Info section to find the relevant line.
  2. Check whether the VM Manager Tool is working.
    • For local VM Manager Tool, run the following script.
      • Linux /opt/ibm/LMT/cli/srvstart.sh
      • Windows C:\Program Files\ibm\LMT\cli\srvstart.bat
    • For central and distributed VM Manager Tool, run the following command.
      • Linux /var/opt/BESClient/LMT/VMMAN/vmman.sh -run
      • Windows C:\Program Files (x86)\BigFix Enterprise\BES Client\LMT\VMMAN\vmman.bat -run
    • For disconnected VM Manager Tool, run the following command.
      • Linux <VM Manager Tool install dir>/vmman.sh -run
      • Windows <VM Manager Tool install dir>\vmman.bat -run
  3. To test connections to VM managers, run the following command. If no VM manager connection is defined, the command output is empty.
    • For local VM Manager Tool, run the following command.
      • Linux /opt/ibm/LMT/VMMAN/vmman.sh -testconnection
      • Windows C:\Program Files\ibm\LMT\VMMAN\vmman.bat -testconnection
    • For central and distributed VM Manager Tool, run the following command.
      • Linux /var/opt/BESClient/LMT/VMMAN/vmman.sh -testconnection
      • Windows C:\Program Files (x86)\BigFix Enterprise\BES Client\LMT\VMMAN\vmman.bat -testconnection
    • For disconnected VM Manager Tool, run the following command.
      • Linux <VM Manager Tool install dir>/vmman.sh -testconnection
      • Windows <VM Manager Tool install dir>\vmman.bat -testconnection
  4. To check statuses of VM manager connection, run the following command.
    • For local VM Manager Tool, run the following command.
      • Linux opt/ibm/LMT/VMMAN/vmman.sh -status
      • Windows C:\Program Files\ibm\LMT\VMMAN\vmman.bat -status
    • For central and distributed VM Manager Tool, run the following command.
      • Linux /var/opt/BESClient/LMT/VMMAN/vmman.sh -status
      • Windows C:\Program Files (x86)\BigFix Enterprise\BES Client\LMT\VMMAN\vmman.bat -status
    • For disconnected VM Manager Tool, run the following command.
      • Linux <VM Manager Tool install dir>/vmman.sh -status
      • Windows <VM Manager Tool install dir>\vmman.bat -status

    For more information, see: VM Manager Tool command-line options and Running the distributed and disconnected VM Manager Tool.

V. In case of the central and distributed VM Manager Tool, check whether the VM Manager Tool results are generated and properly communicated to the License Metric Tool server.
  1. Check whether files with scan results are packed for upload.
    1. Log in to License Metric Tool, open the Computer Support Data panel, and collect logs from the computer that you need to investigate. For more information, see: Collecting logs for troubleshooting purposes.
    2. Decompress the folder that you downloaded in step 1a and go to its following subdirectory.
      • Linux /sha1_files/logcollector_*_<computer_id>_logCollectorAgent.tar.gz/<computer_id>_logCollectorAgent.tar/var/opt/BESClient/LMT/VMMAN/<computer_id>_*_vmman.tar.gz
      • Windows \sha1_files\logcollector_*_<computer_id>_logCollectorAgent.zip\Program Files (x86)\BigFix Enterprise\BES Client\LMT\VMMAN\<computer_id>_*_vmman.zip
    3. If files with scan results are not packed for upload, check whether the upload is scheduled.
      1. Log in to the BigFix console.
      2. In the navigation tree, click Actions, and select Schedule VM Manager Tool Scan Results Upload.
      3. Check whether the status of the action is Open.
  2. Check whether files with scan results exist in the files directory in the log package that you downloaded from the affected computer.
    • Linux /files/vmman_scan_*_<computer_id>_*_vmman.tar.gz
    • Windows \files\vmman_scan_*_<computer_id>_*_vmman.zip

    If the Schedule VM Manager Tool Scan Results Upload action completed successfully, but the files are still not uploaded, go to the log package that you downloaded from the affected computer, and check the Index.txt file that is located in the files directory. If the file contains the MaxArchiveSize: Exceeded error, see: Configuring VM manager for subcapacity reporting.

    Note: 9.2.14 Starting from application update 9.2.14, you can check whether the value of _BESClient_ArchiveManager_MaxArchiveSize parameter is exceeded on the Computer Support Data panel. For more information, see: Checking whether the maximum archive size is exceeded.
  3. Check whether files with scan results exist in the BigFix database.
    Go to the log package that you downloaded from the affected computer, and check whether the BIGFIX_Uploads and BIGFIX_uploads_availability files contain the scan results. If you cannot find the results, restart the BES FillDB service on the BigFix server. To restart the service perform the following steps.
    • Linux Run the following command: /etc/init.d/besfilldb restart.
    • Windows Find the BES FillDB service on the list of services, and restart it.
    It takes several minutes for the database to update after the restart.

    If the issue occurs more than once, upgrade the BigFix server. For more information, see: Software/Hardware data may not appear in the ILMT/BFI UI after successful ETL import because of the BigFix FillDB APAR IV83671.

  4. Check whether files with scan results exist in the DatasourceFile file in the log package that you downloaded from the affected computer.

    If the files exist in the BigFix database, but they do no exist in the DatasourceFile file, contact the IBM support.

  5. Check whether files with scan results exist in the adm.last_imported_scan file in the log package that you downloaded from the affected computer.

    If the files exist in both databases that are listed in step 3 and 4, but do not exist in the adm.last_imported_scan file, it means that the import of data fails. Check the License Metric Tool import logs and troubleshoot the issue. The last import log is also attached to the log package that you downloaded from the affected computer.

VI. Check whether the VM Manager Tool collects valid and complete results.
If the scan result file was imported without errors, but some computers still have the Outdated VM Manager Data or No VM Manager Data status, perform the following steps.
  1. Obtain the list of UUIDs of affected computers.
    • 9.2.33 For application update 9.2.33 or higher, perform the following steps.
      1. Log in to License Metric Tool.
      2. In the top navigation bar, click Reports > Hardware Inventory.
      3. To customize the report view, hover over Configure, and click Configure View. Select only Partition ID column for display. Specify the report filter to display only rows which Capacity Scan Status equals No VM Manager Data or Outdated VM Manager Data. Click Submit.
      4. Export the report to CSV.
      5. Open the CSV file and choose one of the obtained UUIDs.
    • For versions older than application update 9.2.33, perform the following steps.
      • If you are troubleshooting the Outdated VM Manager Data status, perform the following steps.
        1. Log in to License Metric Tool, open the Computer Support Data panel, and collect logs from the affected computer.
        2. Open a file that matches the following pattern and has the highest timestamp.
          • Linux /sha1_files/cit_capacity_*_<computer_ID>_tlm_hw.tar.gz/<computer_ID>_tlm_hw.tar/tlm_hw_<date_time>_<timestamp>.xml
          • Windows\sha1_files\cit_capacity_*_<computer_ID>_tlm_hw.zip\tlm_hw_<date_time>_<timestamp>.xml
        3. Obtain the UUID of the computer.
          <?xml version="1.0" encoding="UTF-8"?>
    <Hardware>
    (...)
        <ComponentID version="1">
            <Manufacturer>VMware, Inc.</Manufacturer>
            <Product>VMware Virtual Platform</Product>
            <Version>None</Version>
            <SerialNumber>VMware-12 34 56 78 12 34 12 34-12 34 12 34 56 78 90 12</SerialNumber>
            <Type></Type>
            <UUID>12345678-1234-1234-1234-123456789012</UUID>
        </ComponentID>
    (...)
        <VirtualMachineGuest version="1">
            <UUID>VMware-12 34 56 78 12 34 12 34-12 34 12 34 56 78 90 12</UUID>
            <HypervisorType>VMware</HypervisorType>
        </VirtualMachineGuest>
    (...)
    </Hardware>
        4. Remove any prefixes or spaces in the obtained UUID. Ensure that the UUID conforms to the following pattern: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx. For example, 12345678-1234-1234-1234-123456789012.
      • If you are troubleshooting the No VM Manager Data status, perform the following steps.
        1. Log in to License Metric Tool.
        2. On the Capacity Scan Health widget, click No VM Manager Data. UUIDs of the affected computers are listed in the Server ID column.
          The screen shows the list of computers with no VM manager data on the Hardware Inventory report.
        3. Export the report to CSV.
        4. Open the CSV file and choose one of the obtained UUIDs. Remove the TLM_VM_ prefix from the UUID.
  2. Obtain files with VM Manager Tool scan results.
    • For local or disconnected VM Manager Tool, go to the VM Manager Tool installation directory and run the following command.
      • Linux ./vmman.sh -retrievedebugdata
      • Windows vmman.bat -retrievedebugdata
      Then, go to the debugData.zip file generated in the VM Manager Tool installation directory.
    • For central or distributed VM Manager Tool, open the log package, and go to the debugData.zip file that is stored in the following location.
      • Linux /sha1_files/logcollector_*_<computer_id>_logCollectorAgent.tar.gz/<computer_id>_logCollectorAgent.tar/var/opt/BESClient/LMT/VMMAN/debugData.zip
      • Windows \sha1_files\logcollector_*_<computer_id>_logCollectorAgent.zip\Program Files (x86)\BigFix Enterprise\BES Client\LMT\VMMAN\debugData.zip
  3. Check whether the UUID that you obtained in step 1 exists in the file with scan results that is located in the upload subdirectory of the debugData.zip package.

    The UUID you obtained in step 1 can differ from the UUID that is returned by the VM Manager Tool. Both values consist of 32 identical characters. However, the first 16 characters are arranged in different order. Only the final 16 characters are the same. For example, the value in the Server ID column can be TLM_VM_12345678-1234-1234-1234-123456789012, but the value that is retrieved by the VM Manager Tool is 78563412-3412-3412-1234-123456789012.

    This issue was fixed in application update 9.2.3. To solve the problem, upgrade License Metric Tool to the latest version. After the upgrade, upload the new capacity data to produce new scan results. To do that, run a single capacity scan and force upload of its results. For more information, see: Initiating the capacity scan on all computers (BigFix scenario). The capacity data is updated in up to two days, when the capacity scans and the VM manager data are imported.

    Example of the scan results file
    
    <cluster_layer>
        <id>id</id>
        <name>id</name>
        <host_layer>
            <uuid>01234567-0123-0123-0123-012345678901</uuid>
            <name>host_name</name>
            <cores_count>60</cores_count>
            <active_sockets_count>4</active_sockets_count>
            <available_sockets_count>4</available_sockets_count>
            <cpu_model>Intel(R) Xeon(R) CPU E7-4890 v2 @ 2.80GHz</cpu_model>
            <hardware_vendor>IBM</hardware_vendor>
            <hardware_model>x3850 X6 -[3837C4G]-</hardware_model>
            <connection_state>connected</connection_state>
            <power_state>powered_on</power_state>
            <virtual_technology>VSphere</virtual_technology>
            <guest_layer>
                <uuid>12345678-1234-1234-1234-123456789012</uuid>
                <vp_count>2</vp_count>
                <state>default_not_initialized</state>
                <virtual_technology>VSphere</virtual_technology>
            </guest_layer>
        </host_layer>
    </cluster_layer>
  4. If the UUID of the affected computer does not exist in the scan results files, check the following possible causes.
    • The UUID is duplicated.
      1. To verify whether the UUID is duplicated, search all trace.log_vmmconf_<VM_manager_ID>.log files for the following information. 
        Duplicates of UUIDs are found on VM Manager, 
URL: https://srvvcspr01/sdk/vimService.wsdl.
Discarded UUIDs for guests: 4227e866-7121-d731-235a-343077d7ee93.
        The trace.log_vmmconf_<VM_manager_ID>.log files are stored in the following location.
        • For local VM Manager Tool, log files are in the log package that you downloaded.
          • Linux /VMMAN/logs
          • Windows \VMMAN\logs
        • For central and distributed VM Manager Tool, log files are in the log package that you downloaded.
          • Linux /sha1_files/logcollector_*_<computer_id>_logCollectorAgent.tar.gz/<computer_id>_logCollectorAgent.tar/var/opt/BESClient/LMT/VMMAN/logs
          • Windows \sha1_files\logcollector_*_<computer_id>_logCollectorAgent.zip\Program Files (x86)\BigFix Enterprise\BES Client\LMT\VMMAN\logs
        • For disconnected VM Manager Tool, the logs directory is in the following location on the computer where the VM Manager Tool is installed.
          • Linux <VM Manager Tool install dir>/logs
          • Windows<VM Manager Tool install dir>\logs
      2. If some UUIDs are duplicated, go to the virtual machine that has the duplicated UUID, and change it so that it becomes unique.
    • The user who is connecting with the VM manager on VMware does not have a permission to collect data about the virtual machine. Check the VM manager settings, and permissions. For more information, see: Verifying permissions for VMware communication.
    • The VM manager is not valid for the virtual machine.
  5. If the UUID of the affected computer exists in the scan results file, but the computer still has the Outdated VM Manager Data or No VM Manager Data status, perform the following steps.
    1. Check whether debug data is collected for the affected computer.
      • For local or disconnected VM Manager Tool, go to the VM Manager Tool installation directory and run the following command.
        • Linux ./vmman.sh -retrievedebugdata
        • Windows vmman.bat -retrievedebugdata
        Then, go to the debugData.zip file generated in the VM Manager Tool installation directory.
      • For central or distributed VM Manager Tool, open the log package, and go to the debugData.zip file that is stored in the following location.
        • Linux /sha1_files/logcollector_*_<computer_id>_logCollectorAgent.tar.gz/<computer_id>_logCollectorAgent.tar/var/opt/BESClient/LMT/VMMAN/debugData.zip
        • Windows \sha1_files\logcollector_*_<computer_id>_logCollectorAgent.zip\Program Files (x86)\BigFix Enterprise\BES Client\LMT\VMMAN\debugData.zip
    2. Go to the debug/<VM_manager_connection_ID> subdirectory of the debugData.zip file.
    3. Open the login.xml file and check whether it is error-free. If the file contains the following authentication error, but the VM manager connection test was successful, the user login, or password probably contains one of the following characters: $ or \. 
      <faultcode>ServerFaultCode</faultcode>
<faultstring>Cannot complete login due to an incorrect user name
or password.</faultstring>
<detail>
<InvaildLoginFault xmlns="urn:vim25" xsi:type="InvalidLogin"/>
      To solve the problem, upgrade the VM Manager Tool to the latest version. For more information, see: Upgrading the central or distributed VM Manager Tool or Upgrading the disconnected VM Manager Tool .
      Note: After you upgrade the VM Manager Tool, download fresh log package and check the debug data.
    4. Open the retrieveProperties.xml file. The file contains information about all virtual machines, including VMs with duplicated UUIDs, and the <HostCpuPackage> data. If the <HostCpuPackage> tag is not in the file, the user that connects to the VM manager is underprivileged. Ensure that the user has sufficient permissions. For more information, see: Verifying permissions for VMware communication and How to set the correct permissions for LMT - VMware communication.