IBM Support

Upgrade operation fails with CRLQE0697E error

Troubleshooting


Problem

After upgrading IBM Lifecycle Query Engine (LQE), the application is unusable. 
When you open the LQE administration page you see the Symptoms described, explicitly stating that a migration is required.

Symptom

When you open the LQE administration page, you see the following error:
Migrate
Lifecycle Query Engine has detected that migration is required from 6.0.6.1 M1 to 6.0.6.1
Lifecycle Query Engine encountered problems during migration. Review LQE and server logs files, correct the errors, and migrate again.
CRLQE0657E Unable to upgrade Lucene index. RecordRangeInterator: records not strictly increasing
This corresponding error is reported in lqe.log:
[lqe.UpgradeTask ] [ERROR] m.ibm.team.jis.lqe.maintenance.upgrade.UpgradeTask - CRLQE0697E An error occurred perform an upgrade.
com.ibm.team.jis.lqe.maintenance.upgrade.UpgradeException: CRLQE0657E Unable to upgrade Lucene index. at
com.ibm.team.jis.lqe.maintenance.upgrade.handler.Lucene7ReindexHandler.doUpgrade(Lucene7ReindexHandler.java:114) at
com.ibm.team.jis.lqe.maintenance.upgrade.UpgradeTask.run(UpgradeTask.java:79) at
java.lang.Thread.run(Thread.java:818)
Caused by: com.hp.hpl.jena.tdb.base.StorageException: RecordRangeIterator: records not strictly increasing: 0000000000057e5a0000000000057ee100000000000034bc0000000000000f3c // 000000000004766e00000000000476f50000000000000b5d0100000000000002 at com.hp.hpl.jena.tdb.base.recordbuffer.RecordRangeIterator.hasNext(RecordRangeIterator.java:114) at org.apache.jena.atlas.iterator.Iter$4.hasNext(Iter.java:317) at com.hp.hpl.jena.tdb.sys.DatasetControlMRSW$IteratorCheckNotConcurrent.hasNext(DatasetControlMRSW.java:119) at org.apache.jena.atlas.iterator.Iter$4.hasNext(Iter.java:317) at org.apache.jena.atlas.iterator.Iter$4.hasNext(Iter.java:317) at org.apache.jena.atlas.iterator.Iter.hasNext(Iter.java:915) at com.hp.hpl.jena.util.iterator.WrappedIterator.hasNext(WrappedIterator.java:90) at com.hp.hpl.jena.util.iterator.Map1Iterator.hasNext(Map1Iterator.java:48) at com.hp.hpl.jena.util.iterator.WrappedIterator.hasNext(WrappedIterator.java:90) at com.ibm.team.integration.lqe.lib.dataset.lucene.LuceneIndexBuilder.indexNamedModel(LuceneIndexBuilder.java:283) at com.ibm.team.integration.lqe.lib.dataset.lucene.LuceneIndexBuilder.updateNamedModel(LuceneIndexBuilder.java:254) at com.ibm.team.integration.lqe.lib.dataset.tdb.TdbDatasetProvider.reindexLucene(TdbDatasetProvider.java:2245) at com.ibm.team.jis.lqe.maintenance.upgrade.handler.Lucene7ReindexHandler.doUpgrade(Lucene7ReindexHandler.java:111)

Cause

One or more of the underlying components of the LQE index is corrupted.  Corruption of the Jena TDB data files typically occurs from an application crash or sudden shutdown or during copying index files.

Resolving The Problem

There is no repair tool to repair your index. Anyway there are 2 work-arounds:
WORK-AROUND 1:
  1. Roll back an application to the previous version.
  2. Restore your index from a backup according to Backing up and restoring Lifecycle Query Engine and Link Index Provider (LDX) document.
  3. Perform compaction of the index according to Improving the Lifecycle Query Engine performance by compacting indexed data document.
    NOTE: Compaction operation is a good indicator if your index is corrupted or not. If the compaction is successful, then you can repeat upgrade process and you likely do not get this error. If compaction fails, then use WORK-AROUND 2.
WORK-AROUND 2:
NOTE: This procedure requires full reindex of your data sources but it is the only workaround you can use if WORK-AROUND 1 does not work.
  1. Back up LQE according to Backing up and restoring Lifecycle Query Engine and Link Index Provider (LDX) document.
  2. Stop LQE.
  3. Remove Index Directories (indexTdb, textIndex, shapeTdb, shapeText, historyTdb, historyText, versionTdb).
    NOTE: Sometimes you need to remove only one of the directories. That does not require running reindex for all data sources. If you want to proceed with that path, run an initial upgrade with the following log4j entry for LQE: log4j.logger.com.ibm.team.jis.lqe.maintenance.upgrade.handler.Lucene7ReindexHandler=info. When an upgrade fails, contact IBM support with lqe.log file to check which exactly directories you have to remove.
  4. Upgrade the system.
  5. Reindex all data sources, preferably one at a time for the large TRS.

Document Location

Worldwide

[{"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSTU9C","label":"Jazz Reporting Service"},"Component":"Lifecycle Query Engine","Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"6.0.6.1","Edition":"","Line of Business":{"code":"LOB59","label":"Sustainability Software"}}]

Document Information

Modified date:
20 September 2019

UID

ibm11074116

Page Feedback