IBM Support

RS03574: SLOWNESS IN ENTERPRISE CONSOLE - HASHMAP CALL GETENTRY REPEATED AND RUNNING FOR MANY MINUTES

Subscribe

You can track all active APARs for this component.

 

APAR status

  • Closed as program error.

Error description

  • All of the threads are stuck in getentry function of
    java/util/HashMap.
    
    The hashmap has been corrupted to get into
    this state; all the get<something>Proxy in IlrClassProxy should
    be synchronized to be thread-safe.
    

Local fix

Problem summary

  • ****************************************************************
    * USERS AFFECTED:                                              *
    * Slowness when multiple users open a project with a large BOM *
    * in Enterprise Console.                                       *
    ****************************************************************
    * PROBLEM DESCRIPTION:                                         *
    * Significant slowness is seen in the enterprise console, we   *
    * observe that the thread dump shows a single HashMap call,    *
    * repeated 30 times and running for many minutes.              *
    *                                                              *
    * The stack in the threads is:                                 *
    *                                                              *
    * "WebContainer : 11" J9VMThread:0x0000000004D4FE00,           *
    * j9thread_t:0x00007F127408A4F0,                               *
    * java/lang/Thread:0x000000040E26B910, state:R, prio=5         *
    *                                                              *
    * 3XMJAVALTHREAD      (java/lang/Thread getId:0x28C,           *
    * isDaemon:true)                                               *
    *                                                              *
    * 3XMTHREADINFO1      (native thread ID:0x381F, native         *
    * priority:0x5, native policy:UNKNOWN, vmstate:CW, vm thread   *
    * flags:0x00000001)                                            *
    *                                                              *
    * 3XMTHREADINFO2      (native stack address range              *
    * from:0x00007F13054F9000, to:0x00007F130553A000,              *
    * size:0x41000)                                                *
    *                                                              *
    * 3XMCPUTIME        CPU usage total: 365179.938396542 secs,    *
    * current category="Application"                               *
    *                                                              *
    * 3XMHEAPALLOC       Heap bytes allocated since last GC        *
    * cycle=0 (0x0)                                                *
    *                                                              *
    * 3XMTHREADINFO3      Java callstack:                          *
    *                                                              *
    * 4XESTACKTRACE        at                                      *
    * java/util/HashMap.getEntry(HashMap.java:478(Compiled Code))  *
    *                                                              *
    * 4XESTACKTRACE        at                                      *
    * java/util/HashMap.get(HashMap.java:430(Compiled Code))       *
    *                                                              *
    * 4XESTACKTRACE        at                                      *
    * ilog/rules/factory/proxy/IlrClassProxy.getFieldProxy(IlrClas *
    * sProxy.java:65(Compiled Code))                               *
    *                                                              *
    * 4XESTACKTRACE        at                                      *
    * ilog/rules/factory/proxy/IlrClassProxy.getAttribute(IlrClass *
    * Proxy.java:902(Compiled Code))                               *
    *                                                              *
    * 4XESTACKTRACE        at                                      *
    * ilog/rules/vocabulary/model/bom/io/IlrBOMVocabularyReader.pr *
    * ocessMemberKey(IlrBOMVocabularyReader.java:212(Compiled      *
    * Code))                                                       *
    *                                                              *
    * 4XESTACKTRACE        at                                      *
    * ilog/rules/vocabulary/model/bom/io/IlrBOMVocabularyReader.fi *
    * llVocabulary(IlrBOMVocabularyReader.java:123(Compiled Code)) *
    *                                                              *
    * 4XESTACKTRACE        at                                      *
    * ilog/rules/vocabulary/model/bom/io/IlrBOMVocabularyReader.re *
    * adVocabulary(IlrBOMVocabularyReader.java:95(Compiled Code))  *
    *                                                              *
    * 4XESTACKTRACE        at                                      *
    * ilog/rules/teamserver/brm/impl/IlrBOMEntryImpl.computePartia *
    * lVocabularyInfo(IlrBOMEntryImpl.java:312(Compiled Code))     *
    *                                                              *
    * 4XESTACKTRACE        at                                      *
    * ilog/rules/teamserver/model/IlrGlobalCache.getPartialVocabul *
    * aryInfo(IlrGlobalCache.java:735(Compiled Code))              *
    *                                                              *
    * 4XESTACKTRACE        at                                      *
    * ilog/rules/teamserver/model/IlrBOMPathHelper$VocabularyVisit *
    * or.visitBOMEntry(IlrBOMPathHelper.java:405(Compiled Code))   *
    *                                                              *
    * 4XESTACKTRACE        at                                      *
    * ilog/rules/teamserver/brm/impl/IlrBOMEntryImpl.acceptVisitor *
    * (IlrBOMEntryImpl.java:341(Compiled Code))                    *
    *                                                              *
    * 4XESTACKTRACE        at                                      *
    * ilog/rules/teamserver/model/IlrDefaultBOMPathVisitor.visitPr *
    * ojectBOMEntry(IlrDefaultBOMPathVisitor.java:54(Compiled      *
    * Code))                                                       *
    *                                                              *
    * 4XESTACKTRACE        at                                      *
    * ilog/rules/teamserver/brm/impl/IlrProjectBOMEntryImpl.accept *
    * Visitor(IlrProjectBOMEntryImpl.java:71(Compiled Code))       *
    *                                                              *
    * 4XESTACKTRACE        at                                      *
    * ilog/rules/teamserver/model/IlrDefaultBOMPathVisitor.visitPr *
    * ojectBOMEntry(IlrDefaultBOMPathVisitor.java:54(Compiled      *
    * Code))                                                       *
    *                                                              *
    * 4XESTACKTRACE        at                                      *
    * ilog/rules/teamserver/brm/impl/IlrProjectBOMEntryImpl.accept *
    * Visitor(IlrProjectBOMEntryImpl.java:71(Compiled Code))       *
    *                                                              *
    * 4XESTACKTRACE        at                                      *
    * ilog/rules/teamserver/model/IlrDefaultBOMPathVisitor.visitPr *
    * ojectBOMEntry(IlrDefaultBOMPathVisitor.java:54(Compiled      *
    * Code))                                                       *
    *                                                              *
    * 4XESTACKTRACE        at                                      *
    * ilog/rules/teamserver/brm/impl/IlrProjectBOMEntryImpl.accept *
    * Visitor(IlrProjectBOMEntryImpl.java:71(Compiled Code))       *
    *                                                              *
    * 4XESTACKTRACE        at                                      *
    * ilog/rules/teamserver/model/IlrDefaultBOMPathVisitor.visitPr *
    * ojectBOMEntry(IlrDefaultBOMPathVisitor.java:54(Compiled      *
    * Code))                                                       *
    *                                                              *
    * 4XESTACKTRACE        at                                      *
    * ilog/rules/teamserver/brm/impl/IlrProjectBOMEntryImpl.accept *
    * Visitor(IlrProjectBOMEntryImpl.java:71(Compiled Code))       *
    *                                                              *
    * 4XESTACKTRACE        at                                      *
    * ilog/rules/teamserver/model/IlrDefaultBOMPathVisitor.visitPr *
    * ojectBOMEntry(IlrDefaultBOMPathVisitor.java:54(Compiled      *
    * Code))                                                       *
    *                                                              *
    * 4XESTACKTRACE        at                                      *
    * ilog/rules/teamserver/brm/impl/IlrProjectBOMEntryImpl.accept *
    * Visitor(IlrProjectBOMEntryImpl.java:71(Compiled Code))       *
    *                                                              *
    * 4XESTACKTRACE        at                                      *
    * ilog/rules/teamserver/model/IlrDefaultBOMPathVisitor.run(Ilr *
    * DefaultBOMPathVisitor.java:41(Compiled Code))                *
    *                                                              *
    * 4XESTACKTRACE        at                                      *
    * ilog/rules/teamserver/model/IlrBOMPathHelper$VocabularyVisit *
    * or.run(IlrBOMPathHelper.java:399)                            *
    *                                                              *
    * 4XESTACKTRACE        at                                      *
    * ilog/rules/teamserver/model/IlrBOMPathHelper.computeVocabula *
    * ryInfo(IlrBOMPathHelper.java:372)                            *
    *                                                              *
    * 4XESTACKTRACE        at                                      *
    * ilog/rules/teamserver/model/IlrBOMPathHelper.computeRuleVoca *
    * bulary(IlrBOMPathHelper.java:326(Compiled Code))             *
    *                                                              *
    * 4XESTACKTRACE        at                                      *
    * ilog/rules/teamserver/model/IlrBOMPathHelper.computeVocabula *
    * ry(IlrBOMPathHelper.java:302)                                *
    *                                                              *
    * 4XESTACKTRACE        at                                      *
    * ilog/rules/teamserver/model/IlrGlobalCache.getVocabularyMana *
    * gerInfo(IlrGlobalCache.java:599(Compiled Code))              *
    *                                                              *
    * 4XESTACKTRACE        at                                      *
    * ilog/rules/teamserver/model/IlrBOMPathHelper.getVocabularyMa *
    * nager(IlrBOMPathHelper.java:367(Compiled Code))              *
    *                                                              *
    * 4XESTACKTRACE        at                                      *
    * ilog/rules/teamserver/model/impl/IlrAbstractSession.getVocab *
    * ulary(IlrAbstractSession.java:321(Compiled Code))            *
    *                                                              *
    * 4XESTACKTRACE        at                                      *
    * ilog/rules/teamserver/model/impl/IlrAbstractSession.getWorki *
    * ngVocabulary(IlrAbstractSession.java:315(Compiled Code))     *
    *                                                              *
    * 4XESTACKTRACE        at                                      *
    * ilog/rules/teamserver/model/IlrGlobalCache.buildQueryEnviron *
    * ment(IlrGlobalCache.java:872(Compiled Code))                 *
    *                                                              *
    * 4XESTACKTRACE        at                                      *
    * ilog/rules/teamserver/model/IlrGlobalCache.getQueryEnvironme *
    * nt(IlrGlobalCache.java:854(Compiled Code))                   *
    *                                                              *
    * 4XESTACKTRACE        at                                      *
    * ilog/rules/teamserver/model/IlrGlobalCache.getQueryVariableP *
    * rovider(IlrGlobalCache.java:545(Compiled Code))              *
    *                                                              *
    * 4XESTACKTRACE        at                                      *
    * ilog/rules/teamserver/model/IlrSessionHelperEx.getQueryVaria *
    * bleProvider(IlrSessionHelperEx.java:1415(Compiled Code))     *
    *                                                              *
    * 4XESTACKTRACE        at                                      *
    * ilog/rules/teamserver/model/impl/IlrAbstractSession.getQuery *
    * VariableProvider(IlrAbstractSession.java:359(Compiled Code)) *
    *                                                              *
    * 4XESTACKTRACE        at                                      *
    * ilog/rules/teamserver/model/impl/IlrAbstractCachingSession.g *
    * etQueryVariableProvider(IlrAbstractCachingSession.java:900(C *
    * ompiled Code))                                               *
    *                                                              *
    * 4XESTACKTRACE        at                                      *
    * ilog/rules/teamserver/model/IlrModelUtil.getBQLSyntaxTree(Il *
    * rModelUtil.java:597(Compiled Code))                          *
    *                                                              *
    * 4XESTACKTRACE        at                                      *
    * ilog/rules/teamserver/model/IlrModelUtil.getBQLSyntaxTree(Il *
    * rModelUtil.java:593(Compiled Code))                          *
    *                                                              *
    * 4XESTACKTRACE        at                                      *
    * ilog/rules/teamserver/model/IlrModelUtil.getEClassCondition( *
    * IlrModelUtil.java:520(Compiled Code))                        *
    *                                                              *
    * 4XESTACKTRACE        at                                      *
    * ilog/rules/teamserver/web/tree/impl/ruleexplorer/IlrRuleExpl *
    * orerDataProvider.getEClass(IlrRuleExplorerDataProvider.java: *
    * 1559(Compiled Code))                                         *
    ****************************************************************
    * RECOMMENDATION:                                              *
    ****************************************************************
    

Problem conclusion

  • The code is fixed.
    

Temporary fix

Comments

APAR Information

  • APAR number

    RS03574

  • Reported component name

    WS DECISION CTR

  • Reported component ID

    5725B6900

  • Reported release

    890

  • Status

    CLOSED PER

  • PE

    NoPE

  • HIPER

    NoHIPER

  • Special Attention

    NoSpecatt / Xsystem

  • Submitted date

    2020-02-26

  • Closed date

    2020-03-17

  • Last modified date

    2020-03-17

  • APAR is sysrouted FROM one or more of the following:

  • APAR is sysrouted TO one or more of the following:

Fix information

  • Fixed component name

    WS DECISION CTR

  • Fixed component ID

    5725B6900

Applicable component levels

  • R890 PSY

       UP

[{"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSQP76","label":"IBM Operational Decision Manager"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"890","Line of Business":{"code":"LOB45","label":"Automation"}}]

Document Information

Modified date:
24 November 2021