Troubleshooting
Problem
TroubleShooting for ORB problems with IBM WebSphere Application Server. This should help address common issues with this component before calling IBM support and save you time.
Resolving The Problem
< /div>
Troubleshooting ORB problems in IBM WebSphere Application Server. This page should help you address common issues with ORB before engaging IBM support which can save you time in resolving the issue.
Tab navigation
- Learning more
- Troubleshooting- selected tab,
- Collecting data
Troubleshooting topics:
Tab navigation
- Connection Issues
- WLM Issues
- Deadlocks/Hang
- CORBA.MARSHAL- selected tab,
- Logging Issues
Overview
This topic discusses common ORB issues that can lead to the CORBA.MARSHAL error.
Topics Covered:
CORBA.MARSHAL Exceptions
|
Usually CORBA.MARSHAL exceptions fall into 2 main categories:
- Meta callback failures
- Defect in the Base ORB marshaling code
CORBA.MARSHAL due to meta callback failures
NOTE: Callbacks can be initiated on either a server or client, but most of the time the server is making the callback to the client. The following discussion assumes the server is making the callback.
Callbacks can be necessary when either a client or server ORB reads a request or reply message and attempts to demarshal the data into meaningful data structures. A callback to the remote ORB occurs when the version of a class being demarshaled is not exactly the same on both the client and server. Typically, this class version mismatch occurs when the client and server have different JDK versions because many times request/reply messages include basic JDK classes. Mismatches can also occur with business application classes. The callback process rectifies the differences in the structure of a particular class(es) when unmarshaling data.
Sample Server Stack Trace snipit:
org.omg.CORBA.MARSHAL: Unable to read value from underlying bridge … at com.ibm.rmi.corba.ClientDelegate.createRequest(ClientDelegate.java:1121) at com.ibm.CORBA.iiop.ClientDelegate.createRequest(ClientDelegate.java:1253) at com.ibm.rmi.corba.ClientDelegate.request(ClientDelegate.java:1841) at com.ibm.CORBA.iiop.ClientDelegate.request(ClientDelegate.java:1209) at org.omg.CORBA.portable.ObjectImpl._request(ObjectImpl.java:460) at com.ibm.org.omg.SendingContext._CodeBaseStub.meta(_CodeBaseStub.java:115) <<< callback methods at com.ibm.rmi.iiop.FVDCodeBaseDelegate.meta(FVDCodeBaseDelegate.java:145) <<< callback methods at com.ibm.rmi.io.IIOPInputStream.inputObjectFVD(IIOPInputStream.java:1407) at com.ibm.rmi.io.IIOPInputStream.readSerializable(IIOPInputStream.java:1032) at com.ibm.rmi.io.IIOPInputStream.simpleReadObjectInternal(IIOPInputStream.java:317) at com.ibm.rmi.io.IIOPInputStream.simpleReadObject(IIOPInputStream.java:347) at com.ibm.rmi.io.ValueHandlerImpl.readValue(ValueHandlerImpl.java:200) at com.ibm.rmi.iiop.CDRInputStream.read_value(CDRInputStream.java:1961) at com.ibm.engine._EJSRemoteStatelessConnEngine_da67e2e1_Tie.getEngineInfo(_EJSRemoteStatelessConnEngine_da67e2e1_Tie.java:157) at com.ibm.engine._EJSRemoteStatelessConnEngine_da67e2e1_Tie._invoke(_EJSRemoteStatelessConnEngine_da67e2e1_Tie.java:89) at com.ibm.CORBA.iiop.ServerDelegate.dispatchInvokeHandler(ServerDelegate.java:610) …
Sample Client Side Stack:
Trace from server: 1198777258 at host myhost.com >> org.omg.CORBA.MARSHAL: Unable to read value from underlying bridge : initial and forwarded IOR inaccessible vmcid: IBM minor code: 89A completed: No at com.ibm.rmi.iiop.CDRInputStream.read_value(CDRInputStream.java:1968) at com.ibm.tws.conn.engine._EJSRemoteStatelessConnEngine_da67e2e1_Tie.getEngineInfo(_EJSRemoteStatelessConnEngine_da67e2e1_Tie.java:157) at com.ibm.tws.conn.engine._EJSRemoteStatelessConnEngine_da67e2e1_Tie._invoke(_EJSRemoteStatelessConnEngine_da67e2e1_Tie.java:89) at com.ibm.CORBA.iiop.ServerDelegate.dispatchInvokeHandler(ServerDelegate.java:610) at com.ibm.CORBA.iiop.ServerDelegate.dispatch(ServerDelegate.java:463) at com.ibm.rmi.iiop.ORB.process(ORB.java:439) at com.ibm.CORBA.iiop.ORB.process(ORB.java:1737) at com.ibm.rmi.iiop.Connection.respondTo(Connection.java:2350) at com.ibm.rmi.iiop.Connection.doWork(Connection.java:2195) at com.ibm.rmi.iiop.WorkUnitImpl.doWork(WorkUnitImpl.java:65) at com.ibm.ejs.oa.pool.PooledThread.run(ThreadPool.java:95) at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java(Compiled Code)) << END server: 1198777258 at host myhost.com
ORB trace showing meta/FVD trace:
The presence of inputObjectFVD() trace indicates the ORB is attempting to reconcile a mismatched class. Just before that trace line will be the readSerializable trace line indicating the mismatched class. Since the ORB caches the results of meta callbacks, it's possible to see this trace with no subsequent meta callback (the assumption being that the cache held info on the class in question). This trace will be seen soon after reading either an incoming request or reply message.
[5/31/18 1:45:30:344 CDT] 00000202 ORBRas 3 com.ibm.rmi.io.IIOPInputStream readSerializable:1531 MessageListenerThreadPool : 46 clz = class com.cust.class << mismatched class [5/31/18 1:45:30:345 CDT] 00000202 ORBRas 3com.ibm.rmi.io.IIOPInputStream inputObjectFVD ORB.thread.pool : 1 Entry [5/31/17 1:45:30:345 CDT] 00000202 ORBRas 3com.ibm.rmi.io.IIOPInputStream inputObjectFVD:2120 ORB.thread.pool : 1
Possible causes:
Root Cause 1: Firewall is blocking the meta callback
Symptoms: On the server side, the callback fails with a "java.net.ConnectException: Connection refused" but sometimes the failure can also be a connect timeout. In either case, the firewall has blocked the server's attempt to create a new connection to the client over which the meta callback is to be made.
Solution: Set the ORB property com.ibm.CORBA.enableClientCallbacks=true on the client side. This causes the server to send the meta callback request to the client using the already-existing connection from that client.
Root Cause 2: Client is configured with a 127.0.0.1 or "localhost" address.
Symptoms: Usually this problem occurs with thin clients. If the client's ORB property com.ibm.CORBA.LocalHost is set to either "127.0.0.1" or "localhost" (which resolves to 127.0.0.1), AND if the client and server are on separate machines, then a "java.net.ConnectException: Connection refused" occurs on the server. This is because the server is trying to connect to the client via the loopback IP which will fail. The server needs the "real" IP or hostname of the client.
Solution: Set the client's com.ibm.CORBA.LocalHost property to a "real" hostname or IP address (i.e. NOT 127.0.0.1 or "localhost").
Root Cause 3: Since a meta callback is like any other ORB request, it can fail for all sorts of other reasons that a typical request can (NO_RESPONSE, network issues, etc)
Solution: Matching client/server ORB trace is usually needed to further debug the root cause.
CORBA.MARSHAL (non-meta callback cases)
If meta callback failures have been ruled out, then matching client/server ORB trace will be needed. In some scenarios, a small testcase program will be needed to further debug the issue.
Was this topic helpful?
Document Information
Modified date:
04 February 2020
UID
ibm10736063