Troubleshooting
Problem
Whilst attempting to use a HiveServer2 JDBC connection, the connection hangs providing no error to the client for diagnosis.
Symptom
Attempting the connection, the command prompt is not presented.
Cause
In BigInsights, the default maximum number of HiveServer2 worker threads is 100; when this threshold has been reached a new connection attempt will hang.
Environment
Hive Server is delivered as a component of IBM BigInsights; the default settings may not be enough to support the demand for the number of connections.
Diagnosing The Problem
1. Establish the port that the Hive Server JDBC listener is running on by reviewing:
$BIGINSIGHTS_HOME/hdm/components/hive/conf/hive-site.xml:
<property>
<name>hive.server2.thrift.port</name>
<value>10000</value>
</property>
2. Establish how many current connections there are to the HiveServer2 JDBC listener:
netstat -anp| awk '{print $4}'| grep :10000 | wc -l;
This counts the server listener and connected clients
3. If this count is greater than the number of allocated worker threads plus one, then review the hive log $BIGINSIGHTS_VAR/hive/logs/hive/hive.log for the following error:
ERROR thrift.ThriftCLIService (ThriftBinaryCLIService.java:run(114)) - Error:
java.util.concurrent.RejectedExecutionException: Task org.apache.thrift.server.TThreadPoolServer$WorkerProcess@e94782b5 rejected from java.util.concurrent.ThreadPoolExecutor@3fda3aea[Running, pool size = 10, active threads = 10, queued tasks = 0, completed tasks = 0]
Resolving The Problem
Increase the number of worker threads allocated to the HiveServer 2 JDBC listener:
1. Edit the file $BIGINSIGHTS_HOME/hdm/components/hive/conf/hive-site.xml
2. Increase the number of worker threads to cater for the expected number of connections:
<property>
<name>hive.server2.thrift.max.worker.threads</name>
<value>500</value>
<property>
3. Move the configuration for Hive live by running syncconf.sh hive.
4. Restart hive by running stop.sh hive; start.sh hive
Was this topic helpful?
Document Information
Modified date:
18 July 2020
UID
swg21960847