Troubleshooting
Problem
Unable to push policies from a central manager to one of the collectors.
Error: "system error, please contact your administrator"
Manually install of the policy from within the collector also fails.
Firewalls are open. Restarted the units did not resolve the issue.
Pushing the same policy works on three other collectors.
All appliances are at the same patch level.
Pushing the same policy works on three other collectors.
All appliances are at the same patch level.
Diagnosing The Problem
Collector02.local> support check tables TURBINE QRTZ%
All tables checked okay.
mysql> select * from GDM_POLICY_STATISTICS limit 1 \G
************************** 1. row **************************
POLICY_STATISTICS_ID: 1
START_TIME: 2022-03-30 20:15:33
END_TIME: 2022-03-30 20:25:33
POLICY_STATS: {"installedTimestamp": "2022-03-30 20:12:33", "policies": [{"rules": [
{"fire": 0, "actions": \[51\], "ruleDesc": "Ignore Data Activity for Unknown Connections", "match": 0, "rulePosition": 1}
], "policyDesc": "Default - Ignore Data Activity for Unknown Connections [template]"}], "totalEvaluation": 0}
TYPE: 1
TIMESTAMP: 2022-03-30 20:25:33
1 row in set (0.00 sec)
mysql> select min(TIMESTAMP), max(TIMESTAMP) from GDM_POLICY_STATISTICS;
+--------------------+--------------------+
| min(TIMESTAMP)| max(TIMESTAMP)|
+--------------------+--------------------+
| 2022-03-30 20:25:33 | 2022-04-15 14:23:47 |
+--------------------+--------------------+
1 row in set (0.01 sec)
systemdb mustgathers:
debug.log (CM)
Thread: http-bio-8443-exec-49 - java.sql.SQLSyntaxErrorException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')' at line 1
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:120)
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:97)
at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:122)
at com.mysql.cj.jdbc.StatementImpl.executeQuery(StatementImpl.java:1200)
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:120)
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:97)
at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:122)
at com.mysql.cj.jdbc.StatementImpl.executeQuery(StatementImpl.java:1200)
debug.log (Collector)
Thread: http-bio-8443-exec-8 - org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://xxx.xxx.xxx.xxx:8983/solr/ACCESS_1: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
<title>Error 404 javax.servlet.UnavailableException: Error processing the request. CoreContainer is either not initialized or shutting down.</title>
</head>
<body><h2>HTTP ERROR 404 javax.servlet.UnavailableException: Error processing the request. CoreContainer is either not initialized or shutting down.</h2>
<table>
<tr><th>URI:</th><td>/solr/ACCESS_1/admin/ping</td></tr>
<tr><th>STATUS:</th><td>404</td></tr>
<tr><th>MESSAGE:</th><td>javax.servlet.UnavailableException: Error processing the request. CoreContainer is either not initialized or shutting down.</td></tr>
<tr><th>SERVLET:</th><td>default</td></tr>
<tr><th>CAUSED BY:</th><td>javax.servlet.ServletException: javax.servlet.UnavailableException: Error processing the request. CoreContainer is either not initialized or shutting down.</td></tr>
<tr><th>CAUSED BY:</th><td>javax.servlet.UnavailableException: Error processing the request. CoreContainer is either not initialized or shutting down.</td></tr>
</table>
<h3>Caused by:</h3><pre>javax.servlet.ServletException: javax.servlet.UnavailableException: Error processing the request. CoreContainer is either not initialized or shutting down.
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:162)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:322)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.Server.handle(Server.java:500)
.
.
.
==============Fri May 06 12:36:18 EDT 2022===================
Thread: http-bio-8443-exec-8 - org.quartz.SchedulerException: Could not find job details for : policyInstallJob
at com.guardium.rest.core.JobSchedulerCore.getSchedule(JobSchedulerCore.java:386)
at com.guardium.rest.core.JobSchedulerCore.getScheduler(JobSchedulerCore.java:317)
at com.guardium.rest.core.JobSchedulerCore.getScheduler(JobSchedulerCore.java:348)
at com.guardium.rest.util.ScheduleRESTProcessor.doGet(ScheduleRESTProcessor.java:121)
at com.guardium.rest.RestController.handleRequest(RestController.java:65)
.
.
.
==============Fri May 06 12:48:45 EDT 2022===================
Thread: http-bio-8443-exec-5 - java.sql.SQLSyntaxErrorException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')' at line 1
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:120)
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:97)
at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:122)
at com.mysql.cj.jdbc.StatementImpl.executeQuery(StatementImpl.java:1200)
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
<title>Error 404 javax.servlet.UnavailableException: Error processing the request. CoreContainer is either not initialized or shutting down.</title>
</head>
<body><h2>HTTP ERROR 404 javax.servlet.UnavailableException: Error processing the request. CoreContainer is either not initialized or shutting down.</h2>
<table>
<tr><th>URI:</th><td>/solr/ACCESS_1/admin/ping</td></tr>
<tr><th>STATUS:</th><td>404</td></tr>
<tr><th>MESSAGE:</th><td>javax.servlet.UnavailableException: Error processing the request. CoreContainer is either not initialized or shutting down.</td></tr>
<tr><th>SERVLET:</th><td>default</td></tr>
<tr><th>CAUSED BY:</th><td>javax.servlet.ServletException: javax.servlet.UnavailableException: Error processing the request. CoreContainer is either not initialized or shutting down.</td></tr>
<tr><th>CAUSED BY:</th><td>javax.servlet.UnavailableException: Error processing the request. CoreContainer is either not initialized or shutting down.</td></tr>
</table>
<h3>Caused by:</h3><pre>javax.servlet.ServletException: javax.servlet.UnavailableException: Error processing the request. CoreContainer is either not initialized or shutting down.
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:162)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:322)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.Server.handle(Server.java:500)
.
.
.
==============Fri May 06 12:36:18 EDT 2022===================
Thread: http-bio-8443-exec-8 - org.quartz.SchedulerException: Could not find job details for : policyInstallJob
at com.guardium.rest.core.JobSchedulerCore.getSchedule(JobSchedulerCore.java:386)
at com.guardium.rest.core.JobSchedulerCore.getScheduler(JobSchedulerCore.java:317)
at com.guardium.rest.core.JobSchedulerCore.getScheduler(JobSchedulerCore.java:348)
at com.guardium.rest.util.ScheduleRESTProcessor.doGet(ScheduleRESTProcessor.java:121)
at com.guardium.rest.RestController.handleRequest(RestController.java:65)
.
.
.
==============Fri May 06 12:48:45 EDT 2022===================
Thread: http-bio-8443-exec-5 - java.sql.SQLSyntaxErrorException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')' at line 1
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:120)
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:97)
at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:122)
at com.mysql.cj.jdbc.StatementImpl.executeQuery(StatementImpl.java:1200)
Resolving The Problem
The problem is that the customer is trying to install and override policy when no policy exists/is installed on that collector.
Solution:
Click to install and DO NOT USE THE OVERRIDE OPTION.
Once there is a policy was installed on the collector, we confirmed, checking the override option then worked as designed to push a new policy down.
Document Location
Worldwide
[{"Type":"MASTER","Line of Business":{"code":"LOB24","label":"Security Software"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSMPHH","label":"IBM Security Guardium"},"ARM Category":[{"code":"a8m0z000000Gp0OAAS","label":"POLICY"}],"ARM Case Number":"TS009118250","Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Versions"}]
Was this topic helpful?
Document Information
Modified date:
11 January 2023
UID
ibm16854657