Troubleshooting
Problem
At times, when adding artifacts to incidents, the Hits show as spinning and the scan never completes.
Cause
VirusTotal has changed its public key usage restrictions. According to https://developers.virustotal.com/reference#public-vs-private-api:
- The Public API is limited to 500 requests per day and a rate of 4 requests per minute.
- The Public API must not be used in commercial products, services or business workflows.
- The Private API returns more threat data and exposes more endpoints.
- The Private API is governed by an SLA that guarantees readiness of data.
- The Public API is limited to 500 requests per day and a rate of 4 requests per minute.
- The Public API must not be used in commercial products, services or business workflows.
- The Private API returns more threat data and exposes more endpoints.
- The Private API is governed by an SLA that guarantees readiness of data.
The new daily quota is too low for most customers using VirusTotal's public key. Artifacts submitted for scanning after exceeding the daily quota will wait until the next day. During this time, the artifact Hits will spin. Artifact lookups using public keys may never complete.
Environment
Diagnosing The Problem
Configure IBM Security SOAR to output different logging levels to the client.log
- Create /crypt/logback-custom.xml (sudo vi /crypt/logback-custom.xml)
- Add content in line with the following examples depending on the problem that requires debugging
- Save /crypt/logback-custom.xml
- Set the permissions by running sudo chown root:co3 /crypt/logback-custom.xml
- Restart IBM Security SOAR by running sudo systemctl restart resilient.service
- Debug output will be sent to /usr/share/co3/logs/client.log
Output levels:
• FATAL
• ERROR
• WARN
• INFO
• DEBUG
• TRACE
Enable DEBUG for Threat Service
<included> <logger name="com.co3.threat" level="DEBUG"> <appender-ref ref="Co3File" /> </logger> </included>
Threat Source result status:
0 = no hits
1 = in progress
2 = hit
Below are results when artifact Hits are spinning, showing no Artifact query response and the Threat source result status =1
Now: Mon Mar 08 23:59:34 UTC 2021
18:59:34.283 [Camel (camel-1) thread #9 - JmsConsumer[interprocessevents.schedule-service]] DEBUG [] com.co3.threat.Co3ThreatFeed - Threat source result status: 1, Artifact Value: 8.8.8.8, ThreatSource: VirusTotal, run after seconds: 60, Now: Mon Mar 08 23:59:34 UTC 2021
19:00:36.333 [Camel (camel-1) thread #5 - JmsConsumer[interprocessevents.schedule-service]] DEBUG [] com.co3.threat.Co3ThreatFeed - Number of ThreatSource: 1 for artifact: 8.8.8.8
19:00:36.333 [Camel (camel-1) thread #5 - JmsConsumer[interprocessevents.schedule-service]] DEBUG [] com.co3.threat.Co3ThreatFeed - Number of ThreatSource: 1 for artifact: 8.8.8.8
19:00:36.338 [Camel (camel-1) thread #5 - JmsConsumer[interprocessevents.schedule-service]] DEBUG [] com.co3.threat.Co3ThreatFeed - Threat source result status: 1, Artifact Value: 8.8.8.8, ThreatSource: VirusTotal
19:00:36.338 [Camel (camel-1) thread #5 - JmsConsumer[interprocessevents.schedule-service]] DEBUG [] com.co3.threat.Co3ThreatFeed - Threat source result status: 1, Artifact Value: 8.8.8.8, ThreatSource: VirusTotal
19:00:36.342 [Camel (camel-1) thread #5 - JmsConsumer[interprocessevents.schedule-service]] DEBUG [] com.co3.threat.Co3ThreatFeed - Threat source result status: 1, Artifact Value: 8.8.8.8, ThreatSource: VirusTotal, run after seconds: 60,
Below are results when artifact Hits complete, showing the Artifact query response and the Threat source result=2
Now: Tue Mar 09 00:01:38 UTC 2021
19:01:38.305 [Camel (camel-1) thread #3 - JmsConsumer[interprocessevents.schedule-service]] DEBUG [] com.co3.threat.Co3ThreatFeed - Threat source result status: 1, Artifact Value: 8.8.8.8, ThreatSource: VirusTotal, run after seconds: 60, Now: Tue Mar 09 00:01:38 UTC 2021
19:02:39.999 [Camel (camel-1) thread #9 - JmsConsumer[interprocessevents.schedule-service]] DEBUG [] com.co3.threat.Co3ThreatFeed - Artifact query response from url: https://websvc.resilientsystems.com/rest/artifacts/10933f140989f1c42ce7fe7bda4869d616255ba5a46dd0a11f643486e565bd0a.140827603, content: {"recheck_secs":60,"artifact_results":[{"id":"10933f140989f1c42ce7fe7bda4869d616255ba5a46dd0a11f643486e565bd0a.140827603","threat_sources":[{"id":"5de07056-8448-4155-84a9-a8d1f817c2b7","status":2,"items":[{"value":"8.8.8.8","artifact_type_id":0,"properties":{"total":"86","positives":"13","permalink":"https://www.virustotal.com/ip-address/8.8.8.8/information/","scanDate":"2021-03-08 23:11:00"},"active":true}]}]}]}
19:02:39.999 [Camel (camel-1) thread #9 - JmsConsumer[interprocessevents.schedule-service]] DEBUG [] com.co3.threat.Co3ThreatFeed - Artifact query response from url: https://websvc.resilientsystems.com/rest/artifacts/10933f140989f1c42ce7fe7bda4869d616255ba5a46dd0a11f643486e565bd0a.140827603, content: {"recheck_secs":60,"artifact_results":[{"id":"10933f140989f1c42ce7fe7bda4869d616255ba5a46dd0a11f643486e565bd0a.140827603","threat_sources":[{"id":"5de07056-8448-4155-84a9-a8d1f817c2b7","status":2,"items":[{"value":"8.8.8.8","artifact_type_id":0,"properties":{"total":"86","positives":"13","permalink":"https://www.virustotal.com/ip-address/8.8.8.8/information/","scanDate":"2021-03-08 23:11:00"},"active":true}]}]}]}
19:02:40.005 [Camel (camel-1) thread #9 - JmsConsumer[interprocessevents.schedule-service]] DEBUG [] com.co3.threat.Co3ThreatFeed - Number of ThreatSource: 1 for artifact: 8.8.8.8
19:02:40.005 [Camel (camel-1) thread #9 - JmsConsumer[interprocessevents.schedule-service]] DEBUG [] com.co3.threat.Co3ThreatFeed - Number of ThreatSource: 1 for artifact: 8.8.8.8
19:02:40.009 [Camel (camel-1) thread #9 - JmsConsumer[interprocessevents.schedule-service]] DEBUG [] com.co3.threat.Co3ThreatFeed - Threat source result status: 2, Artifact Value: 8.8.8.8, ThreatSource: VirusTotal
19:02:40.009 [Camel (camel-1) thread #9 - JmsConsumer[interprocessevents.schedule-service]] DEBUG [] com.co3.threat.Co3ThreatFeed - Threat source result status: 2, Artifact Value: 8.8.8.8, ThreatSource: VirusTotal
Resolving The Problem
Public keys are meant for personal use. All enterprise customers are required to obtain a private key.
For assistance with VirusTotal, please contact them at https://www.virustotal.com/gui/contact-us
Related Information
Document Location
Worldwide
[{"Line of Business":{"code":"LOB24","label":"Security Software"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSIP9Q","label":"IBM Security SOAR"},"ARM Category":[{"code":"a8m0z0000001hW8AAI","label":"Resilient Core->Threat Services"}],"ARM Case Number":"","Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Version(s)"}]
Was this topic helpful?
Document Information
Modified date:
26 March 2021
UID
ibm16425903