How To
Summary
We are having issues with our ELK services running on specific servers. We are seeing the following error message.
Objective
Error Message:
Caused by: java.lang.IllegalStateException: failed to obtain node locks, tried [[$EGO_SHARE_TOP/integration/elk/hosts/host1.ibm.com/es.data/elk-conductor_prod]] with lock id [0]; maybe these locations are not writable or multiple nodes were started without increasing [node.max_local_storage_nodes] (was [1])?
Steps
1. Shutdown ELK services and ensure no ELK services is in memory (check all primary/failover hosts)
egosh service stop elk-elasticsearch-master elk-elasticsearch-data elk-elasticsearch elk-indexer elk-shipper elk-manager
ps -ef | grep elasticsearch.bootstrap.Elasticsearch
2. Backup the Following directories:
$EGO_TOP/integration/elk/hosts/es.master/
$EGO_TOP/integration/elk/hosts/bi30.eng.platformlab.ibm.com/es.data/
$EGO_TOP/integration/elk/hosts/bi30.eng.platformlab.ibm.com/es.client
3. Delete all lock files
cd $EGO_TOP/elk/hosts
find -name write.lock -delete
4. Startup the ELK services
egosh service stop elk-elasticsearch-master elk-elasticsearch-data elk-elasticsearch elk-indexer elk-shipper elk-manager
Once ELK Services are started, these write.locks will be regenerated.
Document Location
Worldwide
[{"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SS4H63","label":"IBM Spectrum Conductor"},"Component":"","Platform":[{"code":"PF016","label":"Linux"}],"Version":"All Versions","Edition":"","Line of Business":{"code":"LOB10","label":"Data and AI"}}]
Was this topic helpful?
Document Information
Modified date:
26 April 2021
UID
ibm11107087