Troubleshooting
Problem
You are using Guardium Ranger HDFS Integration with Kerberos authentication and you see messages similar to the following in the stap.log:
HDFS: unable to list files in [hdfs://myhost.com:8020/ranger/audit/kafka/kafka|]
21/06/22 10:04:21 WARN ipc.Client: Exception encountered while connecting to the server : javax.security.sasl.SaslException: GSS initiate failed \[Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)\]"
21/06/25 11:37:25 WARN ipc.Client: Exception encountered while connecting to the server : javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]
hdfsListDirectory(hdfs://myhost.com8020/ranger/audit/kafka/kafka): FileSystem#listStatus error:
GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)java.io.IOException: DestHost:destPort myhost.com:8020 , LocalHost:localPort myhost-s2.x.x/x.x.1.2:0. Failed on local exception: java.io.IOException: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt.
Cause
Getting a ticket from Kerberos for data collection to occur is failing. Either the username is no longer authorized or the keytab configured is no longer valid.
Diagnosing The Problem
The klist command gives some information about the Kerberos ticket cache, for example:
klist
Ticket cache: FILE:/tmp/krb5cc_574100300
Default principal: rangeradmin/myhost.com@MYDOMAIN.COM
Valid starting Expires Service principal
06/22/2021 14:26:04 06/23/2021 00:26:04 krbtgt/MYDOMAIN.COM@MYDOMAIN.COM
renew until 06/29/2021 14:26:04
In order for data collection to occur, the STAP needs a Kerberos ticket. It gets it on the system by using the guard_tap.ini parameters:
ranger_hdfs_keytab=/opt/cloudera/parcels/kafkaaccaesit/ranger.keytab
ranger.hdfs_user=rangeradmin/myhost.com@MYDOMAIN.COM
And running the kinit command, for example:
kinit -kt /opt/cloudera/parcels/kafkaccaesit/ranger.keytab -c /opt/guardium/modules/STAP/11.3.0.0_r109764_1-1620252442/hdfs_reader_ticket randeradmin/myhost.com@MYDOMAIN.COM
Either the username set for ranger.hdfs_user is no longer authorized or the keytab in ranger_hdfs_keyta is invalid. If the username is shared with other software and that is working then the problem is with the keytab.
Resolving The Problem
Verify the guard_tap.ini settings for Kerberos are valid. If the user has a valid ticket in Kerberos, the STAP should be able to find this ticket.
Document Location
Worldwide
[{"Type":"MASTER","Line of Business":{"code":"LOB24","label":"Security Software"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSMPHH","label":"IBM Security Guardium"},"ARM Category":[{"code":"a8m0z000000Gp0IAAS","label":"STAP"}],"ARM Case Number":"TS005649365","Platform":[{"code":"PF016","label":"Linux"}],"Version":"11.3.0;and future releases"}]
Was this topic helpful?
Document Information
Modified date:
26 August 2021
UID
ibm16483691