"Can't get Kerberos realm" on yarn cluster

Question

The situation is as follows:

I'm doing this on Windows 7, with MIT Kerberos client kfw 4.0.1. I'm connecting to a YARN cluster, via OpenVPN, that is secured with Kerberos 5. This cluster has been around for a while and it's been in use by other people, so the error is not likely to be on that side of things.

I can get a ticket via kinit (returns without error). However, once I try to do any of the following commands:

• hdfs dfs -ls
• spark-shell --master yarn
• spark-submit anything --master yarn --deploy-mode cluster
• essentially any spark or hadoop command on the cluster

I get the error: Can't get Kerberos realm (or Unable to locate Kerberos realm).

My krb5.ini file is in C:\ProgramData\MIT\Kerberos5

How can I further troubleshoot this?

Answer 1

Your JVM can not locate the krb5.conf file. You have several options:

1. set JVM property: -Djava.security.krb5.conf=/path/to/krb5.conf
2. or put the krb5.conf file into the <jdk-home>/jre/lib/security folder
3. or put the krb5.conf file into the c:\winnt\ folder

More information about locating the krb5.conf file are placed here: https://docs.oracle.com/javase/7/docs/technotes/guides/security/jgss/tutorials/KerberosReq.html