Datastage to Hadoop Question

A forum for discussing DataStage<sup>®</sup> basics. If you're not sure where your question goes, start here.

Moderators: chulett, rschirm, roy

Post Reply
tanumoy2017
Participant
Posts: 6
Joined: Tue Oct 10, 2017 12:32 pm

Datastage to Hadoop Question

Post by tanumoy2017 »

Hi

We are doing a POC on reading and writing data to Hadoop using File Connector Stage but we are getting an error like "Cannot find KDC for realm"

Full Error :

Caused by: java.lang.RuntimeException: java.lang.RuntimeException: javax.security.auth.login.FailedLoginException: Login error: com.ibm.security.krb5.KrbException, status code: 0
message: Cannot find KDC for realm xxx.com
at com.ibm.iis.cc.filesystem.impl.webhdfs.WebHDFSSPNEGO.getAuthCookie(WebHDFSSPNEGO.java:81)


Things done :

1. We are able to get a kerberos ticket with the keytab from the DS Unix Box
2. We test with the curl command below from the DS Box and able to get successful response

curl -i --negotiate -u : http://XXXXX:50070/webhdfs/v1/data/hado ... FILESTATUS

I has few queries like :

1. Do we need to enable webhdfs protocol in Hadoop
2. What are the prerequisite test which we need to do to test connectivity between DS & Hadoop


Thanks
Tanumoy
Last edited by tanumoy2017 on Wed Oct 11, 2017 10:06 am, edited 2 times in total.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

<Moderator: Split this out into its own topic>
-craig

"You can never have too many knives" -- Logan Nine Fingers
Post Reply