DSXchange

Posted: **Sat Aug 12, 2017 3:17 pm**

DataStage 11.501; Patches applied: Datadirect JDBC Driver for Hive(JR56156) and Hive Connector(JR56097).

We are trying to use File Connector stage for connecting to Cloudera Hadoop server. Connection parameters are:

Code: Select all

 FileSystem: HttpFS
 Use Custom URL: No
 Use SSL: Yes
 Use Ketab: Yes
 Use Kerberos: Yes
 Host: <Cloudera Server>
 PortNo: 14000
 Service principal: 
 User name: <username@domain>
 Keytab:<Path to keytab file>

Cloudera server is setup with an edge node and we are connecting with edge node as host. As per Hadoop admin's recommendation "HttpFS with SSL" is being used. Truststore has been setup. Keytab is provided by Hadoop Admin and we have installed Kerberos on DS server with krb5.conf file copied from Hadoop server to DataStage server.

File connector job fails with "File_Connector_1: com.ascential.e2.common.CC_Exception: java.lang.Exception: java.lang.RuntimeException: java.lang.RuntimeException: javax.security.auth.login.LoginException: Login Failure: all modules ignored at com.ibm.iis.cc.filesystem.FileSystem.initialize(FileSystem.java:540) ....

Appreciate if someone can advise on what could be the issue.

Posted: **Mon Aug 14, 2017 8:58 pm**

The 'Login Error' seems to point more or less to your authentication parameters (be it the keytab or username or something different altogether).
Have you tried perhaps putting in your FQDN for your domain?
Test out the connectivity via command line on your engine?
Confirm that you can connect via WebHDFS to rule out any networking issues?

Posted: **Tue Aug 29, 2017 10:50 am**

Thank you. Failure was due to convergence of two issues:
1. Bad Keytab. Cloudera admin recreated the keytab file for this.

2. Kerberos ticket was generated using knit from "/usr/sbin/". Ticket had to be regenerated using kinit from IBM's JDK:

Code: Select all

$ kdestroy
$ /opt/IBM/InformationServer/jdk/jre/bin/kinit -k -t <keytab_filename> <principal>

We have also updated the "path" to keep IBM's JDK library above the ones from RHEL.

Posted: **Sun Sep 03, 2017 10:44 pm**

Thank you for posting the solution.

DSXchange

Connecting to Hadoop from FileConnector

Connecting to Hadoop from FileConnector