Hi All ,
I am using a File Connector stage to write to the HDFS ecosystem. I have used the HttpFS method to connect to the HDFS ecosystem. However when I run the Job I am getting a time out error.
The exact details are given below :-
File_Connector_31,0: com.ascential.e2.common.CC_Exception: An exception occurred: java.net.SocketTimeoutException Read timed out
at com.ibm.iis.cc.filesystem.FileSystemLogger.createCCException (FileSystemLogger.java: 194)
at com.ibm.iis.cc.filesystem.FileSystem.process (FileSystem.java: 743)
at com.ibm.is.cc.javastage.connector.CC_JavaAdapter.run (CC_JavaAdapter.java: 443)
Has anyone encountered such an issue before ? Any inputs here will be really helpful
File Connector stage :-Java related errors
Moderators: chulett, rschirm, roy
Hi All , Just wanted to close this thread saying that I can now connect Hadoop using the file connector stage.
there were 2 main issues. Hadoop cluster had 1 Name node and 2 Data nodes. Ideally opening the firewalls for Name node and corresponding Name Node Port should suffice. However the JOb was failing by showing that it cant connect to the Data Node port which was strange. We have anyway opened up all ports for all Nodes ( Name node and Data Nodes ). Doing this got the issue fixed and the Job started running fine.
Also note that it is important to know which Hadoop you are working on. In our case it was Apache Hadoop which wasn't SSL enabled and not Kerberos enabled which made is a straightforward process actually.
there were 2 main issues. Hadoop cluster had 1 Name node and 2 Data nodes. Ideally opening the firewalls for Name node and corresponding Name Node Port should suffice. However the JOb was failing by showing that it cant connect to the Data Node port which was strange. We have anyway opened up all ports for all Nodes ( Name node and Data Nodes ). Doing this got the issue fixed and the Job started running fine.
Also note that it is important to know which Hadoop you are working on. In our case it was Apache Hadoop which wasn't SSL enabled and not Kerberos enabled which made is a straightforward process actually.