DSXchange

Thanks Rob4732 - that worked perfectly

Exactly. I have been using the file connector stage now and its a better / faster way to put in data onto Hadoop rather than use a Hive or ODBC connector stage.
The other advantage is that the file connector stage also provides an option to create a Hive table as well which is like 2 steps in one.

Moreover , personally I think it is lot more simpler to use the file connector stage in comparison to the BDFS stage. the BDFS stage in fact is more suited for the Big Insight Hadoop version distributed by IBM ( that doesn't mean it cant connect to other Hadoop versions ). The file connector stage a...

I have been suggested by IBM to run some patches. Will install this and then update .

I have just got some patches to be installed on my services / engine tier as suggested by IBM . This may improve the speed. Will do that and then revert back with my findings

Hi All , Just wanted to close this thread saying that I can now connect Hadoop using the file connector stage. there were 2 main issues. Hadoop cluster had 1 Name node and 2 Data nodes. Ideally opening the firewalls for Name node and corresponding Name Node Port should suffice. However the JOb was f...

Hi All , Just wanted to close this thread saying that I can now connect Hadoop using the file connector stage. there were 2 main issues. Hadoop cluster had 1 Name node and 2 Data nodes. Ideally opening the firewalls for Name node and corresponding Name Node Port should suffice. However the JOb was f...

Hey All, I am trying to use the File Connector Stage to put a file on HDFS. I am getting the following issue when I write a file on HDFS File_Connector_35,0: An exception occurred: java.lang.Exception: Failed to write to file /appl/iip_data/ssd01/data/abhi.txt: HTTP/1.1 403 Forbidden Atul / Satish -...

Yes it is and we have just managed a connection to the Hive database using the ODBC stage and Hive Connector

Thanks Chulett .

That's what I am also thinking but not sure where to do that from. I checked on the net and on some places it is mentioned to perhaps log on to the websphere and make changes to the JVM settings in there. Let me start looking in that direction

Hi Chulett Thanks for your response. This issue is now fixed . I contacted the Hadoop administration team and realized there was some patching activity because of which this issue was cropping. The new problem now is that the performance of the Hive connector stage is pathetic. Opened a case with IB...

Hi All , I am using a File Connector stage to write to the HDFS ecosystem. I have used the HttpFS method to connect to the HDFS ecosystem. However when I run the Job I am getting a time out error. The exact details are given below :- File_Connector_31,0: com.ascential.e2.common.CC_Exception: An exce...

Hi All, I am in the process of using the BDFS to connect to Hadoop . I have gone through the steps and there is one step that states the following To use the Big Data File stage, you must ensure that the stage can access the HDFS libhdfs.so shared library, its associated JAR libraries, and configura...

Hi All , I have recently developed a job to connect to the Hive database in the Hadoop Ecosystem. Now I have used two methods to connect to the Hive database which are :- 1. ODBC Connector 2. Hive Connector However , I am facing massive performance issues with the hive connector stage . Its taking h...

Hi , I have been facing a similar issue. I am using the Hive Connector Stage to load / extract data . However , the speed is dismal . Is there something that we can do to improve the performance of loading into Hive. Having said that , I dont expect Hive loading to be as fast as any other database ,...