Hive Connector Stage and isjdbc.config file set up

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
TNZL_BI
Premium Member
Premium Member
Posts: 24
Joined: Mon Aug 20, 2012 5:15 am
Location: NZ

Hive Connector Stage and isjdbc.config file set up

Post by TNZL_BI »

Hi All,

We have recently connected to the Hive database from DataStage using the ODBC Connector Stage. However we are having issues when we are trying to use the Hive Connector stage. I am not exactly sure as to how to set up the isjdbc.config file . I have read the documentation however I am still not clear as to what do I need to put in the File Class and File Path. Do I need to install any hive related Jar files on the DSEngine Linux server ??

Any information here will be really appreciated . Many Thanks
TNZL_BI
Premium Member
Premium Member
Posts: 24
Joined: Mon Aug 20, 2012 5:15 am
Location: NZ

Re: Hive Connector Stage and isjdbc.config file set up

Post by TNZL_BI »

Hi All ,

I have finally managed to connect using the hive connector . I managed to locate the jar file for Hive which was the ISHive.jar file under the ASBNode/lib directory in datastage.

The contents of the config file is as follows :-

-bash-4.2$ cat isjdbc.config
CLASSPATH=/opt/IBM/InformationServer/ASBNode/lib/java/IShive.jar
CLASS_NAMES=org.apache.hadoop.hive.jdbc.HiveDriver

Reading data is working fine however when I write data into Hive I am getting the following error

FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask

Any ideas as to what might be the reason for this
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Have you contacted support for any of this?
-craig

"You can never have too many knives" -- Logan Nine Fingers
TNZL_BI
Premium Member
Premium Member
Posts: 24
Joined: Mon Aug 20, 2012 5:15 am
Location: NZ

Post by TNZL_BI »

Hi Chulett

Thanks for your response. This issue is now fixed . I contacted the Hadoop administration team and realized there was some patching activity because of which this issue was cropping.

The new problem now is that the performance of the Hive connector stage is pathetic. Opened a case with IBM.. will give all the details if I hear anything positive
rkashyap
Premium Member
Premium Member
Posts: 532
Joined: Fri Dec 02, 2011 12:02 pm
Location: Richmond VA

Post by rkashyap »

TNZL_BI wrote:The new problem now is that the performance of the Hive connector stage is pathetic. Opened a case with IBM.. will give all the details if I hear anything positive
What did IBM suggest? Did it help?
skathaitrooney
Participant
Posts: 103
Joined: Tue Jan 06, 2015 4:30 am

Post by skathaitrooney »

Hi TNZL_BI , one qusetion.
Since you are using the driver supplied by IBM (IShive.jar), are you sure the CLASS_NAMES=org.apache.hadoop.hive.jdbc.HiveDriver
and not com.ibm.isf.jdbc.hive.HiveDriver ?
Post Reply