Is File connector able to read orc partitioned files

A forum for discussing DataStage<sup>®</sup> basics. If you're not sure where your question goes, start here.

Moderators: chulett, rschirm, roy

Post Reply
davdav
Participant
Posts: 1
Joined: Wed Feb 15, 2017 3:33 am

Is File connector able to read orc partitioned files

Post by davdav »

I tried to read orc partitioned files with File Connector Stage in DATASTAGE 11.5 but my test job abort with this error:

IIS-CONN-FICO-00030

Code: Select all

An exception occurred: com.ascential.e2.common.CC_Exception: Failed to initialize the parser: java.lang.ClassCastException: org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableTimestampObjectInspector incompatible with org.apache.hadoop.hive.serde2.objectinspector.primitive.HiveVarcharObjectInspector at com.ibm.iis.cc.filesystem.FileSystemLogger.createCCException(FileSystemLogger.java:194) at com.ibm.iis.cc.filesystem.FileSystem.readFile(FileSystem.java:1409) at com.ibm.iis.cc.filesystem.FileSystem.readRecordFiles(FileSystem.java:1289) at com.ibm.iis.cc.filesystem.FileSystem.process(FileSystem.java:759) at com.ibm.is.cc.javastage.connector.CC_JavaAdapter.run(CC_JavaAdapter.java:443)
I searched on the web this error id but I didn't find anything that could help me.

So my question is: File Connector Stage (on DATASTAGE 11.5 ) is able to read orc partitioned files?

Thank you!!!
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Orc files? Ah... some kind of Big Data / Hadoop thing like Hive or Pig or Saruman. :wink:
-craig

"You can never have too many knives" -- Logan Nine Fingers
joycerecacho
Participant
Posts: 298
Joined: Tue Aug 26, 2008 12:17 pm

Re: Is File connector able to read orc partitioned files

Post by joycerecacho »

Hello.

Did you find any solution for this issue?
I read txt file from HDFS using File Connector and have the same problem.
Actually most of times, I just rerun and it works perfectly - without any other action.

Thank you so much.
Joyce
Joyce A. Recacho
São Paulo/SP
Brazil
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Slightly off topic; I've been doing a little work with the Snowflake Connector in version 11.7, and it supports orc partitioning.

So the technology is in there. Joyce has found that (in v11.5 anyway) it doesn't always work first time, but can work later - I'd be taking that information back to your official support provider.
Last edited by ray.wurlod on Wed Apr 03, 2019 10:52 pm, edited 1 time in total.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
joycerecacho
Participant
Posts: 298
Joined: Tue Aug 26, 2008 12:17 pm

Post by joycerecacho »

ray.wurlod wrote:Slightly off topic; I've been doing a little work with the Snowflake Connector in version 11.7, and it supports orc partitioning.

So the technology is in there. Joyce has found that (in v11.5 anyw ...

Ray,
Your post was truncated.
:/

Could you copy the last part?
Thank you so much.
Joyce A. Recacho
São Paulo/SP
Brazil
Post Reply