Datastage Bulk Load to Oracle DB - JSON data

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
DSFreddie
Participant
Posts: 130
Joined: Wed Nov 25, 2009 2:16 pm

Datastage Bulk Load to Oracle DB - JSON data

Post by DSFreddie »

Hi All -

I am facing issue in one Datastage job that reads the data ( Has fields with JSON data, defined as BLOB datatype in the Oracle table) and performs BULK load to the Oracle table. Note that this table doesnt have any constraints defined, so ideally its supposed to perform the load run faster.

The job takes 15 minutes to load 5 Million records. The commit count/Array size gets defaulted to 1 due to the LongVarBinary field. Can someone pls suggest a better way to handle this scenario so that the loads run faster ?


Flow :

Sequential File ----> Oracle Load ( using Oracle Connector)
JSON data ---> Defined as LongVarBinary in Datastage

Thanks
Freddie
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

JSON is usually pure text, so could be defined as CLOB and treated by DataStage as Long VarChar data type. Then you could probably boost your granularity.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
DSFreddie
Participant
Posts: 130
Joined: Wed Nov 25, 2009 2:16 pm

Post by DSFreddie »

Thanks for your reply Ray. One of the reason why it is BLOB is to bring in more flexibility in terms of what we would store in the future.

Are there any other ways in which we can perform a Bulk Load operation against Oracle Exadata table with BLOB fields in it.

Pls help.

Thanks
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Probably, but not through DataStage. BLOB is not a supported data type for DataStage.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Post Reply