Datastage Loading in Oracle table Speed
Posted: Wed Oct 18, 2017 9:52 am
Hi,
I'm loading one Oracle table with around 19 million records. I was checking speed of the process by making changes in the job. My job had a fetch query and some sort stages, remove duplicates, joins, a transformer, a funnel and ultimately loads to an Oracle table.
This job was taking around 3 hrs among which 2 hr 15 min was just loading into the Oracle table. Then I split this job in two jobs by changing the Oracle connector load to a dataset and adding a second job moving the dataset to Oracle connector table load.
Both jobs in second case took 49 minutes with only 6 minutes for load to Oracle. I'm wondering why the second case the load took just 6 minutes in comparison with 2 hr 15 min in first case. Could you please advise reason for this?
Thanks
[Note - edited for clarity - Andy]
I'm loading one Oracle table with around 19 million records. I was checking speed of the process by making changes in the job. My job had a fetch query and some sort stages, remove duplicates, joins, a transformer, a funnel and ultimately loads to an Oracle table.
This job was taking around 3 hrs among which 2 hr 15 min was just loading into the Oracle table. Then I split this job in two jobs by changing the Oracle connector load to a dataset and adding a second job moving the dataset to Oracle connector table load.
Both jobs in second case took 49 minutes with only 6 minutes for load to Oracle. I'm wondering why the second case the load took just 6 minutes in comparison with 2 hr 15 min in first case. Could you please advise reason for this?
Thanks
[Note - edited for clarity - Andy]