DSXchange: DataStage and IBM Websphere Data Integration Forum
View next topic
View previous topic
Add To Favorites
Author Message
ksingh83



Group memberships:
Premium Members

Joined: 15 Aug 2017
Posts: 1

Points: 7

Post Posted: Thu Aug 17, 2017 10:37 am Reply with quote    Back to top    

DataStage® Release: 11x
Job Type: Parallel
OS: Unix
Hi

We have very high volume of source around 20 millions records. How can we effectively load target table?

Is the splitting of files only option? Can we use loops as another option?

Thanks
UCDI



Group memberships:
Premium Members

Joined: 21 Mar 2016
Posts: 261

Points: 2643

Post Posted: Thu Aug 17, 2017 1:06 pm Reply with quote    Back to top    

high volume is relative ... we have loaded more than 5x that, it just takes time. You are not pushing any limits of the DS tool if your environment/server can handle it, in other words.

Did you configure the database connector load stage for a bulk load instead of insert mode?

What kind of DB is the target?
Loops work, splitting up the work can do it, there may be other system specific tricks. What do you need from it? A long running large load isnt always bad, if you don't mind waiting for it.
Rate this response:  
Not yet rated
Display posts from previous:       

Add To Favorites
View next topic
View previous topic
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum



Powered by phpBB © 2001, 2002 phpBB Group
Theme & Graphics by Daz :: Portal by Smartor
All times are GMT - 6 Hours