DSXchange: DataStage and IBM Websphere Data Integration Forum
View next topic
View previous topic
Add To Favorites
Author Message

Group memberships:
Premium Members

Joined: 01 Apr 2007
Posts: 237
Location: Michigan
Points: 2033

Post Posted: Wed Sep 23, 2020 10:20 am Reply with quote    Back to top    

DataStage® Release: 11x
Job Type: Parallel
OS: Unix
we are joining many tables and doing an order by party_id in a oracle connector stage and using transformer stage in the middle with out doing any transformations we are writing to a file. The record count is 50 million and 1 GB data writing to the file. when we run the query data is sorted but the end file some times we are not getting data sorted.

How can we keep it same.

1. Can we do by using same partition?
2. In the transformer can we use sequential.

we do not have so much volume in dev and we are seeing this issue in production only


Premium Poster

Group memberships:
Premium Members, Inner Circle, Australia Usergroup, Server to Parallel Transition Group

Joined: 23 Oct 2002
Posts: 54595
Location: Sydney, Australia
Points: 296053

Post Posted: Thu Oct 08, 2020 11:31 am Reply with quote    Back to top    

Partition by party_id. Preserve this partitioning throughout the job. "Auto" will give you this for stages executing in parallel mode, or you can explicitly specify "Same" if you prefer. On the ...

IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Rate this response:  
Not yet rated
Display posts from previous:       

Add To Favorites
View next topic
View previous topic
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum

Powered by phpBB © 2001, 2002 phpBB Group
Theme & Graphics by Daz :: Portal by Smartor
All times are GMT - 6 Hours