Calling Stored Procedure without loosing parellelism

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
jacksimpk
Participant
Posts: 1
Joined: Thu Aug 06, 2009 12:39 pm

Calling Stored Procedure without loosing parellelism

Post by jacksimpk »

I'm trying to load and Oracle 10g table using Datastage 8.0.1. My problem comes in when I try to load in replace mode the DBA's have set the privleges so user accounts cannot truncate tables. To truncate a table I have to run a stored procedure.

I solved this problem by using the "Open Command" option in the Oracle Enterprise stage. But as with many things I neglacted to think about parellelism. The stage run the truncation procedure once per node. So in production its tring to trunc the table 4 times simultaineously. This does not work real well as one can imagine.

I've looked into using the Stored procedure stage but it seems like if I put it in my job I'll loose parellelism and have to rehash the partitions to load in parellel.

My current alternative is to build an independant job to preceed my oracle load that will run sequencially. And whose only function is to truncate the table. It seems like there should be a better solution than that.

thanks to any pointers or suggestions,
Jack
Jack Simpkins
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Re: Calling Stored Procedure without loosing parellelism

Post by chulett »

jacksimpk wrote:My current alternative is to build an independant job to preceed my oracle load that will run sequencially. And whose only function is to truncate the table. It seems like there should be a better solution than that.
I'll be curious what others have to say but to me this is a perfectly valid solution and one I've put in place multiple times. I typically have a multi-instance server job as one of the 'Utility' jobs that just truncates a table and the InvocationId is the name of the table to truncate. Works well everywhere I've used it so far.

Food for thought.
-craig

"You can never have too many knives" -- Logan Nine Fingers
kumar_s
Charter Member
Charter Member
Posts: 5245
Joined: Thu Jun 16, 2005 11:00 pm

Post by kumar_s »

Why not Truncating the table on Before job subroutine with a sqlplus call?
Impossible doesn't mean 'it is not possible' actually means... 'NOBODY HAS DONE IT SO FAR'
kumar_s
Charter Member
Charter Member
Posts: 5245
Joined: Thu Jun 16, 2005 11:00 pm

Post by kumar_s »

I was searching for something and replied to this post on reflex without noticing that the post is an year old. :lol:
Impossible doesn't mean 'it is not possible' actually means... 'NOBODY HAS DONE IT SO FAR'
Post Reply