Calling the same job multiple times in parallel

A forum for discussing DataStage<sup>®</sup> basics. If you're not sure where your question goes, start here.

Moderators: chulett, rschirm, roy

Post Reply
vgupta88
Participant
Posts: 32
Joined: Thu Aug 01, 2013 12:26 am
Location: Mumbai

Calling the same job multiple times in parallel

Post by vgupta88 »

Hi All,

I have a sequence that has several number of job activities in it. All these call the same job which was set to run multiple times(check Multiple instance). The invocation id was specified on all the job activities was different. But the sequence was not running parallel, moreover it is failing with the below error -

Exception raised: Error calling DSRunJob(job_name), code=-2 [Job is not in the right state (compiled and not running)]

Please let me know how can I achieve this?

Thanks!
Vaibhav
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Okay... that's almost word for word was is in the other post and you can see that really wasn't enough information for someone to be able to help. :?

So this a single Sequence job that is somehow trying to run a multi-instance job more than once "in parallel" and that's failing? Can you be more specific as to how your Sequence is coded to achieve this? Assuming you are correct in the assertion that the Invocation ID is in fact unique, I would tend to suspect that perhaps one invocation of the job has not fully started up before you are attempting to start the next invocation. But at this point it's just a guess.

Thanks.
-craig

"You can never have too many knives" -- Logan Nine Fingers
vgupta88
Participant
Posts: 32
Joined: Thu Aug 01, 2013 12:26 am
Location: Mumbai

Post by vgupta88 »

Hi Craig,

Below is my job design. After Job 1 is completed, it calls Job 2 with different Invocation ID's in parallel.
All three jobs i.e. Job 1, Job 2 and the main sequencer are Multiple Instance jobs.

Code: Select all

Job 2 
 ^
 |
 |
Job 1 -------> Job 2 
 | 
 | 
 v 
Job 2
Can I achieve this? Appreciate your help!

Thanks!
Vaibhav
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Out of curiosity, why is the main Sequence job multi-instance - are you running multiple copies of it as well?

Unless someone has a better idea as to what might be going on, in your shoes I'd try adding a delay to two for the three Job Activity stages. Build a routine to sleep X seconds (it's just an O/S call) and insert it before the 2nd and 3rd activities. Try values like 10 and 20 seconds or 20 and 40 so that there is a brief pause between the parallel invocations. See if that helps.
-craig

"You can never have too many knives" -- Logan Nine Fingers
vgupta88
Participant
Posts: 32
Joined: Thu Aug 01, 2013 12:26 am
Location: Mumbai

Post by vgupta88 »

Yes, my main sequence also runs multiple times.

My job runs for few minutes so I am not sure if adding few seconds sleep would help. Also, if I do that how I am going to achieve parallelism? Your thoughts please. However, I am going to try on your suggestion and will let you know the outcome. Thanks!
Vaibhav
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

It's not about the runtime and wouldn't affect the "parallelism". I'm just suggesting a few seconds difference in the launch time for the three simultaneous instances of Job2 so they all don't try to start at once, make them a teeny bit less... simultaneous. This so the "startup" of each can complete before the next is requested since they are all really just one job under the covers.
-craig

"You can never have too many knives" -- Logan Nine Fingers
R.K.Glover
Participant
Posts: 8
Joined: Mon Mar 11, 2013 2:51 pm
Location: RTP, North Carolina

Post by R.K.Glover »

vgupta88 wrote:Yes, my main sequence also runs multiple times.

My job runs for few minutes so I am not sure if adding few seconds sleep would help. Also, if I do that how I am going to achieve parallelism? Your thoughts please. However, I am going to try on your suggestion and will let you know the outcome. Thanks!
If the main sequence is running multiple times, and it's spawning off multiple copies of the second job, is it possible that each instance of the main job is spawning off instances that have the same instance name?

e.g.

Code: Select all

Job1.Invoc_One calls Job2.Invoc_One and Job.Invoc_Two
Job1.Invoc_Two calls Job2.Invoc_One and Job.Invoc_Two
... would fail if Job2.Invoc_One from Job1.Invoc_One was running when Job1.Invoc_Two tried starting Job2.Invoc_One
PaulVL
Premium Member
Premium Member
Posts: 1315
Joined: Fri Dec 17, 2010 4:36 pm

Post by PaulVL »

I agree with R.K., your invocation ID for Job2 should now have two qualifiers. the Invocation for your sequence and another for the 1/2/3 aspect.

sequence.invocation1 -> Job1.invocation1 -> job2.invocation1_1, job2.invocation1_2, job2.invocation1_3

sequence.invocation2 -> Job1.invocation2 -> job2.invocation2_1, job2.invocation2_2, job2.invocation2_3

etc...
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Well... hopefully all figured out by now. :wink:

My previous attempts to help were all predicated on the assumption that, as stated by the OP, the Invocation IDs were all in fact unique.
-craig

"You can never have too many knives" -- Logan Nine Fingers
Post Reply