Multiple Instance job

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
gayathrisivakumar
Premium Member
Premium Member
Posts: 60
Joined: Thu Dec 17, 2009 1:36 am

Multiple Instance job

Post by gayathrisivakumar »

Hi All,

I have source query that needs to be run for multiple date values.There are 2 date fields that will keep changing.

I need to find certain accounts that changed since 2016-01-01 to 2017-05-31 now (When we enter June that becomes 2016-01-01 to 2017-06-30. So the date range changes every month)
My requirement is to execute the query first for 2016-01-01 to 2016-01-31, then second instance for 2016-02-01 to 2016-02-29, then March etc. So in a single run of the job multiple instances needs to be executed.How can I provide all these dates as multiple inputs?

Thanks
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

It only needs to be multi-instance is they all need to run at the same time. Your need sounds like a Sequence job looping a normal job to me.
-craig

"You can never have too many knives" -- Logan Nine Fingers
UCDI
Premium Member
Premium Member
Posts: 383
Joined: Mon Mar 21, 2016 2:00 pm

Post by UCDI »

you could do it either way... you could make it multi-instance and fire it off N times with N different parameters. You could loop. The loop sounds like the right way to go here, but both would work if you had a good reason to do one or the other design. Multi instance gets into problems with writing to the same target in parallel (locks, etc) for a task like this one. Where MI seems to shine is a totally reusable job that has varying targets and sources.
PaulVL
Premium Member
Premium Member
Posts: 1315
Joined: Fri Dec 17, 2010 4:36 pm

Post by PaulVL »

Since you are interacting with a database, you also need to understand the impact you will have there. Not knowing the complexity of the SQL you are performing we can't say if running them at the same time is prudent. Assuming you at some point you would be running 12 iterations of the job.

I would vote to run them sequentially just because of the database factor.
FranklinE
Premium Member
Premium Member
Posts: 739
Joined: Tue Nov 25, 2008 2:19 pm
Location: Malvern, PA

Post by FranklinE »

Even if running sequentially in a loop, multi-instance makes sense for the invocation id. I have a job that runs the same Px for ftp mutlitple times, reading a list for each subsequent file name. Reading the log in Director is very much easier because I label each run in the invocation id with the numeric instance of the loop.
Franklin Evans
"Shared pain is lessened, shared joy increased. Thus do we refute entropy." -- Spider Robinson

Using mainframe data FAQ: viewtopic.php?t=143596 Using CFF FAQ: viewtopic.php?t=157872
Post Reply