DSXchange: DataStage and IBM Websphere Data Integration Forum
View next topic
View previous topic
Add To Favorites
This topic has been marked "Resolved."
Author Message
PeteM



Group memberships:
Premium Members

Joined: 15 Dec 2011
Posts: 12
Location: uk
Points: 181

Post Posted: Wed Aug 09, 2017 3:16 am Reply with quote    Back to top    

DataStage® Release: 11x
Job Type: Parallel
OS: Unix
My understanding is that if a job is run twice from the same project, at the same time, data stage ensures that the datasets created by both instances of the job are unique to the instance and no overlap can occur?

_________________
Thanks
PeteM
UCDI



Group memberships:
Premium Members

Joined: 21 Mar 2016
Posts: 250

Points: 2521

Post Posted: Wed Aug 09, 2017 10:36 am Reply with quote    Back to top    

you should apply the invocation ID to the dataset name. If the names and folders are exact matches, it will have problems, and having the invocation id also lets you debug easier.
Rate this response:  
Not yet rated
chulett

Premium Poster


since January 2006

Group memberships:
Premium Members, Inner Circle, Server to Parallel Transition Group

Joined: 12 Nov 2002
Posts: 42105
Location: Denver, CO
Points: 216139

Post Posted: Wed Aug 09, 2017 11:12 am Reply with quote    Back to top    

... in other words, no it does not. Wink

_________________
-craig

Can't keep my eyes from the circling skies
Tongue tied and twisted just an earth bound misfit, I
Rate this response:  
Not yet rated
PeteM



Group memberships:
Premium Members

Joined: 15 Dec 2011
Posts: 12
Location: uk
Points: 181

Post Posted: Thu Aug 10, 2017 6:43 am Reply with quote    Back to top    

we have a requirement whereby a given job has to be run multiple times supporting separate test environments (against different databases), but each instance of the job would be processing different test data, based on the database it was pointing at.

All instances of the job will run on the same data stage engine.

Therefore, there is a chance that two instances of the job could be running at the same time.

Based on your responses am I correct to assume that this requirement cannot be met?

_________________
Thanks
PeteM
Rate this response:  
Not yet rated
eostic

Premium Poster



Group memberships:
Premium Members

Joined: 17 Oct 2005
Posts: 3700

Points: 29539

Post Posted: Thu Aug 10, 2017 6:47 am Reply with quote    Back to top    

This is exactly what Multi-Job Instancing was designed for. One Job...run concurrently multiple times, but with different Job Parameter values...each for their independent needs/environment, etc. .. ...

_________________
Ernie Ostic

blogit!
Open IGC is Here!
Rate this response:  
Not yet rated
PeteM



Group memberships:
Premium Members

Joined: 15 Dec 2011
Posts: 12
Location: uk
Points: 181

Post Posted: Thu Aug 10, 2017 9:33 am Reply with quote    Back to top    

In production only one instance of the job would ever run, which is why we have not made it a multiple instance job. This is a requirement to support multiple test streams.

Therefore, is the consensus of opinion that multiple instances of a job cannot be run on one data stage engine, without making the job multiple instance?

_________________
Thanks
PeteM
Rate this response:  
Not yet rated
PaulVL



Group memberships:
Premium Members

Joined: 17 Dec 2010
Posts: 1129

Points: 7485

Post Posted: Thu Aug 10, 2017 10:43 am Reply with quote    Back to top    

You can have a multi instance job run. Each job must write to a UNIQUE dataset name.

Apply a parameter into the path or filename to make it UNIQUE.

Same job can be loaded into DEV, TEST or PROD later on since it would still work with a unique supplied parameter for that job instance.
Rate this response:  
Not yet rated
chulett

Premium Poster


since January 2006

Group memberships:
Premium Members, Inner Circle, Server to Parallel Transition Group

Joined: 12 Nov 2002
Posts: 42105
Location: Denver, CO
Points: 216139

Post Posted: Thu Aug 10, 2017 11:58 am Reply with quote    Back to top    

PeteM wrote:
Therefore, is the consensus of opinion that multiple instances of a job cannot be run on one data stage engine, without making the job multiple instance?

Yes... that's the whole point of the Multi-Instance option. Now that doesn't mean you need to run it that way in Production, you can still run it without an InvocationID and it would run just like it wasn't multi-instance enabled.

_________________
-craig

Can't keep my eyes from the circling skies
Tongue tied and twisted just an earth bound misfit, I
Rate this response:  
Not yet rated
R.K.Glover
Participant



Joined: 11 Mar 2013
Posts: 7
Location: RTP, North Carolina
Points: 45

Post Posted: Thu Aug 31, 2017 9:31 am Reply with quote    Back to top    

PeteM wrote:
In production only one instance of the job would ever run, which is why we have not made it a multiple instance job. This is a requirement to support multiple test streams.

Therefore, is the consensus of opinion that multiple instances of a job cannot be run on one data stage engine, without making the job multiple instance?


I suppose it really depends on whether you need to have multiple copies of the job running at the same time .

If you do, then you MUST make the job Allow Multiple Instances.

If you don't, then you don't have to - but for debugging purposes, gosh, wouldn't it be nice if all of the logs for each of your test streams had their own log, instead of one big log where you had to look at the passed parameters of each run to figure out which one it was?

As far as input/output files, DataSets, etc.... like others have said, take one of your unique parameters, and tack it into your path or filename for input/output. As long as there is a value for that parameter that matches production, you should be fine.
Rate this response:  
Not yet rated
Display posts from previous:       

Add To Favorites
View next topic
View previous topic
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum



Powered by phpBB © 2001, 2002 phpBB Group
Theme & Graphics by Daz :: Portal by Smartor
All times are GMT - 6 Hours