Loading same table in db by 2 different parallel jobs

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
Suruchi
Participant
Posts: 16
Joined: Tue Dec 13, 2011 3:21 am

Loading same table in db by 2 different parallel jobs

Post by Suruchi »

Hi,
Till now I was of the opinion that two different parallel jobs loading the same table in database cannot run simultaneoulsy.But today, I saw two parallel jobs(DS v8.5.1) loading the same table simultaneoulsy.The target stage was Oracle connector.Am I missing something? Can anyone explain this better.
SURA
Premium Member
Premium Member
Posts: 1229
Joined: Sat Jul 14, 2007 5:16 am
Location: Sydney

Re: Loading same table in db by 2 different parallel jobs

Post by SURA »

Suruchi wrote:loading the same table simultaneoulsy.
Two jobs are running / loading? There is a difference between the statements :)

Whatever may be the source, Oracle need to be configured in such a way to handle the situation like locking systems, timeout, in some cases partitions.

Fine, let us know what is the load method in those two Datastage jobs?
Thanks
Ram
----------------------------------
Revealing your ignorance is fine, because you get a chance to learn.
Suruchi
Participant
Posts: 16
Joined: Tue Dec 13, 2011 3:21 am

Post by Suruchi »

Load method is append in both the jobs.
eph
Premium Member
Premium Member
Posts: 110
Joined: Mon Oct 18, 2010 10:25 am

Post by eph »

Hi,

You have in the bulk load options a "Allow concurrent load sessions" that allow you to run the stage in parallel. I suppose it also allows the behavior that you describe (no locks?)?

Eric
srinivas.nettalam
Participant
Posts: 134
Joined: Tue Jun 15, 2010 2:10 am
Location: Bangalore

Post by srinivas.nettalam »

Parallel inserts are allowed on the tables.Suppose that the table you mentioned is loaded by only one parallel job in your project and the job runs on 8 nodes.It is implicit that it is allowing 8 parallel insert statements.
In your case the table is loaded by two jobs and assuming your job runs in 2 nodes/4 nodes in these cases you will have 4 inserts or 8 inserts respectively which is a normal scenario.
N.Srinivas
India.
prasson_ibm
Premium Member
Premium Member
Posts: 536
Joined: Thu Oct 11, 2007 1:48 am
Location: Bangalore

Post by prasson_ibm »

What load tecnique you are using in connector stage...bulk load or normal load?
Post Reply