reading and writing to the same table

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
datastagedw
Participant
Posts: 53
Joined: Fri Mar 07, 2008 1:17 am

reading and writing to the same table

Post by datastagedw »

hello all,

I have a requirement where in I have to use a oracle table for change capture before link and after that doing some transformation and ultimately loading to the same oracle table. Does it cause any performance issue , using the same table to read and write. I read one article in dsxchange telling about environment variable APT_DISABLE_COMBINED_OPERATOR to true. Sorry I am not able to remember the variable name. Actually my requirement is to check the source with a change capture stage with a history table based on some key columns, and if the record is not there then insert it to the history table else reject it into a reject file. please help me out as I am dealing with millionsof records.

any help appreciated.

thanks
ETL DEVELOPER
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Change your requirement (at least that part of it that prescribes updating the same table from which you are reading). You risk causing a self-deadlock situation.

Use two jobs, staging the data in a Data Set.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
datastagedw
Participant
Posts: 53
Joined: Fri Mar 07, 2008 1:17 am

Post by datastagedw »

hello,

thank you. even i have the same thought, however i tried with few million of records(around 5 million), its not giving any problem even i also beleive staging it to a dataset would be better. well thanks and I would mark this topic as resolved. :D
ETL DEVELOPER
Zyaire
Participant
Posts: 1
Joined: Thu Jun 04, 2020 4:45 am

Suggestion

Post by Zyaire »

The main heading here that can be explained by all those who need to lean is something about Is education really the "backbone" of a "nation"? You can explain us something more about professional personal statement writing services as it was the only source for us. Just let us explain it now.
Post Reply