What if job aborts after 50% or more is done ?

A forum for discussing DataStage<sup>®</sup> basics. If you're not sure where your question goes, start here.

Moderators: chulett, rschirm, roy

Post Reply
urshit_1983
Participant
Posts: 73
Joined: Wed Jun 28, 2006 3:27 pm
Location: NJ

What if job aborts after 50% or more is done ?

Post by urshit_1983 »

I have a random question just came up in my mind.
For eg.

-Suppose I want to do UPSERT 2M rows in oracle table.
-Now due to some reasons the job is aborted after processing 1M rows.
-Either I can reset the job and run again or look for another option as 50% work is done.
-I will go for another option.

Help me in the logic:

-What if I make a temporary reference table if 50% or more rows are processed and give commit to both tables( original and temp) when job is aborted.
-Now I can use this temp table as look up when I start the job again so alot of time is saved.

Please suggest.
"Nobody is expert in Everything,
But Everybody is expert in Something."
DSguru2B
Charter Member
Charter Member
Posts: 6854
Joined: Wed Feb 09, 2005 3:44 pm
Location: Houston, TX

Post by DSguru2B »

This is related to restartability. Search this forum for different techinques.
Creativity is allowing yourself to make mistakes. Art is knowing which ones to keep.
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Your logic looks fine. But consider the question of array size also. You will probably need to unwind the final array because not all its rows were committed.

Possibly better logic would be to flag all records into the target with a unique run ID (or last updated timestamp) and do your lookup against the target constrained on that ID. This works fine for the inserts. Now, how do you expect to be able to reset the updated records back to their previous form? This involves taking a snapshot of the target table before your main processing even starts; the snapshot can be discarded once a successful run completes.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
urshit_1983
Participant
Posts: 73
Joined: Wed Jun 28, 2006 3:27 pm
Location: NJ

Post by urshit_1983 »

Thank you so much Ray. Cheers to ya.
"Nobody is expert in Everything,
But Everybody is expert in Something."
Post Reply