Retry on JDBC connector

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
clarcombe
Premium Member
Premium Member
Posts: 515
Joined: Wed Jun 08, 2005 9:54 am
Location: Europe

Retry on JDBC connector

Post by clarcombe »

Occasionally we get timeout errors when trying to read DB2 on Z/OS using a JDBC stage.

However on the next run of the batch it then works.

For information, this is the error that occurs

Code: Select all

Fatal Error: waitForWriteSignal(): Premature EOF on node vdatastagep.fmsb.be Bad file descriptor
This is one of a number of tables read by a generic routine. Its never the same table that fails twice and the batch can go days without failure


Question

Is there an option in the JDBC stage to allow us to retry x times after a time period has elapsed e.g. 1 minute ?

Are there any other ways to get around this ?
Colin Larcombe
-------------------

Certified IBM Infosphere Datastage Developer
asorrell
Posts: 1707
Joined: Fri Apr 04, 2003 2:00 pm
Location: Colleyville, Texas

Post by asorrell »

We had some jobs that failed intermittently due to bad network latency. Because the jobs were safe to re-run after any failure, we wrapped them in job sequences that re-ran the jobs automatically twice, but aborted on the third failure.
Andy Sorrell
Certified DataStage Consultant
IBM Analytics Champion 2009 - 2020
clarcombe
Premium Member
Premium Member
Posts: 515
Joined: Wed Jun 08, 2005 9:54 am
Location: Europe

Post by clarcombe »

Andy,

Thanks for this.

Can you elaborate on this technique please ?

Our read/write jobs are already being run in Job Sequences and are called from a main Job Sequence.

Are you suggesting that we add a 3rd layer of Job Sequence calls to this ?
Colin Larcombe
-------------------

Certified IBM Infosphere Datastage Developer
qt_ky
Premium Member
Premium Member
Posts: 2895
Joined: Wed Aug 03, 2011 6:16 am
Location: USA

Post by qt_ky »

That is what we had to do also, because our network is so alarmingly unreliable.

Our highest level of sequence jobs--the ones we place on schedule--perform auto-retry logic by using a loop. The loop uses parameters for number of retries and sleep interval. Doing this has saved us countless manual interventions in the middle of the night!

We may schedule one sequence to retry 6 times and wait 10 minutes between each retry, and schedule another to retry 8 times and wait 1 hour between each retry, etc.

Some pointers: take advantage of the job properties to "add checkpoints so sequence is restartable on failure" and when you call jobs make sure to choose execution action of "Reset if required, then run" which by the way is not the default setting.
Choose a job you love, and you will never have to work a day in your life. - Confucius
Post Reply