DSXchange: DataStage and IBM Websphere Data Integration Forum
View next topic
View previous topic
Add To Favorites
Author Message
clarcombe



Group memberships:
Premium Members

Joined: 08 Jun 2005
Posts: 507
Location: Europe
Points: 5290

Post Posted: Wed Aug 23, 2017 2:46 am Reply with quote    Back to top    

DataStage® Release: 11x
Job Type: Parallel
OS: Unix
Additional info: Timeout issue ?
Occasionally we get timeout errors when trying to read DB2 on Z/OS using a JDBC stage.

However on the next run of the batch it then works.

For information, this is the error that occurs

Code:
Fatal Error: waitForWriteSignal(): Premature EOF on node vdatastagep.fmsb.be Bad file descriptor


This is one of a number of tables read by a generic routine. Its never the same table that fails twice and the batch can go days without failure


Question

Is there an option in the JDBC stage to allow us to retry x times after a time period has elapsed e.g. 1 minute ?

Are there any other ways to get around this ?

_________________
Colin Larcombe
-------------------

Certified IBM Infosphere Datastage Developer
asorrell
Site Admin

Group memberships:
Premium Members, DSXchange Team, Inner Circle, Server to Parallel Transition Group

Joined: 04 Apr 2003
Posts: 1637
Location: Colleyville, Texas
Points: 22256

Post Posted: Wed Aug 23, 2017 10:49 am Reply with quote    Back to top    

We had some jobs that failed intermittently due to bad network latency. Because the jobs were safe to re-run after any failure, we wrapped them in job sequences that re-ran the jobs automatically twice, but aborted on the third failure.

_________________
Andy Sorrell
Certified DataStage Consultant
IBM Analytics Champion 2009 - 2017
Rate this response:  
Not yet rated
clarcombe



Group memberships:
Premium Members

Joined: 08 Jun 2005
Posts: 507
Location: Europe
Points: 5290

Post Posted: Thu Nov 16, 2017 3:12 am Reply with quote    Back to top    

Andy,

Thanks for this.

Can you elaborate on this technique please ?

Our read/write jobs are already being run in Job Sequences and are called from a main Job Sequence.

Are you suggesting that we add a 3rd layer of Job Sequence calls to this ?

_________________
Colin Larcombe
-------------------

Certified IBM Infosphere Datastage Developer
Rate this response:  
Not yet rated
qt_ky



Group memberships:
Premium Members

Joined: 03 Aug 2011
Posts: 2666
Location: USA
Points: 19826

Post Posted: Thu Nov 16, 2017 8:43 am Reply with quote    Back to top    

That is what we had to do also, because our network is so alarmingly unreliable.

Our highest level of sequence jobs--the ones we place on schedule--perform auto-retry logic by using a loop. The loop uses parameters for number of retries and sleep interval. Doing this has saved us countless manual interventions in the middle of the night!

We may schedule one sequence to retry 6 times and wait 10 minutes between each retry, and schedule another to retry 8 times and wait 1 hour between each retry, etc.

Some pointers: take advantage of the job properties to "add checkpoints so sequence is restartable on failure" and when you call jobs make sure to choose execution action of "Reset if required, then run" which by the way is not the default setting.

_________________
Choose a job you love, and you will never have to work a day in your life. - Confucius
Rate this response:  
Not yet rated
Display posts from previous:       

Add To Favorites
View next topic
View previous topic
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum



Powered by phpBB © 2001, 2002 phpBB Group
Theme & Graphics by Daz :: Portal by Smartor
All times are GMT - 6 Hours