DSXchange: DataStage and IBM Websphere Data Integration Forum
View next topic
View previous topic
Add To Favorites
Author Message
sensiva



Group memberships:
Premium Members

Joined: 22 Aug 2017
Posts: 7

Points: 115

Post Posted: Fri Jun 22, 2018 8:49 am Reply with quote    Back to top    

DataStage® Release: 11x
Job Type: Parallel
OS: Unix
Hello

I am encountering a strange issue on my job & sequences since last week. The job or sequences that takes around 2 mins to compelte took 15 mins since last week. The disk space or the cpu consumption are normal. There was something strange when I looked at the logs. The job actually completes as expected in a couple of minutes with the
Code:
 INFO log saying "Parallel job reports successful compeltion" which is followed by CONTROL log saying "Finished Job xxx". The strange thing is the differnce between these 2 logs is nearly 5 mins.


We are planning to restart the server next week to see if it resolves, but thought of posting here to see if someone could share any pointers to find the actual root cause.

The delay is for each and every job irrespective of connectors used (file/db etc). If it process millions of records, the difference between info and control log is 5 mins, if the processing is for thousands of records the difference between the info and control is 1-2 mins.

Thanks

_________________
sen
PaulVL



Group memberships:
Premium Members

Joined: 17 Dec 2010
Posts: 1245

Points: 8171

Post Posted: Fri Jun 22, 2018 11:15 am Reply with quote    Back to top    

Log message tip of the day:

The timestamp of the entry in the log is not the timestamp of the event happening, it is the timestamp of the actual WRITE to the log of that event.

Think of the log as a print queue. You "may" have had some messages stuck in the queue to be written to the log, and they finally got written when the flush of the queue was done.



============


This may or may not be associated with what you are seeing, but I thought it best to mention that little tidbit.
Rate this response:  
Not yet rated
ray.wurlod

Premium Poster
Participant

Group memberships:
Premium Members, Inner Circle, Australia Usergroup, Server to Parallel Transition Group

Joined: 23 Oct 2002
Posts: 54259
Location: Sydney, Australia
Points: 294280

Post Posted: Sun Jun 24, 2018 11:46 pm Reply with quote    Back to top    

Do you have a long running (and silent) after-job subroutine?

_________________
RXP Services Ltd
Melbourne | Canberra | Sydney | Hong Kong | Hobart | Brisbane
currently hiring: Canberra, Sydney and Melbourne
Rate this response:  
Not yet rated
sensiva



Group memberships:
Premium Members

Joined: 22 Aug 2017
Posts: 7

Points: 115

Post Posted: Mon Jun 25, 2018 1:32 am Reply with quote    Back to top    

@ Paul --> I completely agree with you and inline with you. Even in that case, all the INFO logs are written to the log file at the same time or with the expected delay for each operation. I could see the log split up for 45 - 50 secs that actually does the job in a sequential and correct order. But the CONTROL log after 2 or 5 mins is completly out of sync and no trace in logs for that delay.

@ Ray --> No we don't have any after job subroutine. It happens for all job.

Another observation was that the Job Monitor which eventually indicates the CPU and number of records treated per sec is always in status "READY" even though the job has finished and doesn't indicate any statistics.

We did restart the server and we dont see that issue any more now. But still eager to know the root cause. Let me check from my side if i could find any other strange observations.

Thanks

_________________
sen
Rate this response:  
Not yet rated
Display posts from previous:       

Add To Favorites
View next topic
View previous topic
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum



Powered by phpBB © 2001, 2002 phpBB Group
Theme & Graphics by Daz :: Portal by Smartor
All times are GMT - 6 Hours