Page 1 of 1

Datastage jobs running extremely slow

Posted: Sun Sep 09, 2018 1:00 pm
by rumu
Hi All,

I have been observing for last 2 days that, our daily batch jobs have started taking longer than usual time. Normally as job that runs for 30 mins, now running for more than 2 hours. Our DB is Netezza, jobs that extracts from DB, are working normal but the other jobs where other datastage stages have been used, are running extremely slow. operation console shows that number of rows/sec has dropped to 1/10 th of the usual processing time.
Could you please shed any light on this?I checked tmp folders and space was no issue. Is server restart required?

Posted: Sun Sep 09, 2018 3:39 pm
by ray.wurlod
I'd first ask what has changed. Note that this could be something in the environment, not necessarily within DataStage.

Performance changes such as this do not happen for no reason.

Posted: Sun Sep 09, 2018 6:19 pm
by rumu
Hi Ray,

Nothing has changed wrt Code. Till Friday, the execution was fine, slowness has been observed first on Saturday then on Sunday.
Only diff I saw that number of rows/sec decreased significantly.
What could be possible environmental issue that might cause this?

Any weekend maintenance application or network could be the issue?

Posted: Mon Sep 10, 2018 6:14 am
by chulett
Of course and there's no way for us to know, that's all on you and your organization to determine. We have routine "maintenance weekends" from our hosting company and of course there are backups and other resource intensive things going on as well. Be curious if things go back to "normal" today.

Posted: Mon Sep 10, 2018 12:44 pm
by rumu
No luck today. The same issue persists. Suddenly some of the jobs are running extremely slow. Looks like environmental issue. Can we check if any phantom processes are stuck and that consumes resources? How can we check that?

Posted: Tue Sep 11, 2018 7:03 am
by chulett
In your shoes I would not concentrate on just DataStage but look at the environment as a whole. Work with your infrastructure folks and system admins to determine what changed recently, see what if any work coincided with the start of your issue.

If you're worried about "stuck" processes and don't know if they exist or how to track them down, shutdown and restart DataStage. Heck, worst case shut it down and then have the sever rebooted. :wink: