DSXchange: DataStage and IBM Websphere Data Integration Forum
View next topic
View previous topic
Add To Favorites
Author Message
rumu
Participant



Joined: 06 Jun 2005
Posts: 227

Points: 2180

Post Posted: Sun Sep 09, 2018 1:00 pm Reply with quote    Back to top    

DataStage® Release: 11x
Job Type: Parallel
OS: Unix
Hi All,

I have been observing for last 2 days that, our daily batch jobs have started taking longer than usual time. Normally as job that runs for 30 mins, now running for more than 2 hours. Our DB is Netezza, jobs that extracts from DB, are working normal but the other jobs where other datastage stages have been used, are running extremely slow. operation console shows that number of rows/sec has dropped to 1/10 th of the usual processing time.
Could you please shed any light on this?I checked tmp folders and space was no issue. Is server restart required?

_________________
Rumu
IT Consultant
ray.wurlod

Premium Poster
Participant

Group memberships:
Premium Members, Inner Circle, Australia Usergroup, Server to Parallel Transition Group

Joined: 23 Oct 2002
Posts: 54407
Location: Sydney, Australia
Points: 295092

Post Posted: Sun Sep 09, 2018 3:39 pm Reply with quote    Back to top    

I'd first ask what has changed. Note that this could be something in the environment, not necessarily within DataStage.

Performance changes such as this do not happen for no reason.

_________________
RXP Services Ltd
Melbourne | Canberra | Sydney | Hong Kong | Hobart | Brisbane
currently hiring: Canberra, Sydney and Melbourne (especially seeking good business analysts)
Rate this response:  
Not yet rated
rumu
Participant



Joined: 06 Jun 2005
Posts: 227

Points: 2180

Post Posted: Sun Sep 09, 2018 6:19 pm Reply with quote    Back to top    

Hi Ray,

Nothing has changed wrt Code. Till Friday, the execution was fine, slowness has been observed first on Saturday then on Sunday.
Only diff I saw that number of rows/sec decreased significantly.
What could be possible environmental issue that might cause this?

Any weekend maintenance application or network could be the issue?

_________________
Rumu
IT Consultant
Rate this response:  
Not yet rated
chulett

Premium Poster


since January 2006

Group memberships:
Premium Members, Inner Circle, Server to Parallel Transition Group

Joined: 12 Nov 2002
Posts: 42790
Location: Denver, CO
Points: 220550

Post Posted: Mon Sep 10, 2018 6:14 am Reply with quote    Back to top    

Of course and there's no way for us to know, that's all on you and your organization to determine. We have routine "maintenance weekends" from our hosting company and of course there are backups and other resource intensive things going on as well. Be curious if things go back to "normal" today.

_________________
-craig

"I don't mind you comin' here and wastin' all my time time"
Rate this response:  
Not yet rated
rumu
Participant



Joined: 06 Jun 2005
Posts: 227

Points: 2180

Post Posted: Mon Sep 10, 2018 12:44 pm Reply with quote    Back to top    

No luck today. The same issue persists. Suddenly some of the jobs are running extremely slow. Looks like environmental issue. Can we check if any phantom processes are stuck and that consumes resources? How can we check that?

_________________
Rumu
IT Consultant
Rate this response:  
Not yet rated
chulett

Premium Poster


since January 2006

Group memberships:
Premium Members, Inner Circle, Server to Parallel Transition Group

Joined: 12 Nov 2002
Posts: 42790
Location: Denver, CO
Points: 220550

Post Posted: Tue Sep 11, 2018 7:03 am Reply with quote    Back to top    

In your shoes I would not concentrate on just DataStage but look at the environment as a whole. Work with your infrastructure folks and system admins to determine what changed recently, see what if any work coincided with the start of your issue.

If you're worried about "stuck" processes and don't know if they exist or how to track them down, shutdown and restart DataStage. Heck, worst case shut it down and then have the sever rebooted. Wink

_________________
-craig

"I don't mind you comin' here and wastin' all my time time"
Rate this response:  
Not yet rated
Display posts from previous:       

Add To Favorites
View next topic
View previous topic
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum



Powered by phpBB © 2001, 2002 phpBB Group
Theme & Graphics by Daz :: Portal by Smartor
All times are GMT - 6 Hours