Performance Issue in DataStage 11.5

A forum for discussing DataStage<sup>®</sup> basics. If you're not sure where your question goes, start here.

Moderators: chulett, rschirm, roy

Post Reply
shv22
Participant
Posts: 3
Joined: Tue Aug 08, 2017 11:17 pm

Performance Issue in DataStage 11.5

Post by shv22 »

We migrated jobs from 9.1 from 11.5 Datastage. We are seeing performance drop in the jobs now, even though we are having fix pack 2 in 11.5 Also, we are using VMsqls00.so driver suggested by IBM here -

https://www.ibm.com/support/knowledgece ... ivers.html

Even a simple select in ODBC stage we can see a difference of atleast 7000 rows/sec (throughput from ODBC) between 9.1 and 11.5. How can be there this much difference when we are upgrading our DataStage version. Is this a normal behaviour or where can we go wrong?

Also, I want to know whether upgrading the driver will have any data impact?
shubham
thompsonp
Premium Member
Premium Member
Posts: 205
Joined: Tue Mar 01, 2005 8:41 am

Post by thompsonp »

This is not what you would expect.

You have also posted about whether the driver might affect the data.

Both the functionality and the performance need regression testing after the upgrade to identify such issues and fix them.

With regard to the performance issue what else has changed?
Are the throughput figures you quote from identical jobs running in isolation on equivalent hardware? What else do these jobs do? Has anything changed on the database side?

Have you tried the old driver on 11.5 or the new driver on 9.1 to try and isolate the issue?

You might try contacting your support provider to check if there are any known performance issues in this area for which a fix is available, but if not they will want a reproducible test case.

You should build a test case job (that does nothing but read data from the database) to run in the old and new environments to gather suitable performance metrics, log files etc.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

FYI - There's no need for two discussions on this topic, so I basically "merged" them.
-craig

"You can never have too many knives" -- Logan Nine Fingers
PaulVL
Premium Member
Premium Member
Posts: 1315
Joined: Fri Dec 17, 2010 4:36 pm

Post by PaulVL »

Are you suppressing any messages with handlers?

If so, remove the handlers to see what is being reported.

Hopping from 9.1 to 11.x often means Metadata issues in jobs if they were poor quality to begin with.
shv22
Participant
Posts: 3
Joined: Tue Aug 08, 2017 11:17 pm

Post by shv22 »

Thanks for the reponse. Sorry for asking two question at once which created confusion. Both jobs are have exaclty same configuration, same driver are used, same Database is there. Actually we are copying from 9.1 and pasting 11.5. So, maybe some settings that is not needed in 11.5 is being used. Also, I have made a test job which just read the records from table and load directly into in another table. In this case also we are seeing slow performance. What can be causes of this I am not able to debug?
shubham
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

You should be exporting from 9.1 and importing into the 11.5 version. This so it has a chance to properly upgrade everything which the import handles automagically. Now, perhaps someone with more 'under the covers' knowledge will let us know if that's really an issue or not but it seems to me that is a much more proper way to do things.

And for the record, there was no "confusion", there was just no need for two topics on the same issue.
-craig

"You can never have too many knives" -- Logan Nine Fingers
shv22
Participant
Posts: 3
Joined: Tue Aug 08, 2017 11:17 pm

Post by shv22 »

Yes we are exporting jobs from 9.1 and importing the same in 11.5. Yes, if there is something I missing out, I will be happy to know.
shubham
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Here's the only reason I made the comment...
shv22 wrote:Actually we are copying from 9.1 and pasting 11.5. So, maybe some settings that is not needed in 11.5 is being used.
-craig

"You can never have too many knives" -- Logan Nine Fingers
PaulVL
Premium Member
Premium Member
Posts: 1315
Joined: Fri Dec 17, 2010 4:36 pm

Post by PaulVL »

I read it the same way Craig.
Post Reply