We migrated jobs from 9.1 from 11.5 Datastage. We are seeing performance drop in the jobs now, even though we are having fix pack 2 in 11.5 Also, we are using VMsqls00.so driver suggested by IBM here -
https://www.ibm.com/support/knowledgece ... ivers.html
Even a simple select in ODBC stage we can see a difference of atleast 7000 rows/sec (throughput from ODBC) between 9.1 and 11.5. How can be there this much difference when we are upgrading our DataStage version. Is this a normal behaviour or where can we go wrong?
Also, I want to know whether upgrading the driver will have any data impact?
Performance Issue in DataStage 11.5
Moderators: chulett, rschirm, roy
This is not what you would expect.
You have also posted about whether the driver might affect the data.
Both the functionality and the performance need regression testing after the upgrade to identify such issues and fix them.
With regard to the performance issue what else has changed?
Are the throughput figures you quote from identical jobs running in isolation on equivalent hardware? What else do these jobs do? Has anything changed on the database side?
Have you tried the old driver on 11.5 or the new driver on 9.1 to try and isolate the issue?
You might try contacting your support provider to check if there are any known performance issues in this area for which a fix is available, but if not they will want a reproducible test case.
You should build a test case job (that does nothing but read data from the database) to run in the old and new environments to gather suitable performance metrics, log files etc.
You have also posted about whether the driver might affect the data.
Both the functionality and the performance need regression testing after the upgrade to identify such issues and fix them.
With regard to the performance issue what else has changed?
Are the throughput figures you quote from identical jobs running in isolation on equivalent hardware? What else do these jobs do? Has anything changed on the database side?
Have you tried the old driver on 11.5 or the new driver on 9.1 to try and isolate the issue?
You might try contacting your support provider to check if there are any known performance issues in this area for which a fix is available, but if not they will want a reproducible test case.
You should build a test case job (that does nothing but read data from the database) to run in the old and new environments to gather suitable performance metrics, log files etc.
Thanks for the reponse. Sorry for asking two question at once which created confusion. Both jobs are have exaclty same configuration, same driver are used, same Database is there. Actually we are copying from 9.1 and pasting 11.5. So, maybe some settings that is not needed in 11.5 is being used. Also, I have made a test job which just read the records from table and load directly into in another table. In this case also we are seeing slow performance. What can be causes of this I am not able to debug?
shubham
You should be exporting from 9.1 and importing into the 11.5 version. This so it has a chance to properly upgrade everything which the import handles automagically. Now, perhaps someone with more 'under the covers' knowledge will let us know if that's really an issue or not but it seems to me that is a much more proper way to do things.
And for the record, there was no "confusion", there was just no need for two topics on the same issue.
And for the record, there was no "confusion", there was just no need for two topics on the same issue.
-craig
"You can never have too many knives" -- Logan Nine Fingers
"You can never have too many knives" -- Logan Nine Fingers