Performance issues : QS Plugin vs. raw QS

PilotBaha · Post by **PilotBaha** » Wed Feb 09, 2005 5:34 pm

I have been having a major drop in performance when I execute the same QS job from DS. The job is a match with 270K records.
The run from QS completes in 2 minutes. On the other hand the same job takes 20 minutes when I run it from DS.

I checked the tracing levels on qsrt manager which I was suspecting as a reason and it is 0. I also know that I should not expect the exact same performace from DS because of plugin being in the way, but 2 vs. 20 is a major difference.

My hands are tied in terms of accessing the server (and the admin is on training to boot) to run scripts or run the match from a command stage in DS instead of QS. ..

Any ideas where else I should look for this performance issue?

vmcburney · Post by **vmcburney** » Thu Feb 10, 2005 2:46 am

When you say tracing level are you talking about the trace setting within the QualityStage plugin? Make sure that is set to 0. Is there anything else in your DataStage job that could be a bottleneck such as a lookup or a database source or target?

I would expect the performance to be the same between both methods.

PilotBaha · Post by **PilotBaha** » Thu Feb 10, 2005 11:32 am

The plugin value is set to 0 on the DS side, but since I don't have the sys admin priviledge or even a straight access to the server I cannot check what's going on when the qsrtmngr is invoked. (Gotta love the corporate politics)

The sys admin says that the trace is set to 0 on the qsrtmngr as well..

It's just weird ..

vmcburney · Post by **vmcburney** » Thu Feb 10, 2005 6:54 pm

There are two things I would do at this point. First I would look in the QS trace log directories while the job is running and verify that the trace files are not getting bigger.

Second I would go to the QS project - job log directory and compare the log file and log files dates for a QS run against a DS run. This log file will show the start and end times for each part of the QS job. This will show you exactly where the DS run is slower then the QS run. It may help you narrow down where your bottleneck is.

You could also run both a QS and a DS run with tracing turned on and compare the dates in the trace file.

PilotBaha · Post by **PilotBaha** » Fri Feb 11, 2005 3:01 pm

vmcburney wrote:There are two things I would do at this point. First I would look in the QS trace log directories while the job is running and verify that the trace files are not getting bigger.

Sort of done that.. The qsrtmanager logs in production are not getting bigger. Since I don't have access to the QualityStageServer70 directory I can only verify this by asking the admin.

vmcburney wrote:Second I would go to the QS project - job log directory and compare the log file and log files dates for a QS run against a DS run. This log file will show the start and end times for each part of the QS job. This will show you exactly where the DS run is slower then the QS run. It may help you narrow down where your bottleneck is.

Ds runs are not producing the regular QS logs on the server. My comparison is based on the qs logs on the dev server and the DS Director information on the prod server.

vmcburney wrote:You could also run both a QS and a DS run with tracing turned on and compare the dates in the trace file.

That is the plan for monday.. Gotta love the corporate politics that doesn't allow the QS guy the access to the server..