Performance Issue with Big HASH File

[sysmq]

Hi
I have a problem
we have a JOB that loads large amount of hash files into memory
in one server (w2k3 sp1 dual core 3.5 gb RAM) it works great
this is the DEV computer
on the production computer (same os but stronger CPU (4 quad) with 3.5 gb RAM)
we encounter a performance issue all of tha HASH files are taking over the entier RAM and pageFile (6.6 gb ) and than crashes with the error of :unable to alocate memory.
i have compered the uvconfig file and the project parameters.

just abit more infromation :
the job extract data from a DB perform a JOIN on the tables, then manipulate the data to create a unified row with all data from the join and updateds the destination DB, more over it is all done in the HASH files and loaded to the memory.
everage HASH file is about 300mb at least, the project has a write cache and read cache of 512.
i tried to enlarge it to 999mb but it just failed faster.

what can be the reason to this huge gap between the two computers performances ?

please help

sysmq

[ray.wurlod]

What ELSE is happening on the production computer (that is, consuming resources) that is not happening on the development computer?

Don't say "nothing"; that is clearly incorrect. Monitor the systems and find out.

Are you processing the same data volumes in each environment?

It's HASHED file, incidentally, not HASH file.

[sysmq]

Hi
I did monitor the system and the only major consumer was the uvsh.exe of that same project and the reason why i know this for sure is that we've aborted the project the system performance monitor was quiet down immidiatly.

on the production server it was the only project running since it is a new production server and we are in the middle of migrating the projects/

any thing else i should look into ?


10x alot

sysmq

[ray.wurlod]

Is the database on the same machine. A quad processor might be able to consume more resources IN THE DATABASE, leaving less for the hashed file that the dual processors on the development machine could provide.

[sysmq]

Is the database on the same machine. A quad processor might be able to consume more resources IN THE DATABASE, leaving less for the hashed file that the dual processors on the development machine could provide.

Hi
The source DB and Destination DB of the project is not on the same machine.

respectfully,
sysmq

[ArndW]

...we encounter a performance issue all of tha HASH files are taking over the entier RAM and pageFile (6.6 gb ) and than crashes with the error of :unable to alocate memory...

Have you measured memory usage by process, or how have you determined this?

Do you have just one DataStage job or many? What changes to the uvconfig file have you done?

[sysmq]

...we encounter a performance issue all of tha HASH files are taking over the entier RAM and pageFile (6.6 gb ) and than crashes with the error of :unable to alocate memory...

Have you measured memory usage by process, or how have you determined this?

Do you have just one DataStage job or many? What changes to the uvconfig file have you done?

1. each process consumes 512 mb as defined in the administrator
i opend the performance monitor and sorted it by process; i used pstools to determine what job execute this process.

The changes made to the uvconfig are the same as already running on the test server and there the preformance of the same job is great.

All changes where made according to our local provider recommandations

The changes are :
MFILES from 150 to 550
T30Files from 200 to 1500
RLTABSZ from 75 to 300
MAXRLOCK from 74 to 299


Thats all
With respect

Sysmq

[ArndW]

...
RLTABSZ from 75 to 300
MAXRLOCK from 74 to 299

Where did you get the recommendations to change these values? Unless you have specific problems that are addressed by this change it is unwise to change these, as they can have an impact on performance.

[sysmq]

...
RLTABSZ from 75 to 300
MAXRLOCK from 74 to 299

Where did you get the recommendations to change these values? Unless you have specific problems that are addressed by this change it is unwise to change these, as they can have an impact on performance.

our local vendor recomanded this due to problems we had on our production server due to the large amount of projects running on the server
the parametrs changed solved the performance issues we had.

We have more than 20 projects running on the current Production server.
this is that main reason for the migration project now running.

[sysmq]

Hi All
After talking to IBM i got a recomandation to edit some values in the uvconfig file
here are the new values:
T30FILE to 500 or 600
GLTABSZ 120 (change to 160)
RLTABSZ 120 (change to 160)
MAXRLOCK 119 (change to 159)

I've tried to figgure out what will be the effect and how can i fine tune it acording to the job behaviour.
i did not get a reply regarding where can i read about each parameter and it's affect on the system
where can i read , beside the uvconfig file itself, about each parameter and it's affect on the system ?

Respectfully,

sysmq

[ray.wurlod]

Again, the UniVerse manuals, but even they are fairly sparse on tuning.

The best knowledge can be gained by attending the IBM class UV910 (UniVerse Theory and Practice), formerly known as UniVerse Internals.