Hi,
I am processing around 24 Million records and the job takes around 7 hrs.
How should i tune the performance of the hash file.
How to enhance Hash file performance ?
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 5
- Joined: Sat Sep 28, 2013 2:24 pm
- Location: kolkata
How to enhance Hash file performance ?
Thanks- S Maji
(Be Simple but Think Different)
(Be Simple but Think Different)
I'm going to assume the issue is writing to it...
Use the Hashed File Calculator that (I believe) still ships with the product to determine how large to pre-create the hashed file, i.e. what Minimum Modulus to specify. The default is 1 which means it starts off tiny and spends a great deal of time expanding and reallocating space over and over.
Use the Hashed File Calculator that (I believe) still ships with the product to determine how large to pre-create the hashed file, i.e. what Minimum Modulus to specify. The default is 1 which means it starts off tiny and spends a great deal of time expanding and reallocating space over and over.
-craig
"You can never have too many knives" -- Logan Nine Fingers
"You can never have too many knives" -- Logan Nine Fingers
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Performance on read is not tunable. If the hashed file is properly sized, this should be one I/O operation per key-based access (lookup), or one I/O operation per group (stream).
Performance on write is tunable by preallocating the space for the hashed file, as Craig suggests. The Hashed File Calculator utility does still ship with the product - it's in the unsupported utilities folder on the installation media.
Write can be made still faster by using the write cache, but only if the total amount of data fits in the hashed file cache (the size of which is set in the Administrator client, and is limited to 999MB).
Performance on write is tunable by preallocating the space for the hashed file, as Craig suggests. The Hashed File Calculator utility does still ship with the product - it's in the unsupported utilities folder on the installation media.
Write can be made still faster by using the write cache, but only if the total amount of data fits in the hashed file cache (the size of which is set in the Administrator client, and is limited to 999MB).
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.