HashFile size

Post questions here relative to DataStage Server Edition for such areas as Server job design, DS Basic, Routines, Job Sequences, etc.

Moderators: chulett, rschirm, roy

Post Reply
Leonie
Participant
Posts: 16
Joined: Thu Jul 31, 2003 7:16 am
Location: South Africa
Contact:

HashFile size

Post by Leonie »

So I have set the ulimit on OS level and in the dsenv file

ulimit -d unlimited
ulimit -m unlimited
# ulimit -s unlimited
ulimit -s 4194304
ulimit -f unlimited
ulimit -n 4194304

In the uvconfig file I have set the 64BIT_FILES 1

Database to Hash - about 220 Mil records to be written to the hashfile

The Jobs runs up to a point then it just hangs - no errors - the filesize is round 2gb - so it has reach the limit for 32bit - but it is set to 64 bit -still it doesn't grow.

We have recently migrated from 8.x to 11 and it works on 8.

Is there anything else that I can do , did I miss a step?

(Info above it what I could find in this Forum and IBM's website)

Thanks in advance
JRodriguez
Premium Member
Premium Member
Posts: 425
Joined: Sat Nov 19, 2005 9:26 am
Location: New York City
Contact:

Post by JRodriguez »

I would ensure to do a regen after adjusting the uvconfig file. Recycling the engine is mandatory too. Below the info from IBM support


http://www-01.ibm.com/support/docview.w ... wg21637567
Julio Rodriguez
ETL Developer by choice

"Sure we have lots of reasons for being rude - But no excuses
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Before Ray comes along and says something, they are hashed files not hash files. :wink:

I would also note that "best practice" suggests you should only set the one specific hashed file to 64bit, not every single hashed file on the server, which is what happens when you change that uvconfig option. Best to stick with option #2 in the posted link IMHO.
-craig

"You can never have too many knives" -- Logan Nine Fingers
Leonie
Participant
Posts: 16
Joined: Thu Jul 31, 2003 7:16 am
Location: South Africa
Contact:

Post by Leonie »

Did the uv regen
Added the file via the command line.
It gets up to 2.1GB and then just hang there - job doesn't fail - doesn't complete.
Could it be a limit in version (11)?
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

There's no new limit in version 11.

As Craig noted it is unwise to set 64BITFILES in uvconfig, as this will force every hashed file to use 64-bit addressing.

Better would be to delete your problematic hashed file, and recreated it with the -64bit option.

On your install media you will find a Hashed File Calculator in the unsuppored utilities; this will allow you to specify a size and generate a command for creating the hashed file.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Leonie
Participant
Posts: 16
Joined: Thu Jul 31, 2003 7:16 am
Location: South Africa
Contact:

Post by Leonie »

The 64BITFILES option in the uvconfig file is working

We missed one of the regen steps - when we did it - it worked fine.

Thanks for all the help
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

It's still by far the poorer choice.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Post Reply