DSXchange: DataStage and IBM Websphere Data Integration Forum
View next topic
View previous topic
Add To Favorites
This topic has been marked "Resolved."
Author Message
Leonie
Participant



Joined: 31 Jul 2003
Posts: 16
Location: South Africa
Points: 150

Post Posted: Fri Jun 30, 2017 6:12 am Reply with quote    Back to top    

DataStage® Release: 11x
Job Type: Server
OS: Unix
Additional info: Hash doesn't grow beyond 2gb
So I have set the ulimit on OS level and in the dsenv file

ulimit -d unlimited
ulimit -m unlimited
# ulimit -s unlimited
ulimit -s 4194304
ulimit -f unlimited
ulimit -n 4194304

In the uvconfig file I have set the 64BIT_FILES 1

Database to Hash - about 220 Mil records to be written to the hashfile

The Jobs runs up to a point then it just hangs - no errors - the filesize is round 2gb - so it has reach the limit for 32bit - but it is set to 64 bit -still it doesn't grow.

We have recently migrated from 8.x to 11 and it works on 8.

Is there anything else that I can do , did I miss a step?

(Info above it what I could find in this Forum and IBM's website)

Thanks in advance
JRodriguez



Group memberships:
Premium Members

Joined: 19 Nov 2005
Posts: 413
Location: New York City
Points: 4531

Post Posted: Sat Jul 01, 2017 8:08 pm Reply with quote    Back to top    

I would ensure to do a regen after adjusting the uvconfig file. Recycling the engine is mandatory too. Below the info from IBM support


http://www-01.ibm.com/support/docview.wss?uid=swg21637567

_________________
Julio Rodriguez
ETL Developer by choice

"Sure we have lots of reasons for being rude - But no excuses
Rate this response:  
Not yet rated
chulett

Premium Poster


since January 2006

Group memberships:
Premium Members, Inner Circle, Server to Parallel Transition Group

Joined: 12 Nov 2002
Posts: 42762
Location: Denver, CO
Points: 220350

Post Posted: Sun Jul 02, 2017 7:15 am Reply with quote    Back to top    

Before Ray comes along and says something, they are hashed files not hash files. Wink

I would also note that "best practice" suggests you should only set the one specific hashed file to 64bit, not every single hashed file on the server, which is what happens when you change that uvconfig option. Best to stick with option #2 in the posted link IMHO.

_________________
-craig

Research shows that 6 out of 7 dwarves aren't happy
Rate this response:  
Not yet rated
Leonie
Participant



Joined: 31 Jul 2003
Posts: 16
Location: South Africa
Points: 150

Post Posted: Mon Jul 03, 2017 1:55 am Reply with quote    Back to top    

Did the uv regen
Added the file via the command line.
It gets up to 2.1GB and then just hang there - job doesn't fail - doesn't complete.
Could it be a limit in version (11)?
Rate this response:  
Not yet rated
ray.wurlod

Premium Poster
Participant

Group memberships:
Premium Members, Inner Circle, Australia Usergroup, Server to Parallel Transition Group

Joined: 23 Oct 2002
Posts: 54395
Location: Sydney, Australia
Points: 295036

Post Posted: Mon Jul 03, 2017 5:15 am Reply with quote    Back to top    

There's no new limit in version 11.

As Craig noted it is unwise to set 64BITFILES in uvconfig, as this will force every hashed file to use 64-bit addressing.

Better would be to delete your problematic hashed file, and recreated it with the -64bit option.

On your install media you will find a Hashed File Calculator in the unsuppored utilities; this will allow you to specify a size and generate a command for creating the hashed file.

_________________
RXP Services Ltd
Melbourne | Canberra | Sydney | Hong Kong | Hobart | Brisbane
currently hiring: Canberra, Sydney and Melbourne (especially seeking good business analysts)
Rate this response:  
Not yet rated
Leonie
Participant



Joined: 31 Jul 2003
Posts: 16
Location: South Africa
Points: 150

Post Posted: Mon Jul 17, 2017 2:12 am Reply with quote    Back to top    

The 64BITFILES option in the uvconfig file is working

We missed one of the regen steps - when we did it - it worked fine.

Thanks for all the help
Rate this response:  
Not yet rated
ray.wurlod

Premium Poster
Participant

Group memberships:
Premium Members, Inner Circle, Australia Usergroup, Server to Parallel Transition Group

Joined: 23 Oct 2002
Posts: 54395
Location: Sydney, Australia
Points: 295036

Post Posted: Mon Jul 17, 2017 3:59 am Reply with quote    Back to top    

It's still by far the poorer choice.

_________________
RXP Services Ltd
Melbourne | Canberra | Sydney | Hong Kong | Hobart | Brisbane
currently hiring: Canberra, Sydney and Melbourne (especially seeking good business analysts)
Rate this response:  
Not yet rated
Display posts from previous:       

Add To Favorites
View next topic
View previous topic
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum



Powered by phpBB © 2001, 2002 phpBB Group
Theme & Graphics by Daz :: Portal by Smartor
All times are GMT - 6 Hours