Data Stage Ver 7.5.2 with 4 Quad Core CPU

Post questions here relative to DataStage Server Edition for such areas as Server job design, DS Basic, Routines, Job Sequences, etc.

Moderators: chulett, rschirm, roy

Post Reply
sysmq
Premium Member
Premium Member
Posts: 29
Joined: Wed Aug 22, 2007 12:58 am

Data Stage Ver 7.5.2 with 4 Quad Core CPU

Post by sysmq »

Hi
I have a job that create a hash file and then loads it to the memory for manipulation the output is written to DB

on the old server (2 cpu) windows 2003 sp1 3.5G Ram,
it works great and the cpu is 50-70% work during the job execution

we bought a new server with a 4 quad core CPU with windows 2003 sp1 with 3.5G Ram and the job crushes with "unable_to_alocate_memory()"
during the job execution there is no CPU activity but i have 7.2G Of ram
alocated (the entier RAM and PAGEFILE )

My question is :
Does Version 7.5.2 knows to work with a QuadCore CPU ?

sysmq
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Of course. Your CPUs, quad or otherwise, have nothing to do with this. You simply ran out of allocatable memory, which is nowhere near the 7.2GB you think you have. This is 32bit architecture we're talking about here, after all.

What is the default hashed file memory allocation size on the new server, did you bump it up higher than the default of 128MB? Or at least higher than the old server? That's typically what causes this error, that combined with caching, which is what I assume you mean by "then loads it to memory for manipulation".
-craig

"You can never have too many knives" -- Logan Nine Fingers
sysmq
Premium Member
Premium Member
Posts: 29
Joined: Wed Aug 22, 2007 12:58 am

Post by sysmq »

HI
I know that my server went out of memory
The question is why.
let me elaborate:
In the old server with the same Os and RAM running the same project against the same DB finishes the job on time (2-3 hours) with no errors the CPU is at 50-70 % and memory is within OS limitation (less then 3.5G)

The new server does not finish the job and crashes with the memory issue os finishes after almost 14 hours.
more over the CPU does not participate at all 0-1%
and memory reaches to 7.2G (3.5 RAM and the rest Page File)

The alocated memory in the admin to the project is 512 on both servers
they have the same UVCONFIG file

What else can i check that can help me understand the difference in behaviour?

sysmq
:twisted:
Mike
Premium Member
Premium Member
Posts: 1021
Joined: Sun Mar 03, 2002 6:01 pm
Location: Tampa, FL

Post by Mike »

A server is a lot more than just an OS and RAM. What is different about the new server such that it is not able to supply as many resources to your DataStage application as the old server? What else is consuming resources?

Mike
sysmq
Premium Member
Premium Member
Posts: 29
Joined: Wed Aug 22, 2007 12:58 am

Post by sysmq »

hi
nothing else is consuming the memory
we use performance monitor
the whole project loads more then 16 uvsh process each with 400 mb memory

after the job finishes the memory is reduced to 812 mb total.

the main differance between the server is :
on is 2 cpu P4 3.0 GHz and the new is Quadcore 2.8 GHz exeon

one is a DL 380 server and the new one is a BLADE server

that is all

sysmq
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Not sure how anyone here can really help, seeing as we're not there and not able to examine both servers ourselves. However, you are, which is why that duty falls to you and yours.

Something is different between the two, something other than simply the CPU. Sure, there are teeny differences between the two chips, but you'll not notice a .2GHz drop in speed, nor probably any difference in things like L2 Cache that they may have. And they certainly shouldn't be the prime suspects in an 'out of memory' case IMHO. Look elsewhere. Involve your administrators, the people who's job it is to look after the machines. It could be a difference in patch levels, in spite of the fact that you say both have "SP1" installed. Really shouldn't be DataStage itself if you are certain that both servers have the exact same version installed, with the exact same uvconfig & administrator settings, etc etc.

Who knows? Difficult to diagnose from this side of the glass, which is why this falls squarely on your shoulders.
-craig

"You can never have too many knives" -- Logan Nine Fingers
sysmq
Premium Member
Premium Member
Posts: 29
Joined: Wed Aug 22, 2007 12:58 am

Post by sysmq »

hi
nothing else is consuming the memory
we use performance monitor
the whole project loads more then 16 uvsh process each with 400 mb memory

after the job finishes the memory is reduced to 812 mb total.

the main differance between the server is :
on is 2 cpu P4 3.0 GHz and the new is Quadcore 2.8 GHz exeon

one is a DL 380 server and the new one is a BLADE server

that is all

sysmq
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Not sure how anyone here can really help, seeing as we're not there and not able to examine both servers ourselves. However, you are, which is why that duty falls to you and yours.

Something is different between the two, something other than simply the CPU. Sure, there are teeny differences between the two chips, but you'll not notice a .2GHz drop in speed, nor probably any difference in things like L2 Cache that they may have. And they certainly shouldn't be the prime suspects in an 'out of memory' case IMHO. Look elsewhere. Involve your administrators, the people who's job it is to look after the machines. It could be a difference in patch levels, in spite of the fact that you say both have "SP1" installed. Really shouldn't be DataStage itself if you are certain that both servers have the exact same version installed, with the exact same uvconfig & administrator settings, etc etc.

Who knows? Difficult to diagnose from this side of the glass, which is why this falls squarely on your shoulders.
-craig

"You can never have too many knives" -- Logan Nine Fingers
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Two can play that game. :wink:
-craig

"You can never have too many knives" -- Logan Nine Fingers
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

sysmq wrote:My apologies. I send it by mistake due to an error on my browser.
I know, just mess'n with ya, hence the smilie. We see plenty of double posts here, just not all that many 47 minutes apart. Couldn't resist.
-craig

"You can never have too many knives" -- Logan Nine Fingers
Post Reply