Not Enough space

srekant · Post by **srekant** » Tue Jul 14, 2009 12:32 pm

Hi,

I am facing a unique problem when i am using a 4 node configuration problem for Datastage 7.5.2 on HP-UX Itanium 64 bit box. We are running our jobs in production successfully till yesterday and all of a sudden we are facing below error:

Could not map table file "/scratch/proj/DataSets/lookuptable.20090714.mf0ymnc": Not enough space

It looks simple and though of space problem on the partition but i was wrong as i ran the job and started monitoring the server. 250GB of free space is available for this partition.

When i use single node the job is running fine but is very slower than a 4 node configuration file.

Any suggestion on where the problem could be?

jcthornton · Post by **jcthornton** » Tue Jul 14, 2009 4:11 pm

While it could be a disk space issue, it is more likely to be running out of memory.

A recommendation to try would be to monitor the swap memory space while the process is running using both the single node and the 4-node configurations.

ArndW · Post by **ArndW** » Wed Jul 15, 2009 2:23 am

Do you have a lookup stage and if so, how big is your lookup data (approximately)?

srekant · Post by **srekant** » Wed Jul 15, 2009 5:37 am

I have a look up and it is approximately 500 MB

srekant · Post by **srekant** » Wed Jul 15, 2009 5:54 am

I was using glance to monitor SWAP and DISK utilisations and i noticed that after few minutes of starting this job it is spiking to 100 % utilisation and then after few minutes the job abends. But not able to interpret what 100% means since the disk partiitons have enough space.

ArndW · Post by **ArndW** » Wed Jul 15, 2009 6:05 am

If you are on AIX then this thread will help solve the problem. I thought that the solution was in another thread, with some more details, but couldn't find it.

I think that if the lookup exceeds 512Mb you get this error unless extended the space. Note that if you have large VarChar columns in the lookup you might save memory space by removing the explicit length declarations.

srekant · Post by **srekant** » Wed Jul 15, 2009 6:40 am

I ran my process and below is the glance report

GlancePlus C.04.50.00 07:38:27 usplcpd1 ia64 Current Avg High
------------------------------------------------------------------------------------------------------------------------------------------------------
CPU Util S SU | 7% 7% 19%
Disk Util F F |100% 50% 100%
Mem Util S SU UB B | 31% 31% 31%
Swap Util U UR R | 10% 10% 10%
------------------------------------------------------------------------------------------------------------------------------------------------------
IO BY DISK Users= 4

chulett · Post by **chulett** » Wed Jul 15, 2009 6:54 am

So, H-PUX. And that's disk I/O not anything related to disk space.

sanjay · Post by **sanjay** » Wed Jul 15, 2009 8:35 pm

Sreekant

you have implement memory window option in HP unix to resolve above issue below issue or join stage .

Thanks
Sanjay

chulett wrote:So, H-PUX. And that's disk I/O not anything related to disk space.

ray.wurlod · Post by **ray.wurlod** » Wed Jul 15, 2009 11:35 pm

sanjay wrote:you have implement memory window option in HP unix to resolve above issue below issue .

Please explain how you believe the memory window option solves lack of physical disk space.

I'm not really certain what you mean by "above issue below issue" either.

ArndW · Post by **ArndW** » Thu Jul 16, 2009 1:27 am

The "Not enough space" message almost always refers to disk space. I have fought with this issue before. After intensive checking during runtime it became clear that we had no disk issues, but that the lookup stage we were using tried to allocate more than 512 Mb. The solution is to use "ldedit" to change the osh executable so that it can allocate more virtual memory using the MAXDATA subcommand.

sanjay · Post by **sanjay** » Thu Jul 16, 2009 2:55 am

Ray , Arndw

Ray its not lack of physical disk space 250GB of free space is available

above issue means
Could not map table file "/scratch/proj/DataSets/lookuptable.20090714.mf0ymnc": Not enough space

In HP unix it can allocate only 1 GB of shared memory for 32 bit application . so if it goes above 1 GB job aborts . so there is feature available in HP unix i.e memory window which will allocate 1 GB for each node suppose u have 4 nodes

in datastage 7.5.2 osh is a 32 bit application file . not sure about datastage 8 ldedit is available only in AIX . equivalent change in HP unix in maxdsize but changing maxdsize doesn't reslove issue

Thanks
Sanjay

ArndW wrote:The "Not enough space" message almost always refers to disk space. I have fought with this issue before. After intensive checking during runtime it became clear that we had no disk issues, but that the lookup stage we were using tried to allocate more than 512 Mb. The solution is to use "ldedit" to change the osh executable so that it can allocate more virtual memory using the MAXDATA subcommand.