failed to open project after running successfully in a loop

[hsahay]

Hi

We just recently upgraded to 11.3 on AIX. This is a clean fresh installation.

We have one particular job which consist of a script that loads multiple input files in a loop until all are loaded.

In each loop iteration this script does the following -

1. Calls an oracle stored proc.
2. Sends a reset command to the datastage job
3. runs the datastage job to load the file

Yesterday, after successfully iterating through the loop 31 times, the scripts failed with the following error on the 32nd iteration -

---LOG EXCERPT------------------------------------------------------------
Resetting the DataStage Job NascPrep_CwrAckSeq
Waiting for job...
Finished waiting for job
Job Status : (21)

Status code = 21
The DataStage Job NascPrep_CwrAckSeq is reset
and is ready to run............
ERROR: Failed to open project

Last recorded error message =
java.rmi.RemoteException: Read timed out; nested exception is:
java.net.SocketTimeoutException: Read timed out

Status code = -1010 DSJE_DOMAINLOGTOFAILED
----------------------------------------------------------

Failed to open project ? The same project that it had opened just fine 31 times previously ?

Any ideas what might be happening here ?

[chulett]

With it being a brand new project / etc I would have to guess you exhausted resources and in your shoes I would double-check your kernel parameter settings as a start.

[Teej]

This sounds like a resource issue with the project folder. There are some tuning options starting on the universe level and going down to the system level (including the filesystem).

AIX is very fickle in term of tuning options -- basically, they give an administrator enough rope to hang themselves with.

Try this page first:

http://www.ibm.com/support/knowledgecen ... _dsqs.html