QS webservice jobs compilation issue

Infosphere's Quality Product

Moderators: chulett, rschirm

Post Reply
dj
Participant
Posts: 78
Joined: Thu Aug 24, 2006 5:03 am
Location: india

QS webservice jobs compilation issue

Post by dj »

Hi,

Few of the web service enabled quality jobs which were running fine earlier aborted.
And the compilation failed with below error

Code: Select all


> ISD_Address_Standardization - Failed <<
Failed to invoke GenRuntime using phantom process helper.
Internal data error.
  File '/path/IBM/InformationServer/Server/Projects/MDMQS/RT_STATUS1/OVER.30':
  Computed blink of 0x90000 does not match expected blink of 0xE800!
  Detected within group starting at address 0xA1000!
  Cannot find a job number 0
Attempting to Cleanup after ABORT raised in job 0
Attempting to Cleanup after ABORT raised in job 0

DataStage Phantom Aborting with @ABORT.CODE = 3

We will try to reimport the dsx or make copy of the jobs and try to recompile.
1. Would like to understand why the job status file got corrupted?
2. How to check administrator, there are no RT_LOGS number
3. The webservice jobs are using Isd stages. After server restart do these jobs try to run automatically, I could see multiple invocation id in abort status until any instance is in running state.

Thanks
qt_ky
Premium Member
Premium Member
Posts: 2895
Joined: Wed Aug 03, 2011 6:16 am
Location: USA

Post by qt_ky »

I would guess your log file logged a HUGE number of warnings or possibly multi-instance aborts, probably exceeding 2 GB of job log details. I've run into that exact case and error before. Enable auto-purge settings.

RT_STATUS1 tells you the job ID is 1. You could try command COUNT RT_LOG1 to see how many detailed job log entries you've got.

To avoid crashed job status with ISD related jobs when stopping or bouncing the server, make sure to stop each ISD application first. This can be done from the command line and scripted. Do the opposite after startup.
Choose a job you love, and you will never have to work a day in your life. - Confucius
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Do clear the log. It's just good practice to do so. However, the hashed file that has become corrupted is RT_STATUS1, not RT_LOG1.

If you have administrative tools enabled in Director client, you can clear the job's status information from there. If you do not have these tools enabled, then execute CLEAR.FILE RT_STATUS1 (for example from the Administrator client's Command window for the project).

Then you will need to recompile the job before it can be executed again.

And before you do that, you may need to change its status to stopped via the Information Server console, if it's exposed as a web service.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
dj
Participant
Posts: 78
Joined: Thu Aug 24, 2006 5:03 am
Location: india

Post by dj »

thanks for the replies.

After trying to view the RT Log files which caused the issue, icould only find the below error in all the jobs.

Code: Select all

Fatal Error: waitForWriteSignal(): Premature EOF on node server Bad file descriptor
1) Is this error is related to server down? we are just trying to identify what caused the huge logs.

2) What happens actually when the server restarts?

Before Server restart- To stop the jobs dsjob commands or do we need to undeploy them

After restart- Do we need re-deploy or just start commands would suffice?

Thanks
qt_ky
Premium Member
Premium Member
Posts: 2895
Joined: Wed Aug 03, 2011 6:16 am
Location: USA

Post by qt_ky »

As Ray pointed out, it wasn't your job log that got filled up, rather its status file overflowed. I'm not sure what would cause that.

If you were to reach out and stop the DataStage job, ISD would notice and immediately spawn a new instance.

To script stops and starts, you need to undeploy and deploy. Here is a UNIX example of the syntax you can use within shell scripts.

ISDAdmin.sh -authfile <AUTH_FILE> -action undeploy -application <ISD_application_name>

ISDAdmin.sh -authfile <AUTH_FILE> -action deploy -project <ISD_project_name> -application <ISD_application_name>
Choose a job you love, and you will never have to work a day in your life. - Confucius
dj
Participant
Posts: 78
Joined: Thu Aug 24, 2006 5:03 am
Location: india

Post by dj »

Thanks for the details.

In addition to the status file overflowed, for the ISD jobs we could also see the /tmp folder getting full.

What are these files? Are there any harm in removing these files and why do they go to tmp?

Are there any settings we could change the directory

ad1.proj.ISD_jobname.input
ad1.proj.ISD_jobname.output

Thanks
qt_ky
Premium Member
Premium Member
Posts: 2895
Joined: Wed Aug 03, 2011 6:16 am
Location: USA

Post by qt_ky »

The TMPDIR environment variable may be set to any path you like. You can change it using DataStage Administrator, per project. After you set a new one and run the ISD jobs, or redeploy, then remove the old tmp files.
Choose a job you love, and you will never have to work a day in your life. - Confucius
dj
Participant
Posts: 78
Joined: Thu Aug 24, 2006 5:03 am
Location: india

Post by dj »

we are working with PMR team to figure out why ther server went down.

After clearing up the tmp and logs files, imported the dsx again and the jobs got compiled.

Thanks.
Post Reply