Abnormal termination of DataStage.

Post questions here relative to DataStage Server Edition for such areas as Server job design, DS Basic, Routines, Job Sequences, etc.

Moderators: chulett, rschirm, roy

zam62
Participant
Posts: 42
Joined: Tue Apr 29, 2003 12:21 pm

Abnormal termination of DataStage.

Post by zam62 »

We have a job that runs for around 40 minutes and then aborts with no reason given in Director. Looking at the log in the &PH& directory, this is the error I see. Whare can I go to find out what is causing it?


./DSD_BP.O/DSD_Update.BDataStage Job 1426 Phantom 28495
Abnormal termination of DataStage.
Fault type is 11. Layer type is BASIC run machine.
Fault occurred in BASIC program DSD.Update at address 124
kcbland
Participant
Posts: 5208
Joined: Wed Jan 15, 2003 8:56 am
Location: Lutz, FL
Contact:

Post by kcbland »

Since you've looked at the &PH& entry, your next step is to identify the transformer stage that blew up tragically. Was the job running any kind of after-stage routine or after-job routine when it died? A lot of the time, these subroutines can have issues such as calls to other subroutines that don't exist or aren't compiled and therefore cause abnormal terminations.

The other thing to look at is what your job is doing. If it's building a default 32-bit hash file and exceeds 2.2 gigabytes then this message happens. Can you give us a little more design information?
Kenneth Bland

Rank: Sempai
Belt: First degree black
Fight name: Captain Hook
Signature knockout: right upper cut followed by left hook
Signature submission: Crucifix combined with leg triangle
zam62
Participant
Posts: 42
Joined: Tue Apr 29, 2003 12:21 pm

Post by zam62 »

There are no pre/post job or Trn routines. The job has an OCI8 primary linkinto a trn with an OCI8 lookup and a hash file lookup. It has output links for INSERT into an OCI8 table and a reject text file if either lookup is not found. Ther is some calculations and checks in the trn, but nothing out of the ordinary.
kduke
Charter Member
Charter Member
Posts: 5227
Joined: Thu May 29, 2003 9:47 am
Location: Dallas, TX
Contact:

Post by kduke »

I hate it when things blow up tragically. Nice blow ups are so much better ways to die.

I think what Ken is saying is there are failures which DataStage can easily trap. This jobs will end under DataStage control. There are other times when a job fails by violating something that the OS does not allow therefore the OS will terminate that process before DataStage can trap it. It terminates or dies more abruptly or tragically.

Geez Ken, it sounds like your best friend died. Blame Dennis because he always gives me a hard time about how I phrase things. You can take the boy out of Oklahoma but not the Okie out of the boy. Good thing your apostrophes are correct. You don't want those police too.
Mamu Kim
zam62
Participant
Posts: 42
Joined: Tue Apr 29, 2003 12:21 pm

Post by zam62 »

I also noticed that there is a 'core' file in the project directory. Is there and easy way to read this file?

The job processed 3.3 milion records, reading all from hash and OCI lookups. The database is actually an Oracle 9i database and we usually use OCI9 stages, but the developer grabbed the wrong stage. IS that an issue?
kcbland
Participant
Posts: 5208
Joined: Wed Jan 15, 2003 8:56 am
Location: Lutz, FL
Contact:

Post by kcbland »

Could be an issue, one customer of mine uses OCI8 with 8i client to talk to 9i databases, but OCI8 with 9i client talking to 9i, not sure.

One last thing to look at is if you have row buffering turned on. Search the forum, but this is a finicky feature and you may wish to try turning it off after you swap out the stages.
Kenneth Bland

Rank: Sempai
Belt: First degree black
Fight name: Captain Hook
Signature knockout: right upper cut followed by left hook
Signature submission: Crucifix combined with leg triangle
ketfos
Participant
Posts: 562
Joined: Mon May 03, 2004 8:58 pm
Location: san francisco
Contact:

Post by ketfos »

Hi,

We have been using OCI8 with 9i client talking to 9i and it works fine.


Ketfos
smohamme
Participant
Posts: 9
Joined: Sat Aug 07, 2004 2:48 pm
Location: Dallas
Contact:

Abnornal Termination

Post by smohamme »

We had a similar issue and was related to a 32-bit Oracle client on the ETL box talking to a 64-bit Oracle server. Apparently there is a memory leak when a 32-bit client talks to a 64-bit server and fails with an abnormal termination error and fails while processing around 15.7 million records.
smohamme
MukundShastri
Premium Member
Premium Member
Posts: 103
Joined: Tue Oct 14, 2003 4:07 am

Post by MukundShastri »

I have tried ORAOCI 8 with 9i client . It doesn't work. I checked with forum on this and got the reply for ORAOCI 8 we should use 8 i client even if database is 9i.
Try doing that. I should work.

Thanks
Mukund
kduke
Charter Member
Charter Member
Posts: 5227
Joined: Thu May 29, 2003 9:47 am
Location: Dallas, TX
Contact:

Post by kduke »

We use a Oracle 8 client with a Oracle 9 database. Works flawlessly.
Mamu Kim
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

As have I...
-craig

"You can never have too many knives" -- Logan Nine Fingers
datastagedummy
Participant
Posts: 56
Joined: Thu Feb 13, 2003 6:08 pm
Location: USA

OCI to ODBC

Post by datastagedummy »

We had this issue and it was resolved by replacing the OCI stage with ODBC stage try using that
smohamme
Participant
Posts: 9
Joined: Sat Aug 07, 2004 2:48 pm
Location: Dallas
Contact:

Re: Abnornal Termination

Post by smohamme »

smohamme wrote:
We had a similar issue and was related to a 32-bit Oracle client on the ETL box talking to a 64-bit Oracle server. Apparently there is a memory leak when a 32-bit client talks to a 64-bit server and fails with an abnormal termination error and fails while processing around 15.7 million records.
Update on this is that we have applied an Oracle Patch to the 32-bit client and it fixed the abnormal termination issues.
smohamme
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Please advise the patch identification. Others are seeing this too.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
smohamme
Participant
Posts: 9
Joined: Sat Aug 07, 2004 2:48 pm
Location: Dallas
Contact:

Post by smohamme »

ray.wurlod wrote:Please advise the patch identification. Others are seeing this too.
The details of the patch that allowed us to fix the "Abnormal Termination..." are as follows:

Oracle Bug number is 2298968 and detail information is as follows:

********************( BUGTAG NOTES - See <Note:66631.1> )******************
BugTag: Support notes on <Bug:2298968> - DDR info <BugDesc:2298968>
Affects: RDBMS (81-92)
NB: *Serious* FIXED
Abstract: Client side memory leak for 32<->64 bit SELECTs
Fixed-Releases: 8175 9014 9201
Tags: EXP INTEROP LEAK/MEM R8173
Details:
Client side memory leak for 32<->64 bit SELECTs.
This client side bug can have a severe effect on EXPORT.
A 32bit EXP process may grow dramatically if exporting
a 64 bit database. ("exp" is 32 bit by default for most
64bit releases of Oracle).

The client (on etl box) and server (on database box) version of oracle are both 8.1.7.4, only the client is 32-bit and server is 64-bit.
smohamme
Post Reply