Data Not Passing through Transformer

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
samyamkrishna
Premium Member
Premium Member
Posts: 258
Joined: Tue Jul 04, 2006 10:35 pm
Location: Toronto

Data Not Passing through Transformer

Post by samyamkrishna »

Hi,

The parallel job is not letting any data pass through the transformer.
There are not constraints.
There are no warnings in the director.

I am completely lost.
:?

I read a post here regarding transformer issue between 8.5 and 9.1.
But I am designing on 8.5 and running on 8.5 too.

Regards,
Samyam
Cheers,
Samyam
asorrell
Posts: 1707
Joined: Fri Apr 04, 2003 2:00 pm
Location: Colleyville, Texas

Post by asorrell »

Samyam - we are also lost. You give us no details about how the transformer is setup, so we can't help.
Andy Sorrell
Certified DataStage Consultant
IBM Analytics Champion 2009 - 2020
samyamkrishna
Premium Member
Premium Member
Posts: 258
Joined: Tue Jul 04, 2006 10:35 pm
Location: Toronto

Post by samyamkrishna »

So the code is extracting data from DB2 into a transformer and then writing it into a seq file.

In the performance stats I can see 10 records extracted from the DB2 and 0 records after the transformer.
Cheers,
Samyam
Thomas.B
Participant
Posts: 63
Joined: Thu Apr 09, 2015 6:40 am
Location: France - Nantes

Post by Thomas.B »

Are you sure the lines are not written to your output file? Sometimes DataStage removes some useless stages while running.
Because the link is believed useless the performance stats are not written under it, but they are actually drag to the output.
Last edited by Thomas.B on Fri Apr 08, 2016 10:40 am, edited 1 time in total.
BI Consultant
DSXConsult
samyamkrishna
Premium Member
Premium Member
Posts: 258
Joined: Tue Jul 04, 2006 10:35 pm
Location: Toronto

Post by samyamkrishna »

File is empty.
Cheers,
Samyam
FranklinE
Premium Member
Premium Member
Posts: 739
Joined: Tue Nov 25, 2008 2:19 pm
Location: Malvern, PA

Post by FranklinE »

I believe the best help you can provide is to post the Director messages for the job. I would guess you have warnings that might help explain this.
Franklin Evans
"Shared pain is lessened, shared joy increased. Thus do we refute entropy." -- Spider Robinson

Using mainframe data FAQ: viewtopic.php?t=143596 Using CFF FAQ: viewtopic.php?t=157872
samyamkrishna
Premium Member
Premium Member
Posts: 258
Joined: Tue Jul 04, 2006 10:35 pm
Location: Toronto

Post by samyamkrishna »

Nope no warning.
Cheers,
Samyam
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

I'm going to guess you have a metadata issue between the DB2 data and the flat file and they are being 'rejected' by the target stage. You might want to change the Reject Mode on it to something other than the default of 'Continue' and see if that helps shed some light.
-craig

"You can never have too many knives" -- Logan Nine Fingers
samyamkrishna
Premium Member
Premium Member
Posts: 258
Joined: Tue Jul 04, 2006 10:35 pm
Location: Toronto

Post by samyamkrishna »

Hi,

yes it turned out to be a metadata issue.
specifically turned out to be NULL handling capability in the transformer.

Once we added the NULL handling function the code worked as expected.

The only surprising thing about this whole episode is there are no warning on the director.

Regards,
Samyam
Cheers,
Samyam
asorrell
Posts: 1707
Joined: Fri Apr 04, 2003 2:00 pm
Location: Colleyville, Texas

Post by asorrell »

There should have been a warning about trying to write a NULL to the sequential file. I suspect that somebody suppressed the message at the project level, not realizing that is a VERY bad precedent.

I would suggest checking out your project level message handler.
Andy Sorrell
Certified DataStage Consultant
IBM Analytics Champion 2009 - 2020
samyamkrishna
Premium Member
Premium Member
Posts: 258
Joined: Tue Jul 04, 2006 10:35 pm
Location: Toronto

Post by samyamkrishna »

Hi Andy,

I checked the MSG HANDLER.
There is nothing to suppress any NULL related warnings.

Regards,
Samyam
Cheers,
Samyam
rschirm
Premium Member
Premium Member
Posts: 27
Joined: Fri Dec 13, 2002 2:53 pm

Post by rschirm »

Find the entry in the log or the sequential output. It will state that 0 rows written and NNNN# of rows rejected.
Post Reply