RCP effect on Modify Stage

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
pbttbis
Premium Member
Premium Member
Posts: 36
Joined: Thu Dec 11, 2014 3:30 am
Location: South Africa
Contact:

RCP effect on Modify Stage

Post by pbttbis »

Hi All,

I have a job reading a dataset, with many stages preceding a Modify stage going into a lookup stage, all of which have RCP enabled, except the modify stages before the lookup stage.

There is a parallel flow which is funneled with the output of the lookup stage above. This stream also uses RCP with the min cols specified explicitly.

What I have found is that enabling or disabling RCP on the Modify stage does not have an impact on the columns sent on the lookup leg of the lookup stage. It then causes warnings in the log as these columns needs to be dropped in the subsequent funnel stage and defaulting cols not on the other/right link going into the funnel stage.

I was expecting the Modify stage to only propagate the column specified on the output link when RCP is not enabled (without having to drop these columns explicitly).

Is my understanding of RCP on this stage incorrect?

Regards
Paul
PBT TBIS Consultant
Post Reply