Page 1 of 1

Record Count from Lookup is only showing in Node 0.

Posted: Tue Jun 28, 2016 8:21 am
by anajitKS
I have developed a parallel job where source data is coming from Netezza. I have used Netezza Connector. Records are read in Sequential Mode.
The next stage is LookUp Stage where I have partioned the source data using Modulous. Partition scheme for the reference data ( coming through Netezza connector and read in sequential mode) is set to 'Entire' Partition scaheme for all subsequent stages are set to 'Same' .

I set to $APT_DUMP_SCORE to True.

After running the job the log shows the lookup data loaded only in Node 0.
But the target table ( Netezza) got data from 2 nodes.

I was expecting to see the record count from Lookup data to show up in 2 nodes.
Is my expectation wrong ? or I need to do something else see the lookup data count in 2 nodes?

Posted: Tue Jun 28, 2016 11:53 pm
by ray.wurlod
For Entire partitioning there is no point in reporting the other nodes.

Indeed, in an SMP environment, only one copy of the data exists, stored in shared memory that is accessible by all nodes.