Record Count from Lookup is only showing in Node 0.
Posted: Tue Jun 28, 2016 8:21 am
I have developed a parallel job where source data is coming from Netezza. I have used Netezza Connector. Records are read in Sequential Mode.
The next stage is LookUp Stage where I have partioned the source data using Modulous. Partition scheme for the reference data ( coming through Netezza connector and read in sequential mode) is set to 'Entire' Partition scaheme for all subsequent stages are set to 'Same' .
I set to $APT_DUMP_SCORE to True.
After running the job the log shows the lookup data loaded only in Node 0.
But the target table ( Netezza) got data from 2 nodes.
I was expecting to see the record count from Lookup data to show up in 2 nodes.
Is my expectation wrong ? or I need to do something else see the lookup data count in 2 nodes?
The next stage is LookUp Stage where I have partioned the source data using Modulous. Partition scheme for the reference data ( coming through Netezza connector and read in sequential mode) is set to 'Entire' Partition scaheme for all subsequent stages are set to 'Same' .
I set to $APT_DUMP_SCORE to True.
After running the job the log shows the lookup data loaded only in Node 0.
But the target table ( Netezza) got data from 2 nodes.
I was expecting to see the record count from Lookup data to show up in 2 nodes.
Is my expectation wrong ? or I need to do something else see the lookup data count in 2 nodes?