Hi
My job uses a funnel stage. The output of funnel is fed to a filter stage where it directs to various tables based on the value of a field. The problem I encountered here is like this. The volume of my source tables are in the order of 20 million rows. My extraction query fetches the rows within 15 minutes. But all the records remain in the funnel stage's input link, making the funnel as a bottleneck. This severely affects the performance of my job. Infact, the funnel type I set is 'Continuous Funnel' and partition type is 'Auto'. Still the stage remains as a bottleneck for my job. Is there any other properties to be set ? Is there any replacement for funnel stage ? Please do reply if you know since the requirement is very urgent.
Thanks & Regards,
Vignesh.
Performance issue with Funnel Stage
Moderators: chulett, rschirm, roy