key change column with sort stage
Posted: Thu Feb 06, 2020 2:51 am
Hi
I have designed a job to remove duplicates using the key change column in sort stage. I have written the records with the key change value=1 into target1 file with constraint "keychange value =1" in transformer and other records key change column value=0 with constraint in transformer with "keychange value=0" into other target file.
designed job as below
but can I remove the copy stage and use only one transformer rather than using two transformers as above and write any condition in transformer and then output to two target files?
Please let me know. I am not able to find attachments option in this message otherwise i would have shown the job design in this message
Thanks
Rakesh
I have designed a job to remove duplicates using the key change column in sort stage. I have written the records with the key change value=1 into target1 file with constraint "keychange value =1" in transformer and other records key change column value=0 with constraint in transformer with "keychange value=0" into other target file.
designed job as below
Code: Select all
seqfile --sort stage--copystage--transformer 1 ---target 1
--transformer 2 ---target 2
Please let me know. I am not able to find attachments option in this message otherwise i would have shown the job design in this message
Thanks
Rakesh