filter command with multiple readers per node

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
zulfi123786
Premium Member
Premium Member
Posts: 730
Joined: Tue Nov 04, 2008 10:14 am
Location: Bangalore

filter command with multiple readers per node

Post by zulfi123786 »

I was hoping to understand how filter command with multiple readers per node works in seq file stage.

I was trying to read a file with the filter command "sed '1d;$d'" to remove the first and last lines of the file and reading it with 2 readers per node and all the time the jobs starts reading the file with no issue but when the file is just on completion of the read process the job aborts saying "broken pipe"

All works fine with 1 reader per node. Guess the data is broken and read by 2 readers then filter command applies.
Post Reply