Filter option of Sequential Files stage hanging

[bobyon]

Having searched for a solution to a problem I was facing, fortunately I found someone else had the same problem. JeroenDmt described the situation identical to the situation I was facing. So I was excited to find a solution as well. See:

viewtopic.php?t=151550

However, upon implementing the cut command in the filter option as described by ssnegi I'm getting some unusual results.

My current testing results in 3 records coming out of a hierarchical stage and into the sequential file stage. I've sent them to peek previously to ensure they are formatted as necessary. Each record consists of the desired FILENAME, a | (pipe) and RESULT_STRING which is the desired xml.

The cut statement I entered into the filter option is:

Code: Select all

cut -d '|' -f 2

Which works perfectly...... on the second of the three test records. The output file created has exactly what I am looking for; only the xml. But, the first and third files are produced empty (0 length) and the job never finishes. I have to stop the job from Director.

this same behavior is happenning on two separate legs of the job. That is to say that two different seq file stages are doing the same thing.

Any idea what would cause this and/or how to resolve it?

[chulett]

Interesting... never occurred to me to leverage the filter in that stage on the way out, thought it was all about on the way in. I'm guessing it would be working more better if you weren't doing the 'create multiple output files' thing. You'll probably need to involve support for an answer.

[bobyon]

Help files seem to indicate that this should work, however....


to test things a bit further I added some input to my test base and it seems that the cut command works fine on all but the first and last files produced. Those files are still left empty and the job never ends on its own.