CFF and merging records back together
Posted: Wed Dec 17, 2008 12:32 pm
I have a CFF stage that's receiving data from a COBOL copy book with REDEFINES and OCCURS clauses. (Brings back memories from old days).
What i'd like to do is to produce some kind of a sequential record number so that i can match the records together later on.
To give you an example here's the input data :
In this case the header record should be bypassed but BILling, SHiPping and DRop Ship data will have their own look ups and quality checks. (So, they go to different processing paths)
Since the first ### header record defines a group eventually i would like to group the data together the same way that they came in.
A single join between the different streams are not going to give me what i want. If this was a server job, i'd add @INROWNUM to the flat file and eventually merge them together according to their record sequence number. The problem is i cannot use in multi node PX and when i have different levels of OCCURS .
Any ideas on what would be a good solution to this?
Thanks..
What i'd like to do is to produce some kind of a sequential record number so that i can match the records together later on.
To give you an example here's the input data :
Code: Select all
###Header aaa bbb ccc
BIL John Doe 124 main st.
SHP John Doe 5445 good ave.
DRP Jane Doe 2345242 tttyyy %10 gghgh
### header aaa bbb ddd
Since the first ### header record defines a group eventually i would like to group the data together the same way that they came in.
A single join between the different streams are not going to give me what i want. If this was a server job, i'd add @INROWNUM to the flat file and eventually merge them together according to their record sequence number. The problem is i cannot use in multi node PX and when i have different levels of OCCURS .
Any ideas on what would be a good solution to this?
Thanks..