Identifying Header,Trailer and Detail in CFF Stage

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
shiva459
Premium Member
Premium Member
Posts: 100
Joined: Wed Jun 11, 2003 1:49 am
Location: Bangalore

Identifying Header,Trailer and Detail in CFF Stage

Post by shiva459 »

I have a EBCDIC data file which has a header, trailer and detail records. The Cobol file descriptor has a REC-ID field to identify header and trailer but no identifier for detail records. There is only one detail records type so if it is not header or trailer then it is detail.

I am using CFF stage and configured it to multiple records mode but the problem is I have to define a record identifier for detail in the records ID tab which is mandatory.

Can any one please let me know the work around? I tried creating two diff jobs to read header/trailer and detail respectively but getting some errors. I trying to ascertain that the source file is not corrupted but any thought with the design itself?

Thanks for your time !
FranklinE
Premium Member
Premium Member
Posts: 739
Joined: Tue Nov 25, 2008 2:19 pm
Location: Malvern, PA

Post by FranklinE »

I've offered general disdain about this in the past, but your situation quite seriously is a result of bad design. One simply doesn't break consistency like that. The lack of a an explicit identifier for details is a defect that should be fixed.

That said, I don't think you can use CFF for this. First thought is a filter or constraint that lets you put details on a "reject" link after finding the header and trailer, from which you must then parse the record to its original layout.

The other thought is to edit the data before it reaches the CFF stage, and prefix the details with a record id.
Franklin Evans
"Shared pain is lessened, shared joy increased. Thus do we refute entropy." -- Spider Robinson

Using mainframe data FAQ: viewtopic.php?t=143596 Using CFF FAQ: viewtopic.php?t=157872
FranklinE
Premium Member
Premium Member
Posts: 739
Joined: Tue Nov 25, 2008 2:19 pm
Location: Malvern, PA

Post by FranklinE »

I thought of a possible workaround. It assumes that you can read the source data as a sequential file, and ignores CFF entirely.

1. Set First line is column names to true.
2. Use a constraint or filter to remove the trailer, referencing the bytes of the first field in the details layout that correspond to REC-ID.

This also assumes that you don't need any data from the header or trailer.
Franklin Evans
"Shared pain is lessened, shared joy increased. Thus do we refute entropy." -- Spider Robinson

Using mainframe data FAQ: viewtopic.php?t=143596 Using CFF FAQ: viewtopic.php?t=157872
shiva459
Premium Member
Premium Member
Posts: 100
Joined: Wed Jun 11, 2003 1:49 am
Location: Bangalore

Post by shiva459 »

Thanks, Franklin
I was trying on similar lines. My idea was to read the records as a single string using sequential file stage and choose character set as EBCDIC to ASCII and data format as binary. The only problem is the fields defined as packed decimal are no getting converted to readable format. Since I defined the entire record as a string not able to handle the packed decimals at properties level. Any suggestions?
FranklinE
Premium Member
Premium Member
Posts: 739
Joined: Tue Nov 25, 2008 2:19 pm
Location: Malvern, PA

Post by FranklinE »

I don't have a solution for parsing packed-decimal fields out of a single-column input. I've tried (under 7.5, though) without success. Indeed, CFF is the only way to go with this unless you can edit the input file and remove the header and trailer before reading it.

I wonder if this is worth a try: along with my workaround suggestion, create a reject link so that the trailer record doesn't cause an abend or a long list of warnings. Use the original Cobol FD as the table definition, which solves the packed-decimal problem by not letting it happen, and see if you can get all your details.
Franklin Evans
"Shared pain is lessened, shared joy increased. Thus do we refute entropy." -- Spider Robinson

Using mainframe data FAQ: viewtopic.php?t=143596 Using CFF FAQ: viewtopic.php?t=157872
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

If Server jobs are at all an acceptable solution, from what I recall there are several 'EBCDIC' routines in the SDK including ones that would unpack those fields for you.
-craig

"You can never have too many knives" -- Logan Nine Fingers
Post Reply