Fixed width Cobol File

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
rumu
Participant
Posts: 286
Joined: Mon Jun 06, 2005 4:07 am

Fixed width Cobol File

Post by rumu »

Hi All,
My next file is a Fixed width with Record length 5636(as per cobol copy book).

I used the following settings:
ByteOrder=Bigendian
CharacterSet=EBCDIC
dataformat=binary

Record Delimeter I set blank.
In the copybook there is one FIXED OCCUR clause which repeats 99 times.I flatten this array.
When I run this job I got issue after 32967 records as input buffer overrun ar the field where the above FIXED OCCUR mentioned. But the 32967 records are also not read correctly.
With blank record delimeter, the number of record from the file was wrong.

When I change the Record delimeter to Unix newline, not a single record read as it started throing warning for input buffer overrun at the FIXED OCCUR field. But in this case, total number of fields read from the file(warning message) are same with the one getting in Unix using wc-1.

For fixed width record, do we need to use Record delimeter?
Rumu
IT Consultant
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

A true fixed-width file won't have one. It reads a "record" at a time based on the length you define.
-craig

"You can never have too many knives" -- Logan Nine Fingers
rumu
Participant
Posts: 286
Joined: Mon Jun 06, 2005 4:07 am

Post by rumu »

Hi Craig,

I agree. The copybook has record length 5636 and there is no OCCURS depending clause and team said it is fixed width.

In unix the file length is as below:

Code: Select all

-bash-4.2$ ls -l MONDET.D1181108.T191639
-rw-r--r-- 1 bb5g_etl edld_bb5g_etl 185854812 Nov  9 02:15 MONDET.D1181108.T191639
And record count is
-bash-4.2$ wc -l MONDET.D1181108.T191639
224348 MONDET.D1181108.T191639
When I divide size 185854812 by records count 224348 then I got 828.42 which is supposed to be record length and it should have been an integer equal to 5636..
Is my understanding correct? Am I missing anything? I think I need to report it that copybook and datafile is not aligned.
Rumu
IT Consultant
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

You certainly need to have a conversation with the file provider (rather than us) to have them clarify exactly what it is they are sending you rather than have you continue to try to puzzle it out. There are different kinds of files that people call "fixed width" and the kind I was mentioning are basically what you used to find on tape (yay) that when dumped to disk, a "wc -l" would return a record count of 1. Hence my comment about record delimiters. Also note you may be getting a false count if there are packed fields in there that include "0A" as those will be counted as record delimiters by "wc" as it is rather brain dead. I also seem to remember issues with fixed-width files with record delimiters being mangled during file transfers because some transfer mechanisms can remove all of those "unnecessary" trailing spaces at the end of each record.

This article might help with the concepts. Stumbled across it searching this morning.

Bottom line is still you needing to get all of the details you need to process the file from whomever is providing the file to you. Rather than random strangers on the Internet. :wink:
-craig

"You can never have too many knives" -- Logan Nine Fingers
rumu
Participant
Posts: 286
Joined: Mon Jun 06, 2005 4:07 am

Post by rumu »

Hi Craig,
I wrote a script in phython to loop through the file and print length of each line and total file count. I found that each record length is variable. Hence I use a script to pad each record to make it fixed length . Then I use my copybook to read it and was successful. So I am marking this issue resolved.
Rumu
IT Consultant
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Okay, fair enough, but to me that falls squarely into the "Workaround" camp and should still be resolved in a more proper fashion. YMMV, of course.
-craig

"You can never have too many knives" -- Logan Nine Fingers
rumu
Participant
Posts: 286
Joined: Mon Jun 06, 2005 4:07 am

Post by rumu »

I agree Craig. I changed it to Workaround.
Rumu
IT Consultant
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Your arithmetic will be out if there are any COMP fields.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
rumu
Participant
Posts: 286
Joined: Mon Jun 06, 2005 4:07 am

Post by rumu »

Hi Ray,

There are comp fields and after reading it through CFF stage, I compared the data with the existing one(SAS program) and all the fields are matching...
Rumu
IT Consultant
Post Reply