DSXchange

Hi All,
My next file is a Fixed width with Record length 5636(as per cobol copy book).

I used the following settings:
ByteOrder=Bigendian
CharacterSet=EBCDIC
dataformat=binary

Record Delimeter I set blank.
In the copybook there is one FIXED OCCUR clause which repeats 99 times.I flatten this array.
When I run this job I got issue after 32967 records as input buffer overrun ar the field where the above FIXED OCCUR mentioned. But the 32967 records are also not read correctly.
With blank record delimeter, the number of record from the file was wrong.

When I change the Record delimeter to Unix newline, not a single record read as it started throing warning for input buffer overrun at the FIXED OCCUR field. But in this case, total number of fields read from the file(warning message) are same with the one getting in Unix using wc-1.

For fixed width record, do we need to use Record delimeter?

A true fixed-width file won't have one. It reads a "record" at a time based on the length you define.

Hi Craig,

I agree. The copybook has record length 5636 and there is no OCCURS depending clause and team said it is fixed width.

In unix the file length is as below:
And record count is

-bash-4.2$ wc -l MONDET.D1181108.T191639
224348 MONDET.D1181108.T191639

When I divide size 185854812 by records count 224348 then I got 828.42 which is supposed to be record length and it should have been an integer equal to 5636..
Is my understanding correct? Am I missing anything? I think I need to report it that copybook and datafile is not aligned.

You certainly need to have a conversation with the file provider (rather than us) to have them clarify exactly what it is they are sending you rather than have you continue to try to puzzle it out. There are different kinds of files that people call "fixed width" and the kind I was mentioning are basically what you used to find on tape (yay) that when dumped to disk, a "wc -l" would return a record count of 1. Hence my comment about record delimiters. Also note you may be getting a false count if there are packed fields in there that include "0A" as those will be counted as record delimiters by "wc" as it is rather brain dead. I also seem to remember issues with fixed-width files with record delimiters being mangled during file transfers because some transfer mechanisms can remove all of those "unnecessary" trailing spaces at the end of each record.

This article might help with the concepts. Stumbled across it searching this morning.

Bottom line is still you needing to get all of the details you need to process the file from whomever is providing the file to you. Rather than random strangers on the Internet.

Hi Craig,
I wrote a script in phython to loop through the file and print length of each line and total file count. I found that each record length is variable. Hence I use a script to pad each record to make it fixed length . Then I use my copybook to read it and was successful. So I am marking this issue resolved.

Okay, fair enough, but to me that falls squarely into the "Workaround" camp and should still be resolved in a more proper fashion. YMMV, of course.

I agree Craig. I changed it to Workaround.

Your arithmetic will be out if there are any COMP fields.

Hi Ray,

There are comp fields and after reading it through CFF stage, I compared the data with the existing one(SAS program) and all the fields are matching...