[File Connector] Not delimited Files
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 298
- Joined: Tue Aug 26, 2008 12:17 pm
[File Connector] Not delimited Files
Hi everybody.
I am using 'File Connector' object to read txt files from HDFS. but some of them are not delimited, fields are positionals.
Example:
JoyceRecacho34 years oldBrasil
In other words, it would be like:
Field1: Row[1,12]
Field2: Row[13,12]
Field1: Row[25,6]
=======
I noticed that using the File Format like 'Implicit', some of the rows are split correctly however the majority of rows are disaligned, being droped.
Has anybody already passed through this issue?
As a workaround I read the file as it was a delimited one, and cut the fields like:
Field1: Row[1,12]
Field2: Row[13,12]
Field1: Row[25,6]
It worked fine, but the customer prefer the fields delimited conforming its definition at the sheet 'Column'.
Is there any way?
It reminds me the 'Sequential File' that we just inform "Record Length = Fixed" and "Delimiter = None" and it works.
Thanks in advance.
Best regards,
I am using 'File Connector' object to read txt files from HDFS. but some of them are not delimited, fields are positionals.
Example:
JoyceRecacho34 years oldBrasil
In other words, it would be like:
Field1: Row[1,12]
Field2: Row[13,12]
Field1: Row[25,6]
=======
I noticed that using the File Format like 'Implicit', some of the rows are split correctly however the majority of rows are disaligned, being droped.
Has anybody already passed through this issue?
As a workaround I read the file as it was a delimited one, and cut the fields like:
Field1: Row[1,12]
Field2: Row[13,12]
Field1: Row[25,6]
It worked fine, but the customer prefer the fields delimited conforming its definition at the sheet 'Column'.
Is there any way?
It reminds me the 'Sequential File' that we just inform "Record Length = Fixed" and "Delimiter = None" and it works.
Thanks in advance.
Best regards,
Joyce A. Recacho
São Paulo/SP
Brazil
São Paulo/SP
Brazil
-
- Participant
- Posts: 298
- Joined: Tue Aug 26, 2008 12:17 pm
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Try using a Sequential File stage, specify Fixed Width when importing the metadata, and specify the column widths during that process. Or edit the table definition details, and set the column Width property for each column.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
-
- Participant
- Posts: 298
- Joined: Tue Aug 26, 2008 12:17 pm
Oh nice, Ray.
I am still having problems.
Actually I read that you can configure the properties of the 'File Connector' the way it can understand the fixed width of the fields:
* File format: implicit
* Record definition: Delimited String
* Definition source: <name>:<definition>[length]
Between each field definition I tried ' ' or nothing ''
eg:
FieldName1:Char[1]FieldName2:Char[1]
FieldName1:Char[1] FieldName2:Char[1]
I might informed something wrong cause it didnt work.
Any suggestion?
I am still having problems.
Actually I read that you can configure the properties of the 'File Connector' the way it can understand the fixed width of the fields:
* File format: implicit
* Record definition: Delimited String
* Definition source: <name>:<definition>[length]
Between each field definition I tried ' ' or nothing ''
eg:
FieldName1:Char[1]FieldName2:Char[1]
FieldName1:Char[1] FieldName2:Char[1]
I might informed something wrong cause it didnt work.
Any suggestion?
Joyce A. Recacho
São Paulo/SP
Brazil
São Paulo/SP
Brazil
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Try importing the Sequential File definition (not via the File Connector) then proceed as I earlier advised. You may need to share the result into the common metadata repository from DataStage Designer, but that's not difficult.
Last edited by ray.wurlod on Wed Feb 13, 2019 1:33 pm, edited 1 time in total.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
-
- Participant
- Posts: 298
- Joined: Tue Aug 26, 2008 12:17 pm
-
- Participant
- Posts: 298
- Joined: Tue Aug 26, 2008 12:17 pm