DSXchange: DataStage and IBM Websphere Data Integration Forum
View next topic
View previous topic
Add To Favorites
This topic is not resolved, but there is a WORKAROUND.
Author Message
joycerecacho
Participant



Joined: 26 Aug 2008
Posts: 288

Points: 5341

Post Posted: Wed Feb 06, 2019 1:27 am Reply with quote    Back to top    

DataStage® Release: 11x
Job Type: Parallel
OS: Unix
Hi everybody.

I am using 'File Connector' object to read txt files from HDFS. but some of them are not delimited, fields are positionals.

Example:
JoyceRecacho34 years oldBrasil

In other words, it would be like:

Field1: Row[1,12]
Field2: Row[13,12]
Field1: Row[25,6]

=======
I noticed that using the File Format like 'Implicit', some of the rows are split correctly however the majority of rows are disaligned, being droped.

Has anybody already passed through this issue?

As a workaround I read the file as it was a delimited one, and cut the fields like:
Field1: Row[1,12]
Field2: Row[13,12]
Field1: Row[25,6]

It worked fine, but the customer prefer the fields delimited conforming its definition at the sheet 'Column'.
Is there any way?
It reminds me the 'Sequential File' that we just inform "Record Length = Fixed" and "Delimiter = None" and it works.

Thanks in advance.

Best regards,

_________________
Joyce A. Recacho
São Paulo/SP
Brazil
UCDI



Group memberships:
Premium Members

Joined: 21 Mar 2016
Posts: 345

Points: 3470

Post Posted: Wed Feb 06, 2019 8:08 am Reply with quote    Back to top    

for fixed positions, left and right should do it. you can string these together eg
f1 = left(in, 12)
f2 = left(right(in,12),12)
etc
Rate this response:  
Not yet rated
joycerecacho
Participant



Joined: 26 Aug 2008
Posts: 288

Points: 5341

Post Posted: Wed Feb 06, 2019 9:52 am Reply with quote    Back to top    

Did you mean... in a transformer, as I did?
Cause I was thinking about solve it through File Connector only.

_________________
Joyce A. Recacho
São Paulo/SP
Brazil
Rate this response:  
Not yet rated
ray.wurlod

Premium Poster
Participant

Group memberships:
Premium Members, Inner Circle, Australia Usergroup, Server to Parallel Transition Group

Joined: 23 Oct 2002
Posts: 54479
Location: Sydney, Australia
Points: 295436

Post Posted: Wed Feb 06, 2019 9:24 pm Reply with quote    Back to top    

Try using a Sequential File stage, specify Fixed Width when importing the metadata, and specify the column widths during that process. Or edit the table definition details, and set the column Width p ...

_________________
RXP Services Ltd
Melbourne | Canberra | Sydney | Hong Kong | Hobart | Brisbane
Rate this response:  
Not yet rated
joycerecacho
Participant



Joined: 26 Aug 2008
Posts: 288

Points: 5341

Post Posted: Thu Feb 07, 2019 1:14 am Reply with quote    Back to top    

hunfff
:/

I can´t read the full answear.
Sorry Ray ....

_________________
Joyce A. Recacho
São Paulo/SP
Brazil
Rate this response:  
Not yet rated
chulett

Premium Poster


since January 2006

Group memberships:
Premium Members, Inner Circle, Server to Parallel Transition Group

Joined: 12 Nov 2002
Posts: 42896
Location: Denver, CO
Points: 221168

Post Posted: Thu Feb 07, 2019 2:10 am Reply with quote    Back to top    

I know he'd do this since it's only three more words. Here's what is cut off:

"roperty for each column."

_________________
-craig

The Old Ones were, the Old Ones are, and the Old Ones shall be. Not in the spaces we know, but between them. They walk serene and primal, undimensioned and to us unseen.
Rate this response:  
Not yet rated
joycerecacho
Participant



Joined: 26 Aug 2008
Posts: 288

Points: 5341

Post Posted: Thu Feb 07, 2019 2:28 am Reply with quote    Back to top    

Oh nice, Ray.

I am still having problems.

Actually I read that you can configure the properties of the 'File Connector' the way it can understand the fixed width of the fields:

* File format: implicit
* Record definition: Delimited String
* Definition source: <name>:<definition>[length]
Between each field definition I tried ' ' or nothing ''
eg:
FieldName1:Char[1]FieldName2:Char[1]
FieldName1:Char[1] FieldName2:Char[1]

I might informed something wrong cause it didnt work.

Any suggestion?

_________________
Joyce A. Recacho
São Paulo/SP
Brazil
Rate this response:  
Not yet rated
joycerecacho
Participant



Joined: 26 Aug 2008
Posts: 288

Points: 5341

Post Posted: Thu Feb 07, 2019 11:52 pm Reply with quote    Back to top    

Actually I couldn't find any site where people say exactly how to fill the field "Definition Source" when you choose File Format = "Implicit".

Nobody knows how it works, that´s the truth.

_________________
Joyce A. Recacho
São Paulo/SP
Brazil
Rate this response:  
Not yet rated
chulett

Premium Poster


since January 2006

Group memberships:
Premium Members, Inner Circle, Server to Parallel Transition Group

Joined: 12 Nov 2002
Posts: 42896
Location: Denver, CO
Points: 221168

Post Posted: Fri Feb 08, 2019 2:46 am Reply with quote    Back to top    

Guessing IBM would know... <crosses fingers>

_________________
-craig

The Old Ones were, the Old Ones are, and the Old Ones shall be. Not in the spaces we know, but between them. They walk serene and primal, undimensioned and to us unseen.
Rate this response:  
Not yet rated
ray.wurlod

Premium Poster
Participant

Group memberships:
Premium Members, Inner Circle, Australia Usergroup, Server to Parallel Transition Group

Joined: 23 Oct 2002
Posts: 54479
Location: Sydney, Australia
Points: 295436

Post Posted: Sun Feb 10, 2019 10:38 pm Reply with quote    Back to top    

Try importing the Sequential File definition (not via the File Connector) then proceed as I earlier advised. You may need to share the result into the common metadata repository from DataStage Designer, but that's not difficult.

_________________
RXP Services Ltd
Melbourne | Canberra | Sydney | Hong Kong | Hobart | Brisbane

Last edited by ray.wurlod on Wed Feb 13, 2019 1:33 pm; edited 1 time in total
Rate this response:  
Not yet rated
joycerecacho
Participant



Joined: 26 Aug 2008
Posts: 288

Points: 5341

Post Posted: Sun Feb 10, 2019 11:06 pm Reply with quote    Back to top    

ray.wurlod wrote:
Try importing the Sequential File definition (not via the File Connector) then proceed as I earlier advised. You may need to share the result into the common metadata repository from DataStage Designe ...


Ray,
Could you please share the full answer?
I can´t read the whole post.

Thanks in advance.

_________________
Joyce A. Recacho
São Paulo/SP
Brazil
Rate this response:  
Not yet rated
joycerecacho
Participant



Joined: 26 Aug 2008
Posts: 288

Points: 5341

Post Posted: Wed Feb 20, 2019 2:18 am Reply with quote    Back to top    

Hi Ray,
Could you help me?

Tks in advance.

_________________
Joyce A. Recacho
São Paulo/SP
Brazil
Rate this response:  
Not yet rated
chulett

Premium Poster


since January 2006

Group memberships:
Premium Members, Inner Circle, Server to Parallel Transition Group

Joined: 12 Nov 2002
Posts: 42896
Location: Denver, CO
Points: 221168

Post Posted: Wed Feb 20, 2019 7:49 pm Reply with quote    Back to top    

You've got the important bits. However...

"r, but that's not difficult."

_________________
-craig

The Old Ones were, the Old Ones are, and the Old Ones shall be. Not in the spaces we know, but between them. They walk serene and primal, undimensioned and to us unseen.
Rate this response:  
Not yet rated
joycerecacho
Participant



Joined: 26 Aug 2008
Posts: 288

Points: 5341

Post Posted: Wed Feb 27, 2019 7:55 am Reply with quote    Back to top    

It didnt work here, so I let the solution like initially: I split the positional file cutting columns inside transformer.
Unfortunately, just a workaround for now.

_________________
Joyce A. Recacho
São Paulo/SP
Brazil
Rate this response:  
Not yet rated
Display posts from previous:       

Add To Favorites
View next topic
View previous topic
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum



Powered by phpBB © 2001, 2002 phpBB Group
Theme & Graphics by Daz :: Portal by Smartor
All times are GMT - 6 Hours