DSXchange: DataStage and IBM Websphere Data Integration Forum
View next topic
View previous topic
Add To Favorites
This topic is not resolved, but there is a WORKAROUND.
Author Message
joycerecacho
Participant



Joined: 26 Aug 2008
Posts: 292

Points: 5411

Post Posted: Wed Feb 06, 2019 1:27 am Reply with quote    Back to top    

DataStage® Release: 11x
Job Type: Parallel
OS: Unix
Hi everybody.

I am using 'File Connector' object to read txt files from HDFS. but some of them are not delimited, fields are positionals.

Example:
JoyceRecacho34 years oldBrasil

In other words, it would be like:

Field1: Row[1,12]
Field2: Row[13,12]
Field1: Row[25,6]

=======
I noticed that using the File Format like 'Implicit', some of the rows are split correctly however the majority of rows are disaligned, being droped.

Has anybody already passed through this issue?

As a workaround I read the file as it was a delimited one, and cut the fields like:
Field1: Row[1,12]
Field2: Row[13,12]
Field1: Row[25,6]

It worked fine, but the customer prefer the fields delimited conforming its definition at the sheet 'Column'.
Is there any way?
It reminds me the 'Sequential File' that we just inform "Record Length = Fixed" and "Delimiter = None" and it works.

Thanks in advance.

Best regards,

_________________
Joyce A. Recacho
São Paulo/SP
Brazil
UCDI



Group memberships:
Premium Members

Joined: 21 Mar 2016
Posts: 373

Points: 3853

Post Posted: Wed Feb 06, 2019 8:08 am Reply with quote    Back to top    

for fixed positions, left and right should do it. you can string these together eg
f1 = left(in, 12)
f2 = left(right(in,12),12)
etc
Rate this response:  
Not yet rated
joycerecacho
Participant



Joined: 26 Aug 2008
Posts: 292

Points: 5411

Post Posted: Wed Feb 06, 2019 9:52 am Reply with quote    Back to top    

Did you mean... in a transformer, as I did?
Cause I was thinking about solve it through File Connector only.

_________________
Joyce A. Recacho
São Paulo/SP
Brazil
Rate this response:  
Not yet rated
ray.wurlod

Premium Poster
Participant

Group memberships:
Premium Members, Inner Circle, Australia Usergroup, Server to Parallel Transition Group

Joined: 23 Oct 2002
Posts: 54524
Location: Sydney, Australia
Points: 295662

Post Posted: Wed Feb 06, 2019 9:24 pm Reply with quote    Back to top    

Try using a Sequential File stage, specify Fixed Width when importing the metadata, and specify the column widths during that process. Or edit the table definition details, and set the column Width p ...

_________________
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Rate this response:  
Not yet rated
joycerecacho
Participant



Joined: 26 Aug 2008
Posts: 292

Points: 5411

Post Posted: Thu Feb 07, 2019 1:14 am Reply with quote    Back to top    

hunfff
:/

I can´t read the full answear.
Sorry Ray ....

_________________
Joyce A. Recacho
São Paulo/SP
Brazil
Rate this response:  
Not yet rated
chulett

Premium Poster


since January 2006

Group memberships:
Premium Members, Inner Circle, Server to Parallel Transition Group

Joined: 12 Nov 2002
Posts: 42984
Location: Denver, CO
Points: 221720

Post Posted: Thu Feb 07, 2019 2:10 am Reply with quote    Back to top    

I know he'd do this since it's only three more words. Here's what is cut off:

"roperty for each column."

_________________
-craig

Help I'm steppin' into the twilight zone, place is a madhouse, feels like being cold
My beacon's been moved under moon and star, where am I to go now that I've gone too far?
Rate this response:  
Not yet rated
joycerecacho
Participant



Joined: 26 Aug 2008
Posts: 292

Points: 5411

Post Posted: Thu Feb 07, 2019 2:28 am Reply with quote    Back to top    

Oh nice, Ray.

I am still having problems.

Actually I read that you can configure the properties of the 'File Connector' the way it can understand the fixed width of the fields:

* File format: implicit
* Record definition: Delimited String
* Definition source: <name>:<definition>[length]
Between each field definition I tried ' ' or nothing ''
eg:
FieldName1:Char[1]FieldName2:Char[1]
FieldName1:Char[1] FieldName2:Char[1]

I might informed something wrong cause it didnt work.

Any suggestion?

_________________
Joyce A. Recacho
São Paulo/SP
Brazil
Rate this response:  
Not yet rated
joycerecacho
Participant



Joined: 26 Aug 2008
Posts: 292

Points: 5411

Post Posted: Thu Feb 07, 2019 11:52 pm Reply with quote    Back to top    

Actually I couldn't find any site where people say exactly how to fill the field "Definition Source" when you choose File Format = "Implicit".

Nobody knows how it works, that´s the truth.

_________________
Joyce A. Recacho
São Paulo/SP
Brazil
Rate this response:  
Not yet rated
chulett

Premium Poster


since January 2006

Group memberships:
Premium Members, Inner Circle, Server to Parallel Transition Group

Joined: 12 Nov 2002
Posts: 42984
Location: Denver, CO
Points: 221720

Post Posted: Fri Feb 08, 2019 2:46 am Reply with quote    Back to top    

Guessing IBM would know... <crosses fingers>

_________________
-craig

Help I'm steppin' into the twilight zone, place is a madhouse, feels like being cold
My beacon's been moved under moon and star, where am I to go now that I've gone too far?
Rate this response:  
Not yet rated
ray.wurlod

Premium Poster
Participant

Group memberships:
Premium Members, Inner Circle, Australia Usergroup, Server to Parallel Transition Group

Joined: 23 Oct 2002
Posts: 54524
Location: Sydney, Australia
Points: 295662

Post Posted: Sun Feb 10, 2019 10:38 pm Reply with quote    Back to top    

Try importing the Sequential File definition (not via the File Connector) then proceed as I earlier advised. You may need to share the result into the common metadata repository from DataStage Designer, but that's not difficult.

_________________
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.

Last edited by ray.wurlod on Wed Feb 13, 2019 1:33 pm; edited 1 time in total
Rate this response:  
Not yet rated
joycerecacho
Participant



Joined: 26 Aug 2008
Posts: 292

Points: 5411

Post Posted: Sun Feb 10, 2019 11:06 pm Reply with quote    Back to top    

ray.wurlod wrote:
Try importing the Sequential File definition (not via the File Connector) then proceed as I earlier advised. You may need to share the result into the common metadata repository from DataStage Designe ...


Ray,
Could you please share the full answer?
I can´t read the whole post.

Thanks in advance.

_________________
Joyce A. Recacho
São Paulo/SP
Brazil
Rate this response:  
Not yet rated
joycerecacho
Participant



Joined: 26 Aug 2008
Posts: 292

Points: 5411

Post Posted: Wed Feb 20, 2019 2:18 am Reply with quote    Back to top    

Hi Ray,
Could you help me?

Tks in advance.

_________________
Joyce A. Recacho
São Paulo/SP
Brazil
Rate this response:  
Not yet rated
chulett

Premium Poster


since January 2006

Group memberships:
Premium Members, Inner Circle, Server to Parallel Transition Group

Joined: 12 Nov 2002
Posts: 42984
Location: Denver, CO
Points: 221720

Post Posted: Wed Feb 20, 2019 7:49 pm Reply with quote    Back to top    

You've got the important bits. However...

"r, but that's not difficult."

_________________
-craig

Help I'm steppin' into the twilight zone, place is a madhouse, feels like being cold
My beacon's been moved under moon and star, where am I to go now that I've gone too far?
Rate this response:  
Not yet rated
joycerecacho
Participant



Joined: 26 Aug 2008
Posts: 292

Points: 5411

Post Posted: Wed Feb 27, 2019 7:55 am Reply with quote    Back to top    

It didnt work here, so I let the solution like initially: I split the positional file cutting columns inside transformer.
Unfortunately, just a workaround for now.

_________________
Joyce A. Recacho
São Paulo/SP
Brazil
Rate this response:  
Not yet rated
Display posts from previous:       

Add To Favorites
View next topic
View previous topic
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum



Powered by phpBB © 2001, 2002 phpBB Group
Theme & Graphics by Daz :: Portal by Smartor
All times are GMT - 6 Hours