DSXchange: DataStage and IBM Websphere Data Integration Forum
View next topic
View previous topic
Add To Favorites
Author Message
rumu
Participant



Joined: 06 Jun 2005
Posts: 282

Points: 2830

Post Posted: Fri Dec 14, 2018 8:36 am Reply with quote    Back to top    

DataStage® Release: 11x
Job Type: Parallel
OS: Unix
Hi All,

I am reading a cobol EBCDIC file using CFF stage and loading it into Sequential file stage. There are 2 fields defined in CFF as PICX(2) and PIC X(1) which is in Record Layout shown as CHARCTER 2 and CHARACTER 1 respectively.

I directly mapped those 2 fields to sequential file stage using datatype Char(2) and CHAR(1).
Some data are shown in the datastage file viewer for the field with 2 charadters as

Code:
?|


The second character is | like but I can not copy it as when I am pasting only ? is pasted.
I used String to Raw function to display it and I got the following

Code:
{1a 18}


Are these CAN and Linefeed in HEX? How do I remove them ? When I see it in Unix, it shows noting....

The column has 2 distinct Values when I putput in Unix

RE and blank.

I use dthe following commnd to deisplay in hexdump:


Code:
-bash-4.2$ cat RDTDP.txt|cut -d'|' -f1|sort|uniq|hexdump
0000000 181a 1a0a 0a1a 4552 000a
0000009


How can I convert these foreign characters to space?

_________________
Rumu
IT Consultant
ray.wurlod

Premium Poster
Participant

Group memberships:
Premium Members, Inner Circle, Australia Usergroup, Server to Parallel Transition Group

Joined: 23 Oct 2002
Posts: 54501
Location: Sydney, Australia
Points: 295558

Post Posted: Mon Dec 17, 2018 1:06 am Reply with quote    Back to top    

What character set are you using? Could these be double-byte representations of Unicode characters?

_________________
RXP Services Ltd
Melbourne | Canberra | Sydney | Hong Kong | Hobart | Brisbane
Rate this response:  
Not yet rated
rumu
Participant



Joined: 06 Jun 2005
Posts: 282

Points: 2830

Post Posted: Mon Dec 17, 2018 6:47 am Reply with quote    Back to top    

Hi Ray,

The NLS map is set to Project default(UTF-8).

I used following derivation in the transformer and those characters were not seen.

Quote:
Trim(Trim(DSLink3.RDT_ADDL_SEG_KEY_PROD,char(24)),char(26))


I used 24 as Dec representation for hex 18 and 26 is Dec representation for hex 1A.
Is that approach ok?

_________________
Rumu
IT Consultant
Rate this response:  
Not yet rated
ray.wurlod

Premium Poster
Participant

Group memberships:
Premium Members, Inner Circle, Australia Usergroup, Server to Parallel Transition Group

Joined: 23 Oct 2002
Posts: 54501
Location: Sydney, Australia
Points: 295558

Post Posted: Tue Dec 18, 2018 12:46 am Reply with quote    Back to top    

Who knows? You've condemned what might be valid characters to be "junk". I'd examine that assumption pretty closely.

_________________
RXP Services Ltd
Melbourne | Canberra | Sydney | Hong Kong | Hobart | Brisbane
Rate this response:  
Not yet rated
rumu
Participant



Joined: 06 Jun 2005
Posts: 282

Points: 2830

Post Posted: Tue Dec 18, 2018 8:38 am Reply with quote    Back to top    

Hi Ray,

I used StringToRaw function to to check the values. How can I identify whether it is a double byte character?

_________________
Rumu
IT Consultant
Rate this response:  
Not yet rated
Display posts from previous:       

Add To Favorites
View next topic
View previous topic
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum



Powered by phpBB © 2001, 2002 phpBB Group
Theme & Graphics by Daz :: Portal by Smartor
All times are GMT - 6 Hours