DSXchange: DataStage and IBM Websphere Data Integration Forum
View next topic
View previous topic
Add To Favorites
Author Message
sandhya.budhi



Group memberships:
Premium Members

Joined: 15 Nov 2017
Posts: 18

Points: 348

Post Posted: Sun Jun 03, 2018 8:48 pm Reply with quote    Back to top    

DataStage® Release: 11x
Job Type: Parallel
OS: Unix
I am trying to convert the Non printable characters ˙ and › and Spaces using CONVERT() function in Transformer.

My source is providing a file which has non-printable characters. In data stage I am converting the non-printable characters as Spaces.

I am using the below CONVERT function but Spaces is not getting substituted

StageVar155 ---> Char(155,@TRUE)
StageVar255 ---> Char(255,@TRUE)

Code:
Convert(StageVar5: StageVar6,' ', Record)

Defined Record as LongNvarchar in Sequential stage. Also I have set NLS map value as windows-1252 or ASCL_MS1252.

I am not sure why Space is not getting substituted for non-printable characters. I have kept the NLP map as windows-1252 or ASCL_MS1252 because I read from IBM forum to read non-printable characters from sequential stage.

Can you please provide me some suggestions what I am doing wrong.

Input file from Source

Code:
S˙˙˙˙˙˙˙˙˙˙˙˙˙00000000000000


I need to get the output as

Code:
S             00000000000000

_________________
Thanks,
Sandhya
ray.wurlod

Premium Poster
Participant

Group memberships:
Premium Members, Inner Circle, Australia Usergroup, Server to Parallel Transition Group

Joined: 23 Oct 2002
Posts: 54221
Location: Sydney, Australia
Points: 294071

Post Posted: Sun Jun 03, 2018 10:05 pm Reply with quote    Back to top    

Perhaps the code points you are using are not correct. For example, ˙ sometimes represents Char(253), especially in the server job world. I would be very surprised indeed to encounter any representa ...

_________________
RXP Services Ltd
Melbourne | Canberra | Sydney | Hong Kong | Hobart | Brisbane
currently hiring: Canberra, Sydney and Melbourne
Rate this response:  
Not yet rated
chulett

Premium Poster


since January 2006

Group memberships:
Premium Members, Inner Circle, Server to Parallel Transition Group

Joined: 12 Nov 2002
Posts: 42578
Location: Denver, CO
Points: 219144

Post Posted: Mon Jun 04, 2018 6:41 am Reply with quote    Back to top    

How did you determine their values? I'd also suggest you review the syntax for the convert function. You need two spaces in your new.list or only the first character will be replaced. Right ...

_________________
-craig

I know I don't say this enough, but I like when you talk to me. It's much better than when nobody talks to me. Or when people that I don't like will not stop talking to me.
Rate this response:  
Not yet rated
sandhya.budhi



Group memberships:
Premium Members

Joined: 15 Nov 2017
Posts: 18

Points: 348

Post Posted: Mon Jun 04, 2018 7:28 am Reply with quote    Back to top    

I referred the values from the Extended ASCII table.

https://www.ascii-code.com/

I have noticed the values were different in some websites. Is there any way I can validate with any other reference?

_________________
Thanks,
Sandhya
Rate this response:  
Not yet rated
UCDI



Group memberships:
Premium Members

Joined: 21 Mar 2016
Posts: 327

Points: 3307

Post Posted: Mon Jun 04, 2018 8:11 am Reply with quote    Back to top    

validate it with a Visual basic transformer, just convert each byte of the string to its numeric format with | between.
Rate this response:  
Not yet rated
chulett

Premium Poster


since January 2006

Group memberships:
Premium Members, Inner Circle, Server to Parallel Transition Group

Joined: 12 Nov 2002
Posts: 42578
Location: Denver, CO
Points: 219144

Post Posted: Mon Jun 04, 2018 8:54 am Reply with quote    Back to top    

Was worried about that... it's nothing you should be doing visually. If you can, use an "octal dump" on the server to see what those values actually are: Decimal dump od -d <filename> ...

_________________
-craig

I know I don't say this enough, but I like when you talk to me. It's much better than when nobody talks to me. Or when people that I don't like will not stop talking to me.
Rate this response:  
Not yet rated
ray.wurlod

Premium Poster
Participant

Group memberships:
Premium Members, Inner Circle, Australia Usergroup, Server to Parallel Transition Group

Joined: 23 Oct 2002
Posts: 54221
Location: Sydney, Australia
Points: 294071

Post Posted: Tue Jun 05, 2018 11:31 pm Reply with quote    Back to top    

UCDI wrote:
validate it with a Visual basic transformer

You've posted on a number of occasions about a Visual basic transformer. What is this? I currently have projects in versions 8.0.1 through 11.7.0.1 and can find no reference to a stage of this name.

_________________
RXP Services Ltd
Melbourne | Canberra | Sydney | Hong Kong | Hobart | Brisbane
currently hiring: Canberra, Sydney and Melbourne
Rate this response:  
Not yet rated
UCDI



Group memberships:
Premium Members

Joined: 21 Mar 2016
Posts: 327

Points: 3307

Post Posted: Thu Jun 07, 2018 8:17 am Reply with quote    Back to top    

Heh... about that...

Look in your repository under stage types. There are a number of stages in there that are not in the palette. Its under Parallel/Processing/BasicTransformer

Be careful with it. I know for sure that it cannot handle timestamps with microseconds, it loses the extended data. I don't know what else it can damage in this way... I generally pipe only a key & the offending data into it, invoke the routine, and rejoin it back, just for extra safety. I don't know why they won't clean this up or roll the routine calling capability into the standard transform.

Ive used this I think 3, maybe 4 times total in as many years. It feels like a questionable tool, but its too powerful to ignore for a small handful of issues. Remember also that I have been blocked from writing server jobs via a policy, so this may be a work around for oddball string processing that could have been done server side. /shrug. The VB transformer is the compromise I worked out with our powers that be between C (they disliked this a lot) and server jobs (requires too many hoops to be allowed into jobs) for doing the occasional hands-on string processing / cleanup / reformatting/ etc type work.

** I apparently have a hangup about 'visual' and basic. The word visual is only in my own mind, and not anywhere in datastage. Sorry about that.
Rate this response:  
Not yet rated
ray.wurlod

Premium Poster
Participant

Group memberships:
Premium Members, Inner Circle, Australia Usergroup, Server to Parallel Transition Group

Joined: 23 Oct 2002
Posts: 54221
Location: Sydney, Australia
Points: 294071

Post Posted: Thu Jun 07, 2018 3:38 pm Reply with quote    Back to top    

Very good. The BASIC Transformer stage was included in the product to make the transition from server jobs to parallel jobs a bit easier. I use it fairly regularly. I use server jobs a LOT. I use s ...

_________________
RXP Services Ltd
Melbourne | Canberra | Sydney | Hong Kong | Hobart | Brisbane
currently hiring: Canberra, Sydney and Melbourne
Rate this response:  
Not yet rated
rameshrr3



Group memberships:
Premium Members

Joined: 10 May 2004
Posts: 609
Location: BRENTWOOD, TN
Points: 6937

Post Posted: Thu Jun 07, 2018 4:08 pm Reply with quote    Back to top    

Pity that Server Jobs and Basic transformer are nowadays frequently considered 'illicit' workarounds in many places, despite their unique and powerful capabilities, and despite the fact that basic transformer is capable of running on multiple nodes on an SMP system.
Rate this response:  
Not yet rated
Display posts from previous:       

Add To Favorites
View next topic
View previous topic
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum



Powered by phpBB © 2001, 2002 phpBB Group
Theme & Graphics by Daz :: Portal by Smartor
All times are GMT - 6 Hours