DSXchange: DataStage and IBM Websphere Data Integration Forum
View next topic
View previous topic
Add To Favorites
Author Message
jweir
Participant



Joined: 04 Aug 2010
Posts: 131

Points: 1214

Post Posted: Mon Sep 11, 2017 8:37 am Reply with quote    Back to top    

DataStage® Release: 11x
Job Type: Parallel
OS: Unix
Hi all,

Is there a way to use Unicode with RCP?

Thanks in advance.

_________________
Jweir

--- If strength were all, tiger would not fear scorpion.
chulett

Premium Poster


since January 2006

Group memberships:
Premium Members, Inner Circle, Server to Parallel Transition Group

Joined: 12 Nov 2002
Posts: 42107
Location: Denver, CO
Points: 216145

Post Posted: Mon Sep 11, 2017 10:04 am Reply with quote    Back to top    

Have you tried... specifying "ustring" in the schema file? I'm sure there's more to it than that, but that's the first thing that comes to mind.

_________________
-craig

Can't keep my eyes from the circling skies
Tongue tied and twisted just an earth bound misfit, I
Rate this response:  
jweir
Participant



Joined: 04 Aug 2010
Posts: 131

Points: 1214

Post Posted: Mon Sep 11, 2017 10:22 am Reply with quote    Back to top    

I do not use schema files to specify the table definition. My job is a very generic job. It takes data from Sybase to Oracle - a one to one move. The Select statement is generated by a routine by gathering the column names from the Sybase system table. This is done so we can load many tables using only a few jobs, rather than a parallel job for every single table.

_________________
Jweir

--- If strength were all, tiger would not fear scorpion.
Rate this response:  
Mike



Group memberships:
Premium Members

Joined: 03 Mar 2002
Posts: 1010
Location: Omaha, NE
Points: 6513

Post Posted: Mon Sep 11, 2017 11:27 am Reply with quote    Back to top    

RCP isn't going to do any type conversion. If your source is a ustring then it will propagate as a ustring. If it is a string then it will propagate as a string.

If you need to convert between ustring and string, you can use a modify stage with a parameterized specification.

$OSH_PRINT_SCHEMAS will show you what you are working with.

Mike
Rate this response:  
jweir
Participant



Joined: 04 Aug 2010
Posts: 131

Points: 1214

Post Posted: Mon Sep 11, 2017 11:34 am Reply with quote    Back to top    

Ok. So if my job is:

Code:
Sybase stage --> Oracle stage


There is no way to apply Unicode unless I add a Modify stage? And that is with RCP?

_________________
Jweir

--- If strength were all, tiger would not fear scorpion.
Rate this response:  
UCDI



Group memberships:
Premium Members

Joined: 21 Mar 2016
Posts: 250

Points: 2521

Post Posted: Mon Sep 11, 2017 12:10 pm Reply with quote    Back to top    

What exactly is the error?

Assuming its string and not uc in the source, string and not uc in the flow, and at most uc on the target side .... but datastage should be able to write string source to uc target column without any extra code (might warn, but no fail?)

Can you explain exactly where it is failing and what you are trying to do in that region of code?
Rate this response:  
jweir
Participant



Joined: 04 Aug 2010
Posts: 131

Points: 1214

Post Posted: Mon Sep 11, 2017 12:51 pm Reply with quote    Back to top    

It is failing because on of the target columns is VARCHAR(20) and one record is 21 length (it is not 21 in the source). When the data is extracted out of Sybase, it add extra bytes to the data for special characters (it hold telephone data). So the job fails because it cannot insert a record that is 21 length into a 20 length field.

I created a test job and extracted the problem record, and when I applied Unicode, it was not 21 length anymore. It makes it more difficult since I use this "generic" job to load many tables, and pass in the table name and SELECT and INSERT statements from a user-defined IIS routine.

_________________
Jweir

--- If strength were all, tiger would not fear scorpion.
Rate this response:  
Mike



Group memberships:
Premium Members

Joined: 03 Mar 2002
Posts: 1010
Location: Omaha, NE
Points: 6513

Post Posted: Mon Sep 11, 2017 4:26 pm Reply with quote    Back to top    

You could experiment with the NLS Code page at the job and stage levels. See if UTF-16 will help.

Mike
Rate this response:  
JRodriguez



Group memberships:
Premium Members

Joined: 19 Nov 2005
Posts: 397
Location: New York City
Points: 4301

Post Posted: Tue Sep 12, 2017 5:56 am Reply with quote    Back to top    

The Oracle client needs to know the NLS of the target database, just set DS job to go by the NLS of the target database, here is how is done:
-Get the NLS of Oracle database
- Add a NLS_LANG to user defined environmental variables in DS Admin Client. Leave the value empty
- Add the $NLS_LANG as a parameter in the job, ser the default value to the value obtained in step 1

_________________
Julio Rodriguez
ETL Developer by choice

"Sure we have lots of reasons for being rude - But no excuses
Rate this response:  
jweir
Participant



Joined: 04 Aug 2010
Posts: 131

Points: 1214

Post Posted: Tue Sep 12, 2017 2:48 pm Reply with quote    Back to top    

UTF16 did not work. And the problem is before the Oracle client. It is when the data is extracted into IIS, the extra chars are added.

_________________
Jweir

--- If strength were all, tiger would not fear scorpion.
Rate this response:  
cdp



Group memberships:
Premium Members

Joined: 15 Dec 2009
Posts: 111
Location: New Zealand
Points: 1568

Post Posted: Tue Sep 12, 2017 3:46 pm Reply with quote    Back to top    

jweir wrote:
It is failing because on of the target columns is VARCHAR(20) .

VARCHAR(20 byte) or VARCHAR(20 char) ?
https://stackoverflow.com/questions/81448/difference-between-byte-and-char-in-column-datatypes

What happens if alter your target column to VARCHAR(20 char) ?

If you need to data type convert with RCP on, I can only think of the Modify stage.
http://it.toolbox.com/blogs/infosphere/datastage-tutorial-how-to-become-a-modify-stage-zen-master-20368

have fun
Rate this response:  
chulett

Premium Poster


since January 2006

Group memberships:
Premium Members, Inner Circle, Server to Parallel Transition Group

Joined: 12 Nov 2002
Posts: 42107
Location: Denver, CO
Points: 216145

Post Posted: Tue Sep 12, 2017 4:58 pm Reply with quote    Back to top    

It's got to be BYTE semantics or it would be working now, I'd wager.

_________________
-craig

Can't keep my eyes from the circling skies
Tongue tied and twisted just an earth bound misfit, I
Rate this response:  
Display posts from previous:       

Add To Favorites
View next topic
View previous topic
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum



Powered by phpBB © 2001, 2002 phpBB Group
Theme & Graphics by Daz :: Portal by Smartor
All times are GMT - 6 Hours