Conversion from the UTF-8 character set to UTF-16LE warning

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
asorrell
Posts: 1707
Joined: Fri Apr 04, 2003 2:00 pm
Location: Colleyville, Texas

Re: Conversion from the UTF-8 character set to UTF-16LE warn

Post by asorrell »

anandsivan wrote: We have tried with all options...
Could you please list what you've tried?
Andy Sorrell
Certified DataStage Consultant
IBM Analytics Champion 2009 - 2020
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Have you tried setting NLS mapping to "UTF-16 Little Endian" in DataStage?
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Ah... was wondering what the "LE" part of "UTF-16LE" stood for.
-craig

"You can never have too many knives" -- Logan Nine Fingers
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

At my current engagement we run UTF-16BE - so I guess the OP is on an Intel-chipped box.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
anandsivan
Participant
Posts: 8
Joined: Wed Sep 21, 2011 4:27 am
Location: Karnataka

Post by anandsivan »

ray.wurlod wrote:Have you tried setting NLS mapping to "UTF-16 Little Endian" in DataStage? ...
Yes ... i did this ..but in the job we have date fields if i change UTF8 to UTF-16LE job is aborting because of data fields not converting to UTF-16.
2) when i change to UTF-16LE job is aborted with below warning:

Output Schema does not match data from SELECT statement SQL statement: SELECT TOP 10 ITEM_NBR ,BASE_DIV_NBR ,MDS_FAM_ID , COUNTRY_CODE FROM WW_CORE_DIM_VM.ITEM_DIM WHERE BASE_DIV_NBR IN (1,18) ORDER BY BASE_DIV_NBR (CC_TeraParallelTransporter::initiate, file CC_TeraParallelTransporter.cpp, line 1,295)

In the teradata table :
COUNTRY_CODE CHAR(2) CHARACTER SET LATIN NOT CASESPECIFIC COMPRESS ('AR','BR','CA','CN','GB','IN','JP','K1','MX','US'),

Because of the country_code only i am getting this warning if i remove country_code job is completing successfully with out warning.
anandsivan
Participant
Posts: 8
Joined: Wed Sep 21, 2011 4:27 am
Location: Karnataka

Re: Conversion from the UTF-8 character set to UTF-16LE warn

Post by anandsivan »

Andy,
I have tried below activity's
1) In the datastage i have changed NLS UTF-8 to UTF-16LE
2) Project default map : ISO-8859-1
Transaction mode: ANSI
Client character set : ASCII
Automap characterset encoding: No
NLS map name: ISO-8859-1
3)CC_TERA_USE_ISO88591_ASCII_MAP= 1 at the Project level settings in the Administrator
asorrell
Posts: 1707
Joined: Fri Apr 04, 2003 2:00 pm
Location: Colleyville, Texas

Post by asorrell »

Is the Country Code the ONLY field that is marked character set Latin?
Andy Sorrell
Certified DataStage Consultant
IBM Analytics Champion 2009 - 2020
anandsivan
Participant
Posts: 8
Joined: Wed Sep 21, 2011 4:27 am
Location: Karnataka

Post by anandsivan »

Yes Andy. only the Country Code field that is marked character set Latin.
Post Reply