NLS issues in DataStage with WIN-1252 characters

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
puneeth
Participant
Posts: 2
Joined: Sun Nov 27, 2011 8:27 pm
Location: Hyderabad

NLS issues in DataStage with WIN-1252 characters

Post by puneeth »

Windows character set encoding MS1252 ( 0x80 to 0x9F) that is not compatible with ISO-8859-1 is causing problems when reading from Windows MSSQL Server into datastage and loading Netezza. In this case, the character is "Right Single Quotation Mark".

Specifically , when reading from Datastage, these characters are being displayed as character "?".

Non-printable character "SUB" is being added in the target Netezza table.

I have tried modifying the database stage NLS settings to MS1252 such as windows-1252 or ASCL_MS1252. But none of those options worked.

Please help.
mouthou
Participant
Posts: 208
Joined: Sun Jul 04, 2004 11:57 pm

Re: NLS issues in DataStage with WIN-1252 characters

Post by mouthou »

Couple of things wanted to mention:

1. You changed the DB stage setting to have required NLS. I think it should be changed at the job level to try, rather at a single stage level. Try storing sample output in a temp file to see those ? character and this could confirm if it is an issue with Netezza character set(though you seemed to have referred it as the source side reading issue). In that case, you can get the charset of Netezza as per DBA and apply in the job.

2. Throw in some extra UNIODE feature too in metadata definitions

You would have to keep changing the NLS setting as trial and error till the compatible one is found!
puneeth
Participant
Posts: 2
Joined: Sun Nov 27, 2011 8:27 pm
Location: Hyderabad

NLS issues in DataStage with WIN-1252 characters

Post by puneeth »

Thanks for your suggestion. I tried the below options, but none of them worked:

1) Changing NLS Settings at job level to windows-1252 or ASCL_MS1252.
2) Tried Loading a file and Peek stage. WIN-1252 characters are still being displayed as character "?" in the file and job log.
3) Added Unicode feature in source/target metadata.

Tried different NLS languages at job level, but so far no luck.

Looking for more suggestions on this.
mouthou
Participant
Posts: 208
Joined: Sun Jul 04, 2004 11:57 pm

Re: NLS issues in DataStage with WIN-1252 characters

Post by mouthou »

As mentioned earlier, this is a multiple iteration process till the compatible character encoding is known, unless someone responded who exactly worked on the very same encoding. I had similar situations for latin and french accented characters and it looks sometime to finish that trial.

You could do some research related to WIN-1252 compatibility and it seems "ISO 8859-1" may be tried as the character equivalent. For the debugging sake, you can keep the job to the simple as source-->peek/file stage till a character set is figured out.
Post Reply