Special character showing as question marks in table

midmaxds · Post by **midmaxds** » Tue Aug 23, 2016 7:17 am

Hi,
We have a job which extracts records from Oracle table and loads to a DB2 table. No transformations.
The issue is with a NAME field where in Oracle and DB2 it is varchar (80).

one of the names has a non English character such a cross sign, a star etc (My keyboard does not have them, sorry)

So when the name comes to DB2 table these characters are just question marks (??).
After googling a bit we tried giving the unicode option in oracle stage. Now the data is coming as expected, but director has logs saying this could cause lose of data. Please advise on how we can get through this. Thank you.

midmaxds · Post by **midmaxds** » Wed Aug 24, 2016 12:21 pm

Hi,
Any thoughts, please...

UCDI · Post by **UCDI** » Wed Aug 24, 2016 2:18 pm

I think the symbol you see with a ? inside it is the symbol that datastage uses to tell you that there is something there which it does not know how to display.

The loss of data warning tells me that you very likely have a metadata mismatch somewhere that is feeding a unicode enabled string into a non-unicode enabled string which damages the data. Make sure all your links/metadata have unicode on them.

SURA · Post by **SURA** » Wed Aug 24, 2016 6:57 pm

Use oracle tool to export the data into a text file and apply od command.

Try to get better hold of the data and then handle it.

ray.wurlod · Post by **ray.wurlod** » Thu Aug 25, 2016 4:27 pm

Verify with both databases whether the length is specified in BYTES or in CHARACTERS. If BYTES you will likely have problems with longer strings using multi-byte characters.

midmaxds · Post by **midmaxds** » Sun Aug 28, 2016 7:56 pm

Thank you all.
The issue was finally fixed by increasing the number of bytes in target database and changing data type to unicode from varchar in jobs. Thanks again!