DSXchange: DataStage and IBM Websphere Data Integration Forum
View next topic
View previous topic
Add To Favorites
Author Message
qt_ky



Group memberships:
Premium Members

Joined: 03 Aug 2011
Posts: 2855
Location: USA
Points: 21702

Post Posted: Mon Sep 18, 2017 1:00 pm Reply with quote    Back to top    

DataStage® Release: 11x
Job Type: Parallel
OS: Unix
Additional info: 11.5.0.2 + SP1
I am running IA on AIX with ODBC pointed at SQL Server.

Some columns have TEXT data type (like a CLOB) which are variable width up to 2 GB in size. Customer was hoping IA would do data classification on these columns and detect sensitive data if present, which is an advertised feature of the tool. Does it not work on large objects? Is there a workaround?

So far I have found that IA reports such columns as data type "STRING" with length "--" and status as "Error." The option to "Include columns with length over 512 characters" is enabled. The corresponding DataStage job log contains a fatal message about having the array size set to 1 for LOB columns.

_________________
Choose a job you love, and you will never have to work a day in your life. - Confucius
truenorth
Participant



Joined: 18 Jan 2010
Posts: 137
Location: San Antonio
Points: 913

Post Posted: Mon Sep 18, 2017 1:34 pm Reply with quote    Back to top    

We are on 11.3 and have the same issue. Our IT folks have tried different drivers, JDBC, etc, to no avail. We have a PMR open and so far, IBM has not helped resolve the issue. We're hoping we would have better luck when we install 11.5 at yearend but I see you're on 11.5.

The workaround we will attempt is cast the data element in Datastage and write to another table.

_________________
Todd Ramirez
Sr Consultant, Data Quality
San Antonio TX
Rate this response:  
Not yet rated
qt_ky



Group memberships:
Premium Members

Joined: 03 Aug 2011
Posts: 2855
Location: USA
Points: 21702

Post Posted: Tue Sep 19, 2017 7:18 am Reply with quote    Back to top    

I am also leaning towards the same type of workaround.

I am confused about the setting to include columns with length over 512 characters. The seems to suggest that it could read a large column, or at least read the first 512 characters of it. ??

_________________
Choose a job you love, and you will never have to work a day in your life. - Confucius
Rate this response:  
Not yet rated
truenorth
Participant



Joined: 18 Jan 2010
Posts: 137
Location: San Antonio
Points: 913

Post Posted: Tue Sep 19, 2017 9:57 am Reply with quote    Back to top    

Me, too. I made sure it is turned on, but it did not make a difference.

_________________
Todd Ramirez
Sr Consultant, Data Quality
San Antonio TX
Rate this response:  
Not yet rated
Display posts from previous:       

Add To Favorites
View next topic
View previous topic
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum



Powered by phpBB © 2001, 2002 phpBB Group
Theme & Graphics by Daz :: Portal by Smartor
All times are GMT - 6 Hours