DSXchange: DataStage and IBM Websphere Data Integration Forum
View next topic
View previous topic
Add To Favorites
Author Message
dwh_user



Group memberships:
Premium Members

Joined: 18 Oct 2018
Posts: 13
Location: Sydney
Points: 163

Post Posted: Thu Aug 29, 2019 12:03 am Reply with quote    Back to top    

DataStage® Release: 11x
Job Type: Parallel
OS: Windows
Hi Experts,

I have encountered an issue while profiling flat files in information Analyzer v11.5. A field which is more than 1000 characters is being Inferred as STRING 255.
The analysis setting to “Include columns with length over 512 characters” is enabled. Also, I couldn’t edit the defined metadata for this field as part of the import process to make the length longer than 255.

I have found some IBM documentation related to this:
https://www-01.ibm.com/support/docview.wss?uid=swg21698836
https://www-01.ibm.com/support/docview.wss?uid=swg1JR56739

Is this the default behaviour of IA v11.5, where a STRING longer than 255 characters is Inferred as STRING 255? Or is there any setting that has to be changed where the inferred results are accurate for strings longer than 255 characters?

Thanks in advance
chulett

Premium Poster


since January 2006

Group memberships:
Premium Members, Inner Circle, Server to Parallel Transition Group

Joined: 12 Nov 2002
Posts: 43035
Location: Denver, CO
Points: 222143

Post Posted: Sat Aug 31, 2019 10:06 am Reply with quote    Back to top    

FYI - site has been a bit cranky lately, finally able to get in and reply. Also fixed your links so they work now. Neither seems to apply here and the second one notes "Issue is fixed with APAR JR56739 and is delivered with Governance Rollup5" in 2016. If you haven't already, open a case with your official support provider to check.

Never used this tool but going to make an educated guess based on general sequential file metadata handling across the suite. From what I recall - if what it sees during the import looks like a small string, it will treat it as a CHAR rather than a VARCHAR and you'll get the STRING(255) you are seeing... and it only looks at a handful of records to decide what to do. Are your longer values in ALL records or only some records? If the latter, I'd import one where they were right up front and if you don't have one like that, engineer one.

As a general statement, I'm not sure why you can't "edit the defined metadata" if it's not correct. We had to do that all the time when the import didn't get it quite right, be it flat files or Oracle tables or whatever. A people will always be better at inferring metadata than some silly program. Wink

_________________
-craig

I've all my wisdom teeth, Two up top, two beneath
And yet I'll recognize my mouth says things that aren't so wise
Rate this response:  
Not yet rated
Display posts from previous:       

Add To Favorites
View next topic
View previous topic
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum



Powered by phpBB © 2001, 2002 phpBB Group
Theme & Graphics by Daz :: Portal by Smartor
All times are GMT - 6 Hours