IA-Column Analysis-Incorrect inferred results for a column

dwh_user · Posted: Thu Aug 29, 2019 12:03 am

Hi Experts,

I have encountered an issue while profiling flat files in information Analyzer v11.5. A field which is more than 1000 characters is being Inferred as STRING 255.
The analysis setting to “Include columns with length over 512 characters” is enabled. Also, I couldn’t edit the defined metadata for this field as part of the import process to make the length longer than 255.

I have found some IBM documentation related to this:
https://www-01.ibm.com/support/docview.wss?uid=swg21698836
https://www-01.ibm.com/support/docview.wss?uid=swg1JR56739

Is this the default behaviour of IA v11.5, where a STRING longer than 255 characters is Inferred as STRING 255? Or is there any setting that has to be changed where the inferred results are accurate for strings longer than 255 characters?

Thanks in advance

chulett · Posted: Sat Aug 31, 2019 10:06 am

FYI - site has been a bit cranky lately, finally able to get in and reply. Also fixed your links so they work now. Neither seems to apply here and the second one notes "Issue is fixed with APAR JR56739 and is delivered with Governance Rollup5" in 2016. If you haven't already, open a case with your official support provider to check.

Never used this tool but going to make an educated guess based on general sequential file metadata handling across the suite. From what I recall - if what it sees during the import looks like a small string, it will treat it as a CHAR rather than a VARCHAR and you'll get the STRING(255) you are seeing... and it only looks at a handful of records to decide what to do. Are your longer values in ALL records or only some records? If the latter, I'd import one where they were right up front and if you don't have one like that, engineer one.

As a general statement, I'm not sure why you can't "edit the defined metadata" if it's not correct. We had to do that all the time when the import didn't get it quite right, be it flat files or Oracle tables or whatever. A people will always be better at inferring metadata than some silly program. Wink

stuartjvnorton · Posted: Sat Dec 28, 2019 1:41 pm

Out of curiosity, what is the maximum length of the actual data in your field?

dwh_user · Posted: Sat Dec 28, 2019 7:54 pm

The maximum length of the string in the field was around 3000.

ray.wurlod · Posted: Tue Dec 31, 2019 10:05 pm

All strings in delimited text files are reported as length 255, unless you change that fact in QETXT.INI. And that, curiously, limits what can be inferred. I agree that it shouldn't.