DSXchange: DataStage and IBM Websphere Data Integration Forum
View next topic
View previous topic
Add To Favorites
Author Message
dwh_user



Group memberships:
Premium Members

Joined: 18 Oct 2018
Posts: 14
Location: Sydney
Points: 168

Post Posted: Thu Aug 29, 2019 12:03 am Reply with quote    Back to top    

DataStage® Release: 11x
Job Type: Parallel
OS: Windows
Hi Experts,

I have encountered an issue while profiling flat files in information Analyzer v11.5. A field which is more than 1000 characters is being Inferred as STRING 255.
The analysis setting to “Include columns with length over 512 characters” is enabled. Also, I couldn’t edit the defined metadata for this field as part of the import process to make the length longer than 255.

I have found some IBM documentation related to this:
https://www-01.ibm.com/support/docview.wss?uid=swg21698836
https://www-01.ibm.com/support/docview.wss?uid=swg1JR56739

Is this the default behaviour of IA v11.5, where a STRING longer than 255 characters is Inferred as STRING 255? Or is there any setting that has to be changed where the inferred results are accurate for strings longer than 255 characters?

Thanks in advance
chulett

Premium Poster


since January 2006

Group memberships:
Premium Members, Inner Circle, Server to Parallel Transition Group

Joined: 12 Nov 2002
Posts: 43085
Location: Denver, CO
Points: 222463

Post Posted: Sat Aug 31, 2019 10:06 am Reply with quote    Back to top    

FYI - site has been a bit cranky lately, finally able to get in and reply. Also fixed your links so they work now. Neither seems to apply here and the second one notes "Issue is fixed with APAR JR56739 and is delivered with Governance Rollup5" in 2016. If you haven't already, open a case with your official support provider to check.

Never used this tool but going to make an educated guess based on general sequential file metadata handling across the suite. From what I recall - if what it sees during the import looks like a small string, it will treat it as a CHAR rather than a VARCHAR and you'll get the STRING(255) you are seeing... and it only looks at a handful of records to decide what to do. Are your longer values in ALL records or only some records? If the latter, I'd import one where they were right up front and if you don't have one like that, engineer one.

As a general statement, I'm not sure why you can't "edit the defined metadata" if it's not correct. We had to do that all the time when the import didn't get it quite right, be it flat files or Oracle tables or whatever. A people will always be better at inferring metadata than some silly program. Wink

_________________
-craig

"You can never have too many knives" -- Logan Nine Fingers
Rate this response:  
Not yet rated
stuartjvnorton
Participant



Joined: 19 Apr 2007
Posts: 527
Location: Melbourne
Points: 3924

Post Posted: Sat Dec 28, 2019 1:41 pm Reply with quote    Back to top    

Out of curiosity, what is the maximum length of the actual data in your field?
Rate this response:  
Not yet rated
dwh_user



Group memberships:
Premium Members

Joined: 18 Oct 2018
Posts: 14
Location: Sydney
Points: 168

Post Posted: Sat Dec 28, 2019 7:54 pm Reply with quote    Back to top    

The maximum length of the string in the field was around 3000.
Rate this response:  
Not yet rated
ray.wurlod

Premium Poster
Participant

Group memberships:
Premium Members, Inner Circle, Australia Usergroup, Server to Parallel Transition Group

Joined: 23 Oct 2002
Posts: 54581
Location: Sydney, Australia
Points: 295981

Post Posted: Tue Dec 31, 2019 10:05 pm Reply with quote    Back to top    

All strings in delimited text files are reported as length 255, unless you change that fact in QETXT.INI. And that, curiously, limits what can be inferred. I agree that it shouldn't.

_________________
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Rate this response:  
Not yet rated
Display posts from previous:       

Add To Favorites
View next topic
View previous topic
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum



Powered by phpBB © 2001, 2002 phpBB Group
Theme & Graphics by Daz :: Portal by Smartor
All times are GMT - 6 Hours