Column analysis - aborted job

This forum contains ProfileStage posts and now focuses at newer versions Infosphere Information Analyzer.

Moderators: chulett, rschirm

Post Reply
roy
Participant
Posts: 2598
Joined: Wed Jul 30, 2003 2:05 am
Location: Israel

Column analysis - aborted job

Post by roy »

Hi All,
trying to run column analysis on a file +-700MB
OS: RHL 6.2
IA: 11.5
Virtual Server with 8 CPUs
RAM: 32GB

getting the error next in line to the additional info in the topic:
pxbridge(2),0: com.ascential.e2.common.CC_Exception: java.lang.IllegalArgumentException: The value of -472844 for field Scale is either too small or too large to fit into the SmallInt type.
at com.ibm.is.cc.javastage.connector.CC_JavaRecordImpl.checkValue(CC_JavaRecordImpl.java:637)
at com.ibm.is.cc.javastage.connector.CC_JavaRecordImpl.setValue(CC_JavaRecordImpl.java:162)
at com.ibm.infosphere.ia.columnanalysis.operators.ColumnAnalysisResultsAggregationOperator.process(ColumnAnalysisResultsAggregationOperator.java:299)
at com.ibm.is.cc.javastage.connector.CC_JavaAdapter.run(CC_JavaAdapter.java:443)
at com.ibm.is.cc.javastage.connector.CC_JavaAdapter.run (CC_JavaAdapter.java: 458)
this is on the IA automatic OSH jobs for column analysis
Any idea?
(while everyone is on Xmas holiday lol ;))

So far I started analysis of each aborted column individualy to find the column that aborts; its only 1 column.

Also foound in the first warning:
"pxbridge(2): Unable to determine association between statement parameters and table columns. The connector will not be able to obtain external schema and only limited schema reconciliation will be performed"
but all is automatic, examining the data shows up to 20 characters and the default 255 characters were used as default metadata for profiling
next significant notice:
"pxbridge(4): [Input link 0] Unable to determine association between statement parameters and table columns. The connector will not be able to obtain external schema and only limited schema reconciliation will be performed"
This post informs that the warnings are not related

Thanks in advance...
Roy R.
Time is money but when you don't have money time is all you can afford.

Search before posting:)

Join the DataStagers team effort at:
http://www.worldcommunitygrid.org
Image
rjdickson
Participant
Posts: 378
Joined: Mon Jun 16, 2003 5:28 am
Location: Chicago, USA
Contact:

Post by rjdickson »

What is the source of the data? -472844 is certainly not a smallint, so it looks like the declared datatype in the source metadata may be wrong (or you may have data corruption..)
Regards,
Robert
roy
Participant
Posts: 2598
Joined: Wed Jul 30, 2003 2:05 am
Location: Israel

Post by roy »

I suspect either a bug or misuse of the product.
No definition of any kind was set except a string of 255 characters to all columns; the idea is that IA will suggest the type.

Specific analysis for that column only with 50K sample on random with 100% was successful :shock:
Roy R.
Time is money but when you don't have money time is all you can afford.

Search before posting:)

Join the DataStagers team effort at:
http://www.worldcommunitygrid.org
Image
rjdickson
Participant
Posts: 378
Joined: Mon Jun 16, 2003 5:28 am
Location: Chicago, USA
Contact:

Post by rjdickson »

Hi Roy,

Indeed - a PMR may be necessary. :(
Regards,
Robert
roy
Participant
Posts: 2598
Joined: Wed Jul 30, 2003 2:05 am
Location: Israel

Post by roy »

yes Robert,
this is caused when trying to analyse a string like "123456E3456789"
having this format causes the infered data type to be decimal
but the number of the characters after the 'E' is to long for a smallint
E is being translated as an exponent
if we analyse "1234A234323452E453635747" there is no problem
since the "A" negates the posibility of a decimal number

Regards from Alex Lavrov :wink:
A PMR has been opened...
Roy R.
Time is money but when you don't have money time is all you can afford.

Search before posting:)

Join the DataStagers team effort at:
http://www.worldcommunitygrid.org
Image
domenictroilo
Participant
Posts: 1
Joined: Tue Mar 01, 2016 10:41 am
Location: Canada

Post by domenictroilo »

Did you get this resolved? I have just opened up a PMR for myself. We are having the identical issue.

Domenic
rjdickson
Participant
Posts: 378
Joined: Mon Jun 16, 2003 5:28 am
Location: Chicago, USA
Contact:

Post by rjdickson »

For the record, it looks like this was fixed in the Governance Rollup 1 that was released 01 March.
Regards,
Robert
Post Reply