IA Column Analysis fails when running against Netezza data

This forum contains ProfileStage posts and now focuses at newer versions Infosphere Information Analyzer.

Moderators: chulett, rschirm

Post Reply
ktrogers
Participant
Posts: 8
Joined: Tue Sep 27, 2016 9:17 am
Location: Albuquerque, NM

IA Column Analysis fails when running against Netezza data

Post by ktrogers »

I created a connection to a Netezza data source using ODBC. I then imported the data to the repository via Metadata Asset manager.

Tried to run a column analysis on one field in a table. It failed. In the SystemErr.log there is the following messages:
[3/28/17 13:48:15:724 MDT] 0000007f SystemErr R Caused by: org.apache.ojb.broker.accesslayer.OJBBatchUpdateException: com.ibm.db2.jcc.am.SqlSyntaxErrorException: [jcc][10177][11598][3.66.46] Invalid data conversion: Requested conversion would result in a loss of precision of Infinity. ERRORCODE=-4461, SQLSTATE=42815
[3/28/17 13:48:15:724 MDT] 0000007f SystemErr R at org.apache.ojb.broker.accesslayer.ConnectionManagerImpl.executeBatch(ConnectionManagerImpl.java:413)
[3/28/17 13:48:15:725 MDT] 0000007f SystemErr R at com.ibm.xmeta.persistence.orm.ojb.OjbObjectPersistence.executeBatch(OjbObjectPersistence.java:657)
[3/28/17 13:48:15:725 MDT] 0000007f SystemErr R ... 53 more
[3/28/17 13:48:15:725 MDT] 0000007f SystemErr R Caused by: com.ibm.db2.jcc.am.SqlSyntaxErrorException: [jcc][10177][11598][3.66.46] Invalid data conversion: Requested conversion would result in a loss of precision of Infinity. ERRORCODE=-4461, SQLSTATE=42815
Further down in the log there are these messages:
[3/28/17 14:18:43:710 MDT] 00000074 SystemErr R com.ascential.investigate.exception.CreateFrequencyDistributionException: Failed to create FreqDist Summary for Error in post-process for DIMENSION."CDR_DIM"
[3/28/17 14:18:43:711 MDT] 00000074 SystemErr R at com.ascential.investigate.ca.job.BaseProfileJob.doPostProcess(BaseProfileJob.java:597)
[3/28/17 14:18:43:711 MDT] 00000074 SystemErr R at com.ascential.investigate.utils.jobs.JobProcessor$PostProcessor.run(JobProcessor.java:475)
[3/28/17 14:18:43:711 MDT] 00000074 SystemErr R at com.ibm.iis.isf.j2ee.impl.was.thread.WASScheduledExecutorService$RunnableWorkAdapter.run(WASScheduledExecutorService.java:847)
[3/28/17 14:18:43:711 MDT] 00000074 SystemErr R at com.ibm.ws.asynchbeans.J2EEContext$RunProxy.run(J2EEContext.java:269)
[3/28/17 14:18:43:711 MDT] 00000074 SystemErr R at java.security.AccessController.doPrivileged(AccessController.java:274)
[3/28/17 14:18:43:711 MDT] 00000074 SystemErr R at com.ibm.ws.asynchbeans.J2EEContext.run(J2EEContext.java:795)
[3/28/17 14:18:43:711 MDT] 00000074 SystemErr R at com.ibm.ws.asynchbeans.WorkWithExecutionContextImpl.go(WorkWithExecutionContextImpl.java:222)
[3/28/17 14:18:43:711 MDT] 00000074 SystemErr R at com.ibm.ws.asynchbeans.ABWorkItemImpl.run(ABWorkItemImpl.java:206)
[3/28/17 14:18:43:711 MDT] 00000074 SystemErr R at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1862)

In the SystemOut.log file, I found these messages:

[3/28/17 13:48:15:685 MDT] 0000007f PersistenceUt E [jcc][10177][11598][3.66.46] Invalid data conversion: Requested conversion would result in a loss of precision of Infinity. ERRORCODE=-4461, SQLSTATE=42815 - SQLSTATE: 42815 , Error code: -4461
[3/28/17 13:48:15:685 MDT] 0000007f PersistenceUt E [jcc][10177][11598][3.66.46] Invalid data conversion: Requested conversion would result in a loss of precision of Infinity. ERRORCODE=-4461, SQLSTATE=42815 - SQLSTATE: 42815 , Error code: -4461
[3/28/17 13:48:15:687 MDT] 0000007f RepositoryMet E Error executing saveAndDeleteEObjects_251: Error saving the given EObjects to the persistent store.
[3/28/17 13:48:15:719 MDT] 0000007f FfdcProvider W com.ibm.ws.ffdc.impl.FfdcProvider logIncident FFDC1003I: FFDC Incident emitted on /opt/IBM/WebSphere/AppServer/profiles/InfoSphere/logs/ffdc/server1_7c143044_17.03.28_13.48.15.69345441277412925297.txt com.ibm.ws.uow.UOWManagerImpl.runUnderNewUOW 934
[3/28/17 13:48:15:727 MDT] 0000007f SorcererServi E CDIIA0237E: Failed to create FreqDist Summary for
com.ascential.investigate.exception.SorcererGenericException: Something went wrong. More details follow... Something went wrong. More details follow... com.ibm.iis.xmeta.client.exception.ServiceException: Error saving the given EObjects to the persistent store.; nested exception is:
java.lang.RuntimeException: com.ibm.iis.xmeta.client.exception.ServiceException: Error saving the given EObjects to the persistent store.
at com.ascential.investigate.utils.dataclassification.DataClassification.uploadClassificationResultsToModel(DataClassification.java:569)
at com.ascential.investigate.utils.dataclassification.DataClassification.setDataClassification(DataClassification.java:287)
at com.ascential.investigate.ca.job.ProfileSummaryBuilder.setDataClassification(ProfileSummaryBuilder.java:1452)
at com.ascential.investigate.ca.job.BaseProfileJob.doPostProcess(BaseProfileJob.java:474)
at com.ascential.investigate.utils.jobs.JobProcessor$PostProcessor.run(JobProcessor.java:475)
at com.ibm.iis.isf.j2ee.impl.was.thread.WASScheduledExecutorService$RunnableWorkAdapter.run(WASScheduledExecutorService.java:847)
at com.ibm.ws.asynchbeans.J2EEContext$RunProxy.run(J2EEContext.java:269)
at java.security.AccessController.doPrivileged(AccessController.java:274)
at com.ibm.ws.asynchbeans.J2EEContext.run(J2EEContext.java:795)
at com.ibm.ws.asynchbeans.WorkWithExecutionContextImpl.go(WorkWithExecutionContextImpl.java:222)
at com.ibm.ws.asynchbeans.ABWorkItemImpl.run(ABWorkItemImpl.java:206)
at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1862)

[3/28/17 13:48:16:502 MDT] 0000007f SorcererServi E CDIIA0001E: JobProcessor.java:496 ADJ_PRD_IND36065674: Error in post-processing job ->Failed to create FreqDist Summary for Error in post-process for DIMENSION."CDR_DIM"
[3/28/17 13:48:16:537 MDT] 0000007f SorcererServi W CDIIA0002W: IA Job Status


In researching I found several comments about NOt having Data Classes loaded into our system. We do, we have 38 data classes.

I looking at the data, the column in question is defined as:
In Aginity Workbench as an INTEGER
In Metadata Asset manager as an INT32.
In the Director log for the job it is looks like it is trying to convert it to a dfloat.


I did run a column analysis on a different field on this table and it was successful. It is defined in Aginity as a CHARACTER VARYING(30) NOT NULL
In Metadata Asset Manage it is a STRING filed.
In viewing the detail, for Data Class it has NoClassDetected.

For a field that is defined as a DATE in Aginity, it is defined as a DATE in Metadata Asset Manager and the Column Analysis runs fine.


Is anyone run IA Column Analysis against a Netezza data source?
Do I need to load more Data Classes? If yes, where do I get them from?

Thanks
wilsonpenha
Participant
Posts: 1
Joined: Fri Aug 04, 2017 11:37 am

Re: IA Column Analysis fails when running against Netezza da

Post by wilsonpenha »

Hello

Did you solve this issue?

regards,
Post Reply