DSXchange: DataStage and IBM Websphere Data Integration Forum
View next topic
View previous topic
Add To Favorites
Author Message
pk7
Participant



Joined: 03 Aug 2007
Posts: 44
Location: Kansas CIty
Points: 581

Post Posted: Mon Nov 25, 2013 11:49 am Reply with quote    Back to top    

DataStage® Release: 8x
Job Type: Parallel
OS: Unix
Getting this warning which is ultimately causing an abort since our company policy only allows a max of 50 warnings before an abort is issued. Would like to eliminate the root cause of the problem and not just remove the limit of 50 warnings. Can't find any reference to a solution in the posts or on the internet. Help!
asorrell
Site Admin

Group memberships:
Premium Members, DSXchange Team, Inner Circle, Server to Parallel Transition Group

Joined: 04 Apr 2003
Posts: 1694
Location: Colleyville, Texas
Points: 23058

Post Posted: Mon Nov 25, 2013 11:54 am Reply with quote    Back to top    

You'll have a better chance of getting a reply if you post more details about what is generating the error. What stage? What settings? Does this occur on other jobs? More details the better.

_________________
Andy Sorrell
Certified DataStage Consultant
IBM Analytics Champion 2009 - 2017
Rate this response:  
Not yet rated
pk7
Participant



Joined: 03 Aug 2007
Posts: 44
Location: Kansas CIty
Points: 581

Post Posted: Mon Nov 25, 2013 1:37 pm Reply with quote    Back to top    

Yes, my apologies.

This error is occurring in a standardization stage that has USPREP as the Rule and "ZQMIXNZQ" ADDRESS as the Column . Examples of the input data producing the warnings are:

Nick Smith AEB 8900 Burnnet St Ste 33 Houston TX 78754
Acme Mfg Llc 100 Morry Mill Rd Floral VA 24409

The job runs to completion and even gives the correct results, however, it is the high Warning count that is giving us grief. How can we eliminate most or all of these warnings?
Rate this response:  
Not yet rated
asorrell
Site Admin

Group memberships:
Premium Members, DSXchange Team, Inner Circle, Server to Parallel Transition Group

Joined: 04 Apr 2003
Posts: 1694
Location: Colleyville, Texas
Points: 23058

Post Posted: Mon Nov 25, 2013 5:38 pm Reply with quote    Back to top    

I'm not certain what is causing the warning - I defer to some of the QualityStage experts for that.

If you have no other alternative you can establish a message handler, for the job or for the project, and map the message level down from Warning to Informational. I do not suggest elimination of the messages unless their sheer number is overwhelming the logs.

Lots of posts here on the subject - search for message handler.

_________________
Andy Sorrell
Certified DataStage Consultant
IBM Analytics Champion 2009 - 2017
Rate this response:  
Not yet rated
ray.wurlod

Premium Poster
Participant

Group memberships:
Premium Members, Inner Circle, Australia Usergroup, Server to Parallel Transition Group

Joined: 23 Oct 2002
Posts: 54524
Location: Sydney, Australia
Points: 295662

Post Posted: Tue Nov 26, 2013 3:36 am Reply with quote    Back to top    

What overrides exist in your USPREP rule set?

_________________
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Rate this response:  
Not yet rated
pk7
Participant



Joined: 03 Aug 2007
Posts: 44
Location: Kansas CIty
Points: 581

Post Posted: Tue Nov 26, 2013 10:50 am Reply with quote    Back to top    

Hi Ray. I have absolutely no overrides at all in the rule set. I even attempted to put in an override to see if that helped but to no avail so I removed that one.
Rate this response:  
Not yet rated
ray.wurlod

Premium Poster
Participant

Group memberships:
Premium Members, Inner Circle, Australia Usergroup, Server to Parallel Transition Group

Joined: 23 Oct 2002
Posts: 54524
Location: Sydney, Australia
Points: 295662

Post Posted: Tue Nov 26, 2013 3:26 pm Reply with quote    Back to top    

The literal for the PREP rule sets is supposed to encase a two-character ISO country code. This literal specifies the default country code to use if the logic of the rule set cannot determine one - i ...

_________________
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Rate this response:  
Not yet rated
rjdickson
Participant



Joined: 16 Jun 2003
Posts: 378
Location: Chicago, USA
Points: 2531

Post Posted: Mon Dec 02, 2013 6:04 am Reply with quote    Back to top    

ZQMIXNZQ is a valid literal for USPREP.
Can you please provide exactly what (and the order) you are sending to the Standardization Stage?

_________________
Regards,
Robert
Rate this response:  
Not yet rated
pk7
Participant



Joined: 03 Aug 2007
Posts: 44
Location: Kansas CIty
Points: 581

Post Posted: Mon Dec 02, 2013 10:47 am Reply with quote    Back to top    

The structure is:

Transformer----->USPREP Std stage------->Dataset file

Columns coming into the Standardization stage are:

CENSUS_GRP_ID:decimal[38,0] {delim=',', precision=38, scale=0};
SRC_TYP_CD:nullable string[max=5];
SRC_TYP_DESCR:nullable string[max=60];
CASE_ID:nullable decimal[38,0];
SHIPPING_ADDRESS:nullable string[max=100];
CRTD_BY:string[max=30];
CRTD_DT:timestamp;
LAST_CHNGD_BY:nullable string[max=30];
LAST_CHNGD_DT:nullable timestamp {null_field=' '};

The Standardization stage conatins:

Rules:
USPREP

Columns:
"ZQMIXNZQ", SHIPPING_ADDRESS

Sample input data in SHIPPING_ADDRESS is:

Nick Smith AEB 8900 Burnnet St Ste 33 Houston TX 78754
Acme Mfg Llc 100 Morry Mill Rd Floral VA 24409

Result: Stated in earlier posts.
Rate this response:  
Not yet rated
rjdickson
Participant



Joined: 16 Jun 2003
Posts: 378
Location: Chicago, USA
Points: 2531

Post Posted: Mon Dec 02, 2013 11:28 am Reply with quote    Back to top    

Looks good to me. Another thought:

Please go to the server and then the Quality directory for your project. For example, if you are on a Linux box and using project 'dstage1', then go to /opt/IBM/InformationServer/Server/Projects/dstage1/Quality.

Then, please tell us the contents of the following files:
USPREP.ITO
USPREP.IPO
USPREP.FTO
USPREP.FPO

I know you said you already checked overrides, via the GUI. I am just asking you to check via the file system.

_________________
Regards,
Robert
Rate this response:  
Not yet rated
pk7
Participant



Joined: 03 Aug 2007
Posts: 44
Location: Kansas CIty
Points: 581

Post Posted: Mon Dec 02, 2013 11:37 am Reply with quote    Back to top    

As mentioned in an earlier post there are currently no overrides. However, here are the contents:

.FPO file:
;;QualityStage v8.0
\FORMAT\ SORT=Y
ENPREPFP TBL

.FTO file:
;;QualityStage v8.0
\FORMAT\ SORT=Y
ENPREPFT TBL

.IPO file:
;;QualityStage v8.0
\FORMAT\ SORT=Y
ENPREPIP TBL

.ITO file:
;;QualityStage v8.0
\FORMAT\ SORT=Y
ENPREPIT TBL
Rate this response:  
Not yet rated
rjdickson
Participant



Joined: 16 Jun 2003
Posts: 378
Location: Chicago, USA
Points: 2531

Post Posted: Mon Dec 02, 2013 11:47 am Reply with quote    Back to top    

Fascinating...Two things:
1) I just ran your 'Nick Smith' example and did not receive any error or warning messages.
2) The out-of-the-box rule sets (which are read-only) have US PREP... instead of EN PREP... There is no out-of-the-box ENPREP rule set.

Are you are using a modified PREP rule set? If so, what happens when you use the out-of-the-box PREP rule set? Or AUPREP (just to see if there is a warning).

_________________
Regards,
Robert
Rate this response:  
Not yet rated
Display posts from previous:       

Add To Favorites
View next topic
View previous topic
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum



Powered by phpBB © 2001, 2002 phpBB Group
Theme & Graphics by Daz :: Portal by Smartor
All times are GMT - 6 Hours