DSXchange: DataStage and IBM Websphere Data Integration Forum
View next topic
View previous topic
Add To Favorites
This topic has been marked "Resolved."
Author Message
U
Participant



Joined: 17 Apr 2007
Posts: 225
Location: Singapore
Points: 2795

Post Posted: Sun Jun 01, 2014 10:36 pm Reply with quote    Back to top    

DataStage® Release: 8x
Job Type: Parallel
OS: Unix
Additional info: Version 8.7
We are getting the following error message.

QualityStage Test wrote:
Standardization process failed. Table contains an invalid entry.
The above error occurred in the following line of file
C:/IBM/InformationServer/Clients/Classic/TempRules/IDD7DIGIT.TBL
\FORMAT\ SORT


We have created the TBL file in the Quality directory as follows:
Code:
;; QualityStage v8.0
\FORMAT\ SORT=Y
2125288 "Western Sahara"
2125289 "Western Sahara"
3906698 "Vatican City"
6189162 "Cocos Islands"
6189164 "Christmas Island"


The file put into the TempRules directory when we use the Test facility includes the complete first two lines (that is, the "=Y" is present), but none of the data.

Provisioning the rule set (for international telephone numbers) made no difference - this error keeps appearing.

Can anyone suggest what the problem might be?

(This is our first experience using the CONVERT action. We're pretty confident that the PAL is correct syntactically.)

Thank you for your time.
rjdickson
Participant



Joined: 16 Jun 2003
Posts: 378
Location: Chicago, USA
Points: 2531

Post Posted: Mon Jun 02, 2014 11:15 am Reply with quote    Back to top    

Hi,

From your description, it sounds like you are trying to everything from the command line. If true, you should use the GUI instead. You can create the table in the GUI. If you do not, then it will never know about the table, and you will not be able to use the rules tester.

Having said that, the problem may be that you need to make sure the table is UTF-8. The GUI does that for you Smile

_________________
Regards,
Robert
Rate this response:  
Not yet rated
ray.wurlod

Premium Poster
Participant

Group memberships:
Premium Members, Inner Circle, Australia Usergroup, Server to Parallel Transition Group

Joined: 23 Oct 2002
Posts: 54519
Location: Sydney, Australia
Points: 295643

Post Posted: Mon Jun 02, 2014 4:03 pm Reply with quote    Back to top    

To continue Robert's theme, the thing you're looking for in the New menu is Lookup Table (TBL), which you will find under Data Quality. Once the new table has been created, you double click to op ...

_________________
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.

Last edited by ray.wurlod on Mon Jun 02, 2014 7:36 pm; edited 1 time in total
Rate this response:  
Not yet rated
U
Participant



Joined: 17 Apr 2007
Posts: 225
Location: Singapore
Points: 2795

Post Posted: Mon Jun 02, 2014 7:34 pm Reply with quote    Back to top    

Creating the Lookup Table in Designer solved that problem.

It has introduced a secondary problem, in that we get "unable to decode" errors when trying to save a lookup table where there are accented characters in the data. For example "Côte d'Ivoire", "São Tomé and Príncipe" and "Réunion". So maybe UTF-8 is not as automatically handled as Robery suggested.

Replacing the accented characters with non-accented equivalents, while not entirely satisfactory, prevents this error from occurring.

Thank you again for your time.
Rate this response:  
Not yet rated
rjdickson
Participant



Joined: 16 Jun 2003
Posts: 378
Location: Chicago, USA
Points: 2531

Post Posted: Tue Jun 03, 2014 4:57 pm Reply with quote    Back to top    

Hi,

There are many rule sets with tables with values that have accents (think French, German, etc). Notepad, it turns out, is the culprit here...

Do you have an editor other than Notepad where you can save the file as UTF-8 (like the free Notepadd++)? If so, then in QualityStage, you can Right-click on your 'SET' and select Properties. Then select 'Options' to change the editor from Notepad to Notepad++ (in this case).

Now create your table, edit it, and note that the 'Encoding' option (again, in Notepad++) is UTF-8 without BOM.

And you can have all the accents you want Smile
Code:
;;QualityStage v8.0
\FORMAT\ SORT=Y
A "Réunion"


I hope this helps!

_________________
Regards,
Robert
Rate this response:  
Not yet rated
U
Participant



Joined: 17 Apr 2007
Posts: 225
Location: Singapore
Points: 2795

Post Posted: Tue Jun 03, 2014 5:57 pm Reply with quote    Back to top    

Yes, it seems that NotePad was the culprit. Using TextPad allowed accented characters to be included.

Unfortunately the QualityStage Test utility can't display them, but they standardise correctly.

Thank you very much for your time.
Rate this response:  
Not yet rated
Display posts from previous:       

Add To Favorites
View next topic
View previous topic
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum



Powered by phpBB © 2001, 2002 phpBB Group
Theme & Graphics by Daz :: Portal by Smartor
All times are GMT - 6 Hours