Intgrity Doubts

raviyn · Post by **raviyn** » Thu May 29, 2003 9:57 pm

Hi,
I am new to this tool. I have heard Integrity helps in name and address cleansing as well as data enrichment with this using some standard address details.

Does it similarly do this sort of services for product i.e does it follow some standards like UNPSPC or NATO etc .Can it do Product Cleansing and classification and enrichment?

Also, I want one more clarification I read that Integrity client can be accesed from the DS designer window.Does that mean a Plugin is there for Integrity from where i can access the Integrity client or is it something else?

Thanks in advance

ray.wurlod · Post by **ray.wurlod** » Fri May 30, 2003 2:02 am

Q1. Does it similarly do this sort of services for product i.e does it follow some standards like UNPSPC or NATO etc .Can it do Product Cleansing and classification and enrichment?
A1. Several "rule sets" are provided with the INTEGRITY product, some of which implement standards. It can also do Soundex and NYSIIS comparison BOTH FORWARD AND REVERSE (I haven't seen reverse in any other tool). The probabilistic algorithms for multi-domain matching provide confidence levels that allow you to be as fuzzy or as tight as you need to.

Q2. Also, I want one more clarification I read that Integrity client can be accesed from the DS designer window.Does that mean a Plugin is there for Integrity from where i can access the Integrity client or is it something else?
A2. Yes. INTEGRITY, for many reasons, only works with fixed-width format data (for example, redefines are easier). There is an INTEGRITY plug-in for DataStage, which is properly integrated into the Parallel Extender architecture should you want to do the processing using parallel jobs.

raviyn · Post by **raviyn** » Fri May 30, 2003 3:09 am

So, as regards Q1 it means that either that "rule set" should be available by default or need to be manually or customly created. Is it so?

ray.wurlod · Post by **ray.wurlod** » Mon Jun 02, 2003 12:35 am

Several rule sets are supplied with INTEGRITY, for names, for addresses and so on, and for different parts of the world, for example USNAME, GBNAME, etc.
New rule sets can be adapted from these (for example the GBNAME rule set works fairly well in New Zealand, once a few Maori spellings are added), or created "from scratch".

raviyn · Post by **raviyn** » Tue Jun 03, 2003 12:04 am

Also in integrity, there is something called as Pre-built Procedures and just procedures which are created using the set of operators.What is the Difference?
I noticed one more thing if we use the superStan then we need to use the rule sets.
Where would i use just the procedures and where will i use the Pre-built ones?

If say for some sort of Desc matching where as such for eg.
Desc is say

100 W bulb
bulb of 100 W
Bulbs 100W
100W bulbs

All are the same things mentioned in Diff style.So how wld one approach a general case like this, where say I don't have any specific rule set?

Thanks

timwalsh · Post by **timwalsh** » Thu Jun 12, 2003 1:55 pm

Raviyn,

To my knowledge, no DQ product or cleansing product allows you to automatically standardize to UNSPSC codes, or to automatically standardize products, parts, items, or material descriptions.

NO ONE HAS THIS PRE-BUILT!

However, Integrity give you an excellent platform to develop your own standardization algorithms and well as probabalistic matching so that you can try and match to UNSPSC codes.

We will my performing this work in the near future. It should be pretty exciting.

In the past, my client's that have deployed UNSPSC codes, have manually added them to their system's. It's not a fun task, I assure you!

Cheers,

Tim