DSXchange: DataStage and IBM Websphere Data Integration Forum
View next topic
View previous topic
Add To Favorites
Author Message
dj
Participant



Joined: 24 Aug 2006
Posts: 78
Location: india
Points: 1101

Post Posted: Mon Jun 20, 2016 9:04 am Reply with quote    Back to top    

DataStage® Release: 11x
Job Type: Parallel
OS: Unix
Hi All,

I'm trying to create a rule to identify words to ignore and i'm referring to the VDATE/VPHONE ruelsets.

If any of the input column for a row is identified with IGNORABLE words then it is set.

For single pattern it works fine. But the Input string is of 7 Columns with N number of patterns.

Code:

+ ;
COPY_S [1] temp
CONVERT temp @IGNORE.TBL TEMP
** | [temp = "IGNORE"] ; Table hit
COPY "I" {InvalidReason}
COPY [1] {InvalidData}
RETYPE [1] 0
RETURN

** ;Valid if we get here
COPY "T" {ValidFlag} OF  VIGNORE
COPY data {ValidData} OF VIGNORE
RETYPE [1] 0


;other patterns
+|+ ;JAMES DUNGEON
^++++241 ABEL LANE WARD JAMES

Do i have to repeat for all possible number of patterns? Can someone help on this.

Thanks
ray.wurlod

Premium Poster
Participant

Group memberships:
Premium Members, Inner Circle, Australia Usergroup, Server to Parallel Transition Group

Joined: 23 Oct 2002
Posts: 54407
Location: Sydney, Australia
Points: 295092

Post Posted: Mon Jun 20, 2016 3:55 pm Reply with quote    Back to top    

You can use the scanning and "search anywhere" components in PAL, for example *+

It really depends on exactly what you want to be able to do.

_________________
RXP Services Ltd
Melbourne | Canberra | Sydney | Hong Kong | Hobart | Brisbane
currently hiring: Canberra, Sydney and Melbourne (especially seeking good business analysts)
Rate this response:  
Not yet rated
rjdickson
Participant



Joined: 16 Jun 2003
Posts: 378
Location: Chicago, USA
Points: 2531

Post Posted: Tue Jun 21, 2016 5:43 am Reply with quote    Back to top    

Try using %1, %2, %3, etc.

_________________
Regards,
Robert
Rate this response:  
Not yet rated
dj
Participant



Joined: 24 Aug 2006
Posts: 78
Location: india
Points: 1101

Post Posted: Wed Jul 27, 2016 12:10 pm Reply with quote    Back to top    

Thanks, we kind of created a new classfication code for ignorable words.

But for single words with space, as in classfication table we are unable to add , these words are not getting properly handled.

We cannot add w1 w2 as 2 different entries as w1 and w2 when parsed separately will be of valid ones and pattern generated is incorrect.

word - ignorable words
w1 w2 - single word with space

Eg: Smith word :: pattern : FX
Smith w1 w2 :: pattern should get generated as FX
Thanks
Rate this response:  
Not yet rated
rjdickson
Participant



Joined: 16 Jun 2003
Posts: 378
Location: Chicago, USA
Points: 2531

Post Posted: Wed Jul 27, 2016 12:17 pm Reply with quote    Back to top    

If you add w1 and w2 as class X then
Code:
smith w1 w2
will get pattern FXX.
You can then, if you want:
Code:
*X | X ; two ignore words in a row
COPY [1] temp
CONCAT " " temp
CONCAT [2] temp
RETYPE [1] X temp temp
RETYPE [2] 0


Now you have the FX you desire.

You may want to handle FXXX and FXXXX as well...

_________________
Regards,
Robert
Rate this response:  
Not yet rated
dj
Participant



Joined: 24 Aug 2006
Posts: 78
Location: india
Points: 1101

Post Posted: Thu Jul 28, 2016 4:40 am Reply with quote    Back to top    

Thanks Robert.

Eg:
shut up - is a single word provided by customer to ignore them.
smith "shut up"
shut X
up X
Pattern :FXX

But what we doubt is if either "shut" or "up" appears to be valid ones if it is part of any name/address, then it may be incorrect to categorize them as "X".

Is there a way to read them as whole string?

Thanks.
Rate this response:  
Not yet rated
rjdickson
Participant



Joined: 16 Jun 2003
Posts: 378
Location: Chicago, USA
Points: 2531

Post Posted: Thu Jul 28, 2016 4:49 am Reply with quote    Back to top    

Hi,

Yes. After you find XX you then look that phrase up on a table you create that has valid XX combinations. If found, replace. If not, then restype to + to prevent the X's from being handled later.

_________________
Regards,
Robert
Rate this response:  
Not yet rated
Display posts from previous:       

Add To Favorites
View next topic
View previous topic
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum



Powered by phpBB © 2001, 2002 phpBB Group
Theme & Graphics by Daz :: Portal by Smartor
All times are GMT - 6 Hours