duplicate record at lookup

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
qutesanju
Participant
Posts: 373
Joined: Tue Aug 26, 2008 4:52 am

duplicate record at lookup

Post by qutesanju »

I do have a lookup as

Code: Select all

                                DB2 table
                                    |   | 
                                    |   |
----input data stream-->Lookup--target stream
                                     | 
                                     |
                                reject file
Also in lookup i specified conditions as
input.key1=db2table.key1 based on condition Flag = OUT
input.key2=db2table.key2 based on condition Flag = IN


in lookup constraints I specified action as

Code: Select all

condition 	condition not met	lokup on failure
---------	------------------	------------------
Flag = OUT      continue	continue
Flag = IN	      continue	continue
In director its showing warning as two times

lkp,0: Ignoring duplicate entry at table record 403958; no further warnings will be issued for this table

lkp,0: Ignoring duplicate entry at table record 403958; no further warnings will be issued for this table


1 does that mean it has two duplicate records?
And
2 how can i capcture those duplicate records in a file.
will reject file capcture duplicate records? OR rejected records OR both?
chetan.c
Participant
Posts: 112
Joined: Tue Jan 17, 2012 2:09 am
Location: Bangalore

Post by chetan.c »

Search the forum for Ignoring duplicate entry at table record

If the reference set is expected to have multiple records and you are expecting multiple lookup results , then enable return multiple rows in the lookup.

Else de duplicate the data using a remove duplicate stage.
If you have capture the duplicates then use sort stage with ke change column and redirecting rows with keychange as 0 to file.


Thanks,
Chetan.C
qutesanju
Participant
Posts: 373
Joined: Tue Aug 26, 2008 4:52 am

Post by qutesanju »

thanks I used sort stage with key change column and redirecting rows with keychange as 0 to file.

this way i was able to see duplicate records in separate file
srinivas.nettalam
Participant
Posts: 134
Joined: Tue Jun 15, 2010 2:10 am
Location: Bangalore

Post by srinivas.nettalam »

qutesanju wrote:thanks I used sort stage with key change column and redirecting rows with keychange as 0 to file.

this way i was able to see duplicate records in separate file
Were you trying to do the lookup with the same data?
N.Srinivas
India.
Post Reply