Difference between Change Capture Stage and Difference Stage

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
pkothana
Participant
Posts: 50
Joined: Tue Oct 14, 2003 6:12 am

Difference between Change Capture Stage and Difference Stage

Post by pkothana »

It is required that one file is compared with another one (Master File) and we have to get the Delta Records i.e. records which are either new or changed. Can anybody tell me which stage to use Change Capture or Difference and why? As I am new to Data Stage I am little bit confused. Any help in this regard is highly appreciated.
Note: Both the files will be having around 28M records.

Thanks & Regards
Pinkesh
Teej
Participant
Posts: 677
Joined: Fri Aug 08, 2003 9:26 am
Location: USA

Re: Difference between Change Capture Stage and Difference S

Post by Teej »

pkothana wrote:It is required that one file is compared with another one (Master File) and we have to get the Delta Records i.e. records which are either new or changed. Can anybody tell me which stage to use Change Capture or Difference and why? As I am new to Data Stage I am little bit confused. Any help in this regard is highly appreciated.
Note: Both the files will be having around 28M records.
We are using the delta concept and both Join and Lookup stages can handle this concept. Just use the reject rows to identify the new, and the passed records for the change records.

If you have separate rules to identify new and changes, you will need to use multiple lookups and/or join stages to handle this. With the size you refers to, Join stage will provide the best performance unless you got a monster of a computer system.

What you need to do is write down the rules to identify the new and the changed records. Are there any records that should be ignored? What are you supposed to do with new records? What are you supposed to do with changed records? Map it out.

Then try to translate it to DataStage tools. If you are still unsure on what to do, Ascential would love to provide their consultants for a price. So would quite a few folks on here (myself included).

-T.J.
Developer of DataStage Parallel Engine (Orchestrate).
Peytot
Participant
Posts: 145
Joined: Wed Jun 04, 2003 7:56 am
Location: France

Post by Peytot »

The change capture is faster and it is a goo solution for now and the evolution.

Pey
Post Reply