Note sure if this is asked before (could'nt find it in the search)
----
-SCD-II(slowly changind dim) change for customer
-I will be pulling from a DB which have multiple changes within a day
eg: customer data changes 5 times a day at the source
-I will be pulling data once End of Day(EOD), for all the changes happened within that day or depending on my Timestamp
eg: there were 5 different changes at the source and I pull 5 changes for the same Natural Key.
-Is there a way to handle these in the data flow with in the PX
My need is show in the target of 5 changes for the day (start and end date each change) and the last one will be active, as of end of Day
I need to capture the lineage of source data.
PS: I guess we can use stage variables to access previous values, which might be cumbersum!
Also, this can be implemented using Server with a Hash file doing update and lookup on the same file.
Any suggestion from all you experts.
Multiple Delta Changes in the same cycle run
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
For SCD, (with my limited understanding is that),
my reference link should have the latest updated row for the next row to see if its a new or old record.
I am not sure if it can update the target and do a lookup on the same
(Similar to the one we do in Server, using a transform and dynamic HASH file, which can be used to as target and the ref link)
If we are using any database as ref link, I will be concerned on the performance.
my reference link should have the latest updated row for the next row to see if its a new or old record.
I am not sure if it can update the target and do a lookup on the same
(Similar to the one we do in Server, using a transform and dynamic HASH file, which can be used to as target and the ref link)
If we are using any database as ref link, I will be concerned on the performance.
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
The SCD stage caches the dimension table (or at least those rows thus far processed) in memory, so it should be able to detect that the natural key has already been seen in the current run.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.