DSXchange

Hi!

I have 11 sources each source has different conditions .I have to retrive 8 fields from each of the sources an the target is flat file having 16 standard fields .I need to map these 8 values coming from sourec to the respective target columns.Can you ppl give an suggestion to arrive to an simple solution,

Thanks
Tauseef

Hi,
welcome Aboard :D

You have asked, in simple, how to do a datawarehouse project
Do you have you High level design document/Low level Design document. Or only the feild mapping?
Is it all direct mapping?
What type of condition are they?
Do you have any aggregation?
More you provide the detail, more you get the information and answers.

-Kumar

Hi Buddy,

The way you have put up the problem, it seems to be quite simple.
Connect all the source links to a transformer stage and if the sources have a common key column then join the records based on that key.
Then based on the source conditions apply the constraints on the source if it is a database stage like oracle.

After the records are joined map them to respective target columns in the transformer stage and write to the output flat file

But my client wants me to use Single source table .And he wants me to check the data coming from the table in a file which has all the possible combinations of the conditions aas a look up.BUt the proble is that we can stisfy the conditions but we cannot direct which source value should go to which target column.As satisfying the conditions would'nt be a problem.But directing them to the respective columns will definitely be a problem.

No it won't. In a Transformer stage you can direct whatever columns you like to whatever output links you like.

I have 11 source and out of these you have 8 colums from each source coming out to the target based on the condition.So i can take all the possible colums from the source .filter that columns based on the conditions in the flat file.Now the problem is that for one condition there are different combinations of records going in to the target .According to the scenario in the Transfrormer stage there are !2 possible combinations in the input link and 16 columns in the target.So 8 columns should go to the target based on the condition.We can direct the values if there or one or two coinditions.But there are some situations where the conditions are same but data going to the target is different .What should i do

Document all the possibilities, and the columns that need to be transferred in each case.

The result should be a comprehensive target-from-source mapping document.

The process of creating this document will clear your mind as to how to implement the ETL.

I wasn't clear whether you have eleven sources or a single source table. Is there some mechanism whereby the eleven sources might be combined into a single source - for example cat command? That is, are the eight columns in each of your data sources the same?

What is the format of this sequential file that contains the conditions?

can you suggest some fonctions or routiens to direct the values from the source to the respective columns in the target based on the conditions.Because right now the main problem is that we can satisfy the conditions creating a file having all the possible combinations.But in the transformer how can we direct the values with out mapping them physically.Cos we have to map some how using routines or transforms.

Perhaps it might help to give an example of your requirements to illustrate your question. I am assuming that your question is more complicated, as you have self-evaluted yourself as an expert so it is unlikely that you are asking about the base DataStage transform functionality of mapping columns based on conditions.

mohdtausifsh wrote:can you suggest some fonctions or routiens to direct the values from the source to the respective columns in the target based on the conditions.

No.
Because you have not documented the algorithms required.

mohdtausifsh wrote:Because right now the main problem is that we can satisfy the conditions creating a file having all the possible combinations.But in the transformer how can we direct the values with out mapping them physically.Cos we have to map some how using routines or transforms.

Who said you have to do it this way?
What's wrong with regular constraint expressions?
(These can, of course, incorporate routines and/or transforms.)