Multiple Input links and Performance
Posted: Wed Jun 20, 2018 7:23 am
Hi,
It has been around 8 months that I started to work in datastage and dsxchange role was very vital. Now that i am comfortable with datastage basics, would like to go a little further and learn more.
I did learn that it would be performant to split the update and insert operations for DB rather choosing "insert then update" which I implemented in my jobs with 2 DB connectors (one for insert and one for update) and its really good. Recently i came across jobs that contain multiple input links to a single oracle connector with one link doing update and another doing insert. I did learn that mulitple links are made to a connector to handle all the actions in a single unit of work to better maintain transactions, which i totally agree.
I wanted to know if that(multiple links) would equally enhance the performance as well? because in our project context, we dont need to maintain the transaction, we need to insert or update millions of records.. I did make a test but at times the one with 2 connector perform better and at times the single connector with multiple link perform better. I am yet to make a full fledged test, but thought of posting here to get you views.
Would it not be that we overload the connector with different actions which could potentially degrade the performance ? May be i am wrong. It would be great if you could show some light on this topic..
Thanks
It has been around 8 months that I started to work in datastage and dsxchange role was very vital. Now that i am comfortable with datastage basics, would like to go a little further and learn more.
I did learn that it would be performant to split the update and insert operations for DB rather choosing "insert then update" which I implemented in my jobs with 2 DB connectors (one for insert and one for update) and its really good. Recently i came across jobs that contain multiple input links to a single oracle connector with one link doing update and another doing insert. I did learn that mulitple links are made to a connector to handle all the actions in a single unit of work to better maintain transactions, which i totally agree.
I wanted to know if that(multiple links) would equally enhance the performance as well? because in our project context, we dont need to maintain the transaction, we need to insert or update millions of records.. I did make a test but at times the one with 2 connector perform better and at times the single connector with multiple link perform better. I am yet to make a full fledged test, but thought of posting here to get you views.
Would it not be that we overload the connector with different actions which could potentially degrade the performance ? May be i am wrong. It would be great if you could show some light on this topic..
Thanks