DSXchange: DataStage and IBM Websphere Data Integration Forum
View next topic
View previous topic
Add To Favorites
Author Message
jason2018
Participant



Joined: 03 Apr 2018
Posts: 5

Points: 37

Post Posted: Fri Jul 20, 2018 4:33 am Reply with quote    Back to top    

DataStage® Release: 8x
Job Type: Parallel
OS: Unix
Additional info: "source info" means table reference info of data source
I found that datastage jobs would detect the source table, retrieving information through the connector like db2 and informix connector, not just execute by the properties of fields I defined in the "columns tab" .
I found that while using funel and change capture stage.
Funel stage requires all sources have the same column definition, but even I defined the same columns in datastage for all connectors, it still throws an error that the actual tables have different column definitions.
And so is the change capture stage. If I use
Quote:
change mode=Explicit Keys, All Values
it always throws an error
Quote:
"A field cannot be both a key and a value"

But if I use
Quote:
change mode=Explicit Keys & Values
, it runs ok. And I guess that's because the stage can't tell the value fields or retrieves the wrong information from the connector.
And I still found that: only the informix connector causes the problem for change capture stage . The ODBC connector, oracle connector doesn't (I guess it can't retrieve information from the API or due to some other thoughts), and DB2 connector doesn't because it would retrieve the right info.
And, all I want to know is that How to prevent datastage jobs from retrieving source info , no matter the connector is informix,oracle or db2 or any other type.
Thanks all
qt_ky



Group memberships:
Premium Members

Joined: 03 Aug 2011
Posts: 2798
Location: USA
Points: 21145

Post Posted: Fri Jul 20, 2018 5:49 am Reply with quote    Back to top    

The DB2 Connector stage has a few properties related to Schema Reconciliation that you can toggle but I doubt those are going to help you much in this case.

A couple of ideas:

1) update your job properties and stage properties to disable Runtime Column Propagation, in case it is enabled, just to make sure it is not introducing any unexpected surprises

2) insert a Transformer stage in between your database stage and change capture stage where you do any necessary explicit type conversions and the like, because it sounds like you are really wanting to define the columns in your job a bit differently than how they are defined in the database

_________________
Choose a job you love, and you will never have to work a day in your life. - Confucius
Rate this response:  
Not yet rated
jason2018
Participant



Joined: 03 Apr 2018
Posts: 5

Points: 37

Post Posted: Sat Aug 04, 2018 2:38 am Reply with quote    Back to top    

thanks for replying. I tried that before, both, but didn't work. You get my point, I want to define the "keys" on my own, even if there is no unique constraints on the source table
Rate this response:  
Not yet rated
ray.wurlod

Premium Poster
Participant

Group memberships:
Premium Members, Inner Circle, Australia Usergroup, Server to Parallel Transition Group

Joined: 23 Oct 2002
Posts: 54287
Location: Sydney, Australia
Points: 294450

Post Posted: Sun Aug 05, 2018 5:54 pm Reply with quote    Back to top    

If you have Runtime Column Propagation enabled on the Connector stage (Outputs tab, Columns), disable it. A sometimes unwanted side effect of using RCP in a source stage that retrieves data from ...

_________________
RXP Services Ltd
Melbourne | Canberra | Sydney | Hong Kong | Hobart | Brisbane
currently hiring: Canberra, Sydney and Melbourne
Rate this response:  
Not yet rated
Display posts from previous:       

Add To Favorites
View next topic
View previous topic
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum



Powered by phpBB © 2001, 2002 phpBB Group
Theme & Graphics by Daz :: Portal by Smartor
All times are GMT - 6 Hours