DSXchange: DataStage and IBM Websphere Data Integration Forum
View next topic
View previous topic
Add To Favorites
Author Message
anamika
Participant



Joined: 27 Feb 2016
Posts: 14
Location: Ottawa
Points: 193

Post Posted: Thu Nov 24, 2016 7:53 am Reply with quote    Back to top    

DataStage® Release: 9x
Job Type: Parallel
OS: Windows
Hi All,
Disclaimer: I am aware of Cognos Data Manager migration assistant that helps in the migration of Data Manager jobs to Datastage. For certain reasons we do not have access to this tool at our site. Also, the IBM Data Manager Migration assistant will only migrate jobs to Datastage and not to other tools (understandably) like Talend, SSIS, Informatica etc., Plus it has limitations for automation, job analysis etc.,

I have been entrusted with the task of automating the migration of Cognos Data Manager catalogs/jobs to Datastage. I have started the automated migration in small steps by analyzing Cognos Data Manager jobs using parsing tools and pre-generated Datastage job templates. Had some good success for simple Data Manager jobs and now moving on to automated analysis of complex Cognos Data Manager jobs.
I would be interested in hearing from this community about their experiences in analyzing/migrating/automation of Cognos Data Manager jobs, tools & techniques, strategies, pitfalls etc., to enhance my understanding of migration and automation.

Any thoughts, comments, pointers appreciated.

Thanks

_________________
ETL, DW, BI Consultant
chandu123



Group memberships:
Premium Members

Joined: 12 Sep 2016
Posts: 39

Points: 399

Post Posted: Fri Nov 25, 2016 7:23 pm Reply with quote    Back to top    

I am in the same boat. In my case, most of the data manager jobs have transformation logic in source SQL itself. As per my research, there is no way to convert the SQL into datastage jobs using stages. Hence I am developing datastage jobs by copying the SQLs to ODBC connector stage with User-defined SQL option as the write mode.

I am interested to know if there are any better practices in the space.

_________________
HB
Rate this response:  
Not yet rated
ray.wurlod

Premium Poster
Participant

Group memberships:
Premium Members, Inner Circle, Australia Usergroup, Server to Parallel Transition Group

Joined: 23 Oct 2002
Posts: 54042
Location: Sydney, Australia
Points: 293160

Post Posted: Sat Nov 26, 2016 4:20 am Reply with quote    Back to top    

Check out Metadex, from Compact BI. One of their metadata bridges can process SQL into metadata structures suitable for Information Server.

_________________
RXP Services Ltd
Melbourne | Canberra | Sydney | Hong Kong | Hobart | Brisbane
currently hiring: Canberra, Sydney and Melbourne
Rate this response:  
Not yet rated
anamika
Participant



Joined: 27 Feb 2016
Posts: 14
Location: Ottawa
Points: 193

Post Posted: Tue Nov 29, 2016 7:53 am Reply with quote    Back to top    

The sql inside the DM job can be extracted by querying the Cognos DM catalog metadata database. And it can be automated as well.
Once the Cognos DM job sql is extracted, you can insert that sql into a Datastage job template to create a Datastage job.
The above will work for jobs that follow a format/sequence that is known ahead of time. Dynamically generating a DS job is more complicated, but can be done.
Feel free to ask more questions if you need details.

Thx

_________________
ETL, DW, BI Consultant
Rate this response:  
Not yet rated
Display posts from previous:       

Add To Favorites
View next topic
View previous topic
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum



Powered by phpBB © 2001, 2002 phpBB Group
Theme & Graphics by Daz :: Portal by Smartor
All times are GMT - 6 Hours