DSXchange: DataStage and IBM Websphere Data Integration Forum
View next topic
View previous topic
Add To Favorites
Author Message
rsampath



Group memberships:
Premium Members

Joined: 25 Sep 2018
Posts: 1

Points: 25

Post Posted: Tue Oct 30, 2018 7:46 am Reply with quote    Back to top    

DataStage® Release: 11x
Job Type: Parallel
OS: Unix
Additional info: IGC 11.5
Hi all,

Apologies, in advance, for the long post.

Looking for some guidance on setting up our IGC environment (11.5). We're planning to use IGC to get metadata in from various systems in our landscape:

- DataStage/QualityStage/Info. Analyzer - Automatic
- DB (SQL Server/DB2/SAPHANA) - IMAM (ODBC/JDBC or CSV connectors (export))
- IBM Replication (IIDR) - Import via IIDR-IGC project ( Github )
- Hadoop - IMAM (HDFS/Hive or IGC-Atlas connector (we're on HDP 2.6.2; Atlas 0.8.0))
- Cognos BI - IMAM

I've reviewed other posts in this forum on setting up the IGC environment (should separate from core ETL environment (DEV/QA/Staging/PROD)):
- Post 1
- Post 2

I've also reviewed IBM Redbook SG247939 that discusses this topic, in some detail.

My current thinking is to have 2 environments (1 Sandbox & 1 production), each with its own IGC (& IMAM & Metadata interchange servers) environment, for the following reasons:
- To test out connectors/drivers before installing them for the production IGC instance
- To build/test imports/scripting/metadata import automation before implementing them in production

The publishing (surfacing out) of technical metadata as well as mapping of technical to business metadata would occur in the production instance.

Is this a typical environment setup? Is there normally 1 metadata interchange server per environment, or are there multiple? What are the factors that go into this decision?

If unable to have a separate IGC environment, has anyone setup the IGC environment to be part of their mainstream ETL environment? The main benefit of doing this, that I see, is to eliminate the need to copy the DS/QS jobs from the mainstream to the IGC environment, to load that metadata. What are the risks/issues associated with this approach?

Thanks, in advance.

RS
Display posts from previous:       

Add To Favorites
View next topic
View previous topic
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum



Powered by phpBB © 2001, 2002 phpBB Group
Theme & Graphics by Daz :: Portal by Smartor
All times are GMT - 6 Hours