Require Comments on work i m thinking to develop in DS-7.1

akash_nitj · Post by **akash_nitj** » Wed Nov 03, 2004 12:24 am

Hi Techies

I need a favour from your side . I am thinking of doing following work in DataStage 7.1

Please give ur valuable comments

Nature of Work : Data Migration from one data base to another.

Current Implementation : Java

Thinking of Moving to : DataStage7.1

Scope of Work: Migrating data from one database to another. Complexity involved is while migrating data for one table say x to another table y in new database we may have to do 5 to 6 lookups for referenc checks and also new database has effectivity. That is we create versions of records ( based upon the date). Same record can have different versions and based upon the current date we identify which record is effective. Also when adding or updating new record into new database effectivity needs to be maintained.

I think you might have by now general idea of migration involved.

I require your comments on this.
Should we go ahead with the datastage?
Can such a migartion be easily possible in DataStage?
Any risks or major factors that can hinder the process ?

....waiting for valuable reponse from you all.

Thanks in advance
Akash

poorna_76 · Post by **poorna_76** » Wed Nov 03, 2004 7:48 am

We are doing the things way more than that , using DataStage.

I think you can proceed further, to use DataStage for your purpose,

provided you have good Developers in DataStage.

(who has good command over HashFiles and Basic)

ray.wurlod · Post by **ray.wurlod** » Wed Nov 03, 2004 2:29 pm

While DataStage can certainly do all that you specify (without needing great skills), it seems an expensive solution if that is your only task.

On the other hand, if you're likely to have more data migration tasks, or are thinking of developing a data warehouse or similar, then it's probably a wise investment.

There are competitors to DataStage in the marketplace, but we here on this forum have a particular bias towards DataStage.

You tagged your environment as "parallel"; whether you really need this is a judgment you will have to make. The "enterprise edition", which is where you get the parallel execution engine, does cost a bit more, but can process huge volumes of data. On the Ascential web site there is a report of a benchmark where 1TB of data was processed in just under nine hours, processing 192 files with transformations applied to 68% of the columns.

"Server" (the default DataStage engine) can not manage this kind of throughput, but is quite efficient with even medium volumes of data. If you do purchase "enterprise edition" you get server as well as parallel engine.