Hardware configuration for DataStage System (Server)

PrashantJadhav · Post by **PrashantJadhav** » Mon May 25, 2009 9:42 am

Hi Folk,

Just an Info.

What would be the comprehensive hardware configuration for a windows system to executing a build job(on windows platform) of 50 millions of records? If specific, this job perform 25 lookups in several tables and load into remote database server. Consider an environment of seperate source DB server, target DB server and ETl job execution server. This info will helps me to understand idle execution time will required for a concered job. I'm focusing on processor speed and memory space to do possible computations.

Thanks,
Prashant

ArndW · Post by **ArndW** » Tue May 26, 2009 3:29 am

Are you certain that this is a TX question?

There is insufficent information to make any recommendations at all. 50 Million records with a key and one column is quite different from 50 million rows with 100 columns. Just reading & writing the columns could be complex (partitioning, key distribution, updates/inserts, etc.). Is there any processing going on? If so, what sort?

PrashantJadhav · Post by **PrashantJadhav** » Tue May 26, 2009 10:28 am

Yes, It is in TX.
Let consider
1.Migrating data 2GB (x records, y columns, z datatype etc.),
2.Involving tables(25) have referential intergrity and not null constraints,
3.Indexes on lookup columns,
4.No partitioning
5.Insert/Update operation SCD-2 implementation.