Unix vs Sql Severs

[jjrbikes]

Hi -
I have just landed a position in a relatively "small" IT shop - "small" in that they do not process billions of rows of data with their datastage jobs - but they are very heavy on the transform effort of the data they do process. when this shop was set up it was built on SQL Servers - but, evidently, has outgrown the capacity of what these severs can do. More accurately stated, the servers are too slow to do the processing in the time it needs to be done. So, now looking at a server "upgrade" of some sort - the question must be asked - do we "upgrade" with new and improved SQL Servers or do we make a jump to unix boxes given that we anticipate at least 15% growth in our processing needs in each of the next 5 years? It may also be noted that we are currently running ONLY the server edition of datastage - but will most likely be upgrading to enterprise edition with the server upgrade.

I am NOT a datastage admin - nor is anyone on our team - so I'm just looking for some guidance from anyone who feels strongly one way or the other and/or has experience with such an issue!

thanks

[ray.wurlod]

You are seeking to compare apples and oranges. UNIX is an operating system, SQL Server is a database server that runs on Windows operating system.

There IS a scaling strategy with SQL Server. If you have a competent SQL Server DBA, there may be value in leveraging that skill set. It will require additional Windows machines (or more CPUs in the current machine, if it's not already maxed out).

However, there are upper limits on what you can do with Windows. UNIX can, as a general rule, scale up far more than Windows.

But you don't need big, expensive hardware (though various sales dudes, whose commissions depend upon it, may try to convince you that you do).

There are low-cost strategies that will let you work with a cluster or grid of Linux machines that a number of sites have implemented with great success (in terms of throughput). There is good information to be had from IBM on how to do this (for example needing a dynamic configuration file for parallel jobs in a grid environment).

[vmcburney]

What do you have running on those Windows servers? Do you have both the target database and the DataStage engine on the same machines? If so you can improve performance by moving the database onto its own server. You can also upgrade your DataStage server with more RAM and look into upgrading to Enterprise Edition for parallel jobs or using multiple instance jobs to remove bottlenecks.

Do you do a combination of ETL (transformation via DataStage stages) and ELT (transformation via SQL)?

Is your database slow or are your data loads slow? Is your hardware fully utilised? Is your network slow?