DSXchange: DataStage and IBM Websphere Data Integration Forum
View next topic
View previous topic
Add To Favorites
Author Message
admin
Site Admin



Joined: 12 Jan 2003
Posts: 8720

Points: 10

Post Posted: Mon Feb 23, 2004 1:51 am Reply with quote    Back to top    

Hi All,
this may be a 'how long is a piece of string question' but here goes.

We have been doing our development on a win2000 machine with 1x2GHz
processor and 1GB memory on DS 6.0.

We have taken delivery of our shiny new Sun12K. The development
domain has 4x1.2GHz sparc processors and 8GB memory with DataStage
Version 7.

We have done nothing to improve the performance on the 12K, we have
just moved the win2000 database/datastage instance to the 12K.

I am seeing performance levels that I would have thought are well too
slow for the 12K. I'm interested to know if anyone has experience of
moving a job from win->solaris and can make a comment on the
performance differences we should expect to see, if any, on a per
processor basis.

For example, if I take a simple job like loading 1M rows of an oracle table
into a hask file, what should I expect to see in relative performance when
this job runs on one CPU on win2000 and then on one CPU on solaris?

Any feedback most appreciated.

Best Regards

Peter Nolan

_______________________________________________
datastage-users mailing list
datastage-users@oliver.com
http://www.oliver.com/mailman/listinfo/datastage-users

_________________
PLEASE READ
Do not contact admin unless you have technical support or account questions. Do not send email or Private Messages about discussion topics to ADMIN. Contact the webmaster concerning abusive or offensive posts.
admin
Site Admin



Joined: 12 Jan 2003
Posts: 8720

Points: 10

Post Posted: Mon Feb 23, 2004 2:38 am Reply with quote    Back to top    

Hi Peter, long time no see.

In your example, I would have thought that Oracle database performance,
network performance to the database and disk performance for the hash file
would have played a much bigger role.

Unless you are doing anything to parallelise the processing, the 4 CPUs on
the Sparc are of little value to you, the job will just use one of the
arguably slower (1.2Ghz RISC vs 2Ghz CISC) CPUs (conceptually - ie max of
25% CPU utilisation). As for memory, well, I'd be surprised if the job used
much memory in either environment unless you have done something to
configure very large I/O buffers at the OS level.

As you can see, Unix environments, typified by many fairly average CPUs are
not always faster than Windows running on fewer, often faster CPUs. Just
depends on what you are doing.

Now, if you try running many jobs at the same time, you might get a
completely different comparison.


David


-----Original Message-----
From: datastage-users-bounces@oliver.com
[mailto:datastage-users-bounces@oliver.com]On Behalf Of pnolan@attglobal.net
Sent: Monday, 23 February 2004 5:51 PM
To: datastage-users@oliver.com
Subject: Relative Performance for win2000/solaris 9

Hi All,
this may be a 'how long is a piece of string question' but here goes.

We have been doing our development on a win2000 machine with 1x2GHz
processor and 1GB memory on DS 6.0.

We have taken delivery of our shiny new Sun12K. The development
domain has 4x1.2GHz sparc processors and 8GB memory with DataStage
Version 7.

We have done nothing to improve the performance on the 12K, we have
just moved the win2000 database/datastage instance to the 12K.

I am seeing performance levels that I would have thought are well too
slow for the 12K. I'm interested to know if anyone has experience of
moving a job from win->solaris and can make a comment on the
performance differences we should expect to see, if any, on a per
processor basis.

For example, if I take a simple job like loading 1M rows of an oracle table
into a hask file, what should I expect to see in relative performance when
this job runs on one CPU on win2000 and then on one CPU on solaris?

Any feedback most appreciated.

Best Regards

Peter Nolan

_______________________________________________
datastage-users mailing list
datastage-users@oliver.com
http://www.oliver.com/mailman/listinfo/datastage-users


_______________________________________________
datastage-users mailing list
datastage-users@oliver.com
http://www.oliver.com/mailman/listinfo/datastage-users

_________________
PLEASE READ
Do not contact admin unless you have technical support or account questions. Do not send email or Private Messages about discussion topics to ADMIN. Contact the webmaster concerning abusive or offensive posts.
Rate this response:  
Not yet rated
admin
Site Admin



Joined: 12 Jan 2003
Posts: 8720

Points: 10

Post Posted: Mon Feb 23, 2004 11:52 pm Reply with quote    Back to top    

Hi David,
yes, it's been years now!!! Hope you are doing fine.

Yes. We are in the early phases of moving what we have developed on
win2000 across to the 12K. (Which meant we could delay that purchase
for 8 months.)

The job I quoted is very simple with no opportunity for anything to 'go
faster' but the CPU. (That was the intention of the test.)

I just thought (and had been told) that a sun sparc processor 1.2 will 'run
faster' than a 2GHz intel processor......and I was working on the premise
that DS will 'go faster' as well....but it does not seem to be the case for a
simple job bound to one CPU.

We have PX and we are testing with it. The same job I quoted goes 10x
faster when re-written as a PX job. So we are really just coming to grips
with the parallelisation of what we have developed. Production volume
testing starts this week. Our production cluster should be up in 2-3 weeks.

If we put the job unchanged into the PX environment it runs 2 times as
fast as outside the PX environment.

We will be able to take advantage of more memory, fibre connect to disks,
and more processors. I single streamed a batch yesterday and then ran all
12 jobs at once and the overall time was cut in half which was really just
the amount of time the longest running job took.

It looked like it would just run all CPUs as fast as possible for the
workload.


Hopefully I'll have plenty of feedback for oliver and DSXchange in a month
or two.....

Best Regards

Peter Nolan

_______________________________________________
datastage-users mailing list
datastage-users@oliver.com
http://www.oliver.com/mailman/listinfo/datastage-users

_________________
PLEASE READ
Do not contact admin unless you have technical support or account questions. Do not send email or Private Messages about discussion topics to ADMIN. Contact the webmaster concerning abusive or offensive posts.
Rate this response:  
Not yet rated
Display posts from previous:       

Add To Favorites
View next topic
View previous topic
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum



Powered by phpBB © 2001, 2002 phpBB Group
Theme & Graphics by Daz :: Portal by Smartor
All times are GMT - 6 Hours