Processing Nodes

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
Mark j
Participant
Posts: 20
Joined: Tue Apr 20, 2004 9:26 am

Processing Nodes

Post by Mark j »

Hi

This is a general question. How will you configure processing nodes and is there any limit for processing nodes for a processor and do we need to do any scheduling for nodes... i don't know whether these questions are appropriate, but well these thoughts bugs my mind.


Thanks
Mark
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

THere are no technical limits; only sensible ones. Don't overload any machine - for example you would not configure 24 nodes on a four CPU machine in general!
No scheduling by you is necessary; the PX engine looks after allocation of notes based on the configuration file that you specify. Note that you can specify different configuration files for different jobs; for example you would use a single processing node for a tiny job that only processes a few rows.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
richdhan
Premium Member
Premium Member
Posts: 364
Joined: Thu Feb 12, 2004 12:24 am

Post by richdhan »

Hi,

Following Mark's Question What do they mean by CPU and what do they mean by Node? In one of the posts I found through search which says that number of nodes should be the half of the number of CPU's for better performance. Pls clarify

Thanks
--Rich


A little bit of ink is powerful than the strongest memory
Sreenivasulu
Premium Member
Premium Member
Posts: 892
Joined: Thu Oct 16, 2003 5:18 am

Post by Sreenivasulu »

The General rule of the thumb is - Two processing nodes per cpu

Regards
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Even that may be too many, there are times, for example when the database server (on the same machine) also needs CPUs, or you are doing some really intensive processing in your ETL, that you'd configure maybe one processing node per two CPUs.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Post Reply