Grid compute node

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
Curious George
Premium Member
Premium Member
Posts: 11
Joined: Tue Oct 29, 2013 7:25 pm
Location: USA

Grid compute node

Post by Curious George »

Hi All,

Can you please let me know why we would pass 2 values to APT_Grid_computenodes
From what I read about the Apt variable I was thinking it should have a single value passed to it. But in the current environment for a new customer I see values passed like the below:
Apt_grid_computenodes=2,2 or 12,3 etc
lstsaur
Participant
Posts: 1139
Joined: Thu Oct 21, 2004 9:59 pm

Post by lstsaur »

It should be just a single value.
Curious George
Premium Member
Premium Member
Posts: 11
Joined: Tue Oct 29, 2013 7:25 pm
Location: USA

Post by Curious George »

Hi

Thanks for the reply. I was under the same assumption reading the IBM manuals but as I mentioned two numbers are passed to this variable .
lstsaur
Participant
Posts: 1139
Joined: Thu Oct 21, 2004 9:59 pm

Post by lstsaur »

Unless the toolkit has been changed; I don't see why two values are needed. You can take a look the log from the Director; it has more detailed nodes assignments info. It should give you a clue what's going on with the second value. My guess is that the second value dosen't do anything and/or doesn't need at all.
gowrishankar_h
Participant
Posts: 42
Joined: Wed Dec 26, 2012 1:13 pm

Re: Grid compute node

Post by gowrishankar_h »

Hi All,
Did you found the answer... even i have same question why we use for second value?
ureddy
Premium Member
Premium Member
Posts: 36
Joined: Wed May 14, 2008 1:59 pm

Post by ureddy »

12,3 ---> 12 nodes but restrict to 3 servers.

Smaple config for 6,3 is below.

Code: Select all

<Dynamic_grid.sh>Information: Generated Dynamic Grid Parallel Configuration File (TotalPartitionsRule=cores, Resource Disk Allocation=all, Scratch Disk Allocation=all):
{
 node "Conductor"
 {
  fastname "SERVERNAME.com"
  pools "conductor"
  resource disk "/opt/IBM/dev/InformationServer/Server/Datasets" {pools ""}
  resource scratchdisk "/tmp" {pools ""}
 }
 node "Compute1"
 {
  fastname "SERNAME1.localhost"
  pools ""
                resource disk "/iis_dev_dataset2" {pools ""}
                resource disk "/iis_dev_dataset3" {pools ""}
                resource disk "/iis_dev_dataset4" {pools ""}
                resource disk "/iis_dev_dataset5" {pools ""}
                resource disk "/iis_dev_dataset6" {pools ""}
                resource disk "/iis_dev_dataset7" {pools ""}
                resource disk "/iis_dev_dataset8" {pools ""}
                resource disk "/iis_dev_dataset1" {pools ""}
                resource scratchdisk "/iis_scratch02" {pools ""}
                resource scratchdisk "/iis_scratch03" {pools ""}
                resource scratchdisk "/iis_scratch04" {pools ""}
                resource scratchdisk "/iis_scratch01" {pools ""}
 }
 node "Compute2"
 {
  fastname "SERNAME1.localhost"
  pools ""
                resource disk "/iis_dev_dataset3" {pools ""}
                resource disk "/iis_dev_dataset4" {pools ""}
                resource disk "/iis_dev_dataset5" {pools ""}
                resource disk "/iis_dev_dataset6" {pools ""}
                resource disk "/iis_dev_dataset7" {pools ""}
                resource disk "/iis_dev_dataset8" {pools ""}
                resource disk "/iis_dev_dataset1" {pools ""}
                resource disk "/iis_dev_dataset2" {pools ""}
                resource scratchdisk "/iis_scratch03" {pools ""}
                resource scratchdisk "/iis_scratch04" {pools ""}
                resource scratchdisk "/iis_scratch01" {pools ""}
                resource scratchdisk "/iis_scratch02" {pools ""}
 }
 node "Compute3"
 {
  fastname "SERNAME2.localhost"
  pools ""
                resource disk "/iis_dev_dataset4" {pools ""}
                resource disk "/iis_dev_dataset5" {pools ""}
                resource disk "/iis_dev_dataset6" {pools ""}
                resource disk "/iis_dev_dataset7" {pools ""}
                resource disk "/iis_dev_dataset8" {pools ""}
                resource disk "/iis_dev_dataset1" {pools ""}
                resource disk "/iis_dev_dataset2" {pools ""}
                resource disk "/iis_dev_dataset3" {pools ""}
                resource scratchdisk "/iis_scratch04" {pools ""}
                resource scratchdisk "/iis_scratch01" {pools ""}
                resource scratchdisk "/iis_scratch02" {pools ""}
                resource scratchdisk "/iis_scratch03" {pools ""}
 }
 node "Compute4"
 {
  fastname "SERNAME2.localhost"
  pools ""
                resource disk "/iis_dev_dataset5" {pools ""}
                resource disk "/iis_dev_dataset6" {pools ""}
                resource disk "/iis_dev_dataset7" {pools ""}
                resource disk "/iis_dev_dataset8" {pools ""}
                resource disk "/iis_dev_dataset1" {pools ""}
                resource disk "/iis_dev_dataset2" {pools ""}
                resource disk "/iis_dev_dataset3" {pools ""}
                resource disk "/iis_dev_dataset4" {pools ""}
                resource scratchdisk "/iis_scratch01" {pools ""}
                resource scratchdisk "/iis_scratch02" {pools ""}
                resource scratchdisk "/iis_scratch03" {pools ""}
                resource scratchdisk "/iis_scratch04" {pools ""}
 }
 node "Compute5"
 {
  fastname "SERNAME3.localhost"
  pools ""
                resource disk "/iis_dev_dataset6" {pools ""}
                resource disk "/iis_dev_dataset7" {pools ""}
                resource disk "/iis_dev_dataset8" {pools ""}
                resource disk "/iis_dev_dataset1" {pools ""}
                resource disk "/iis_dev_dataset2" {pools ""}
                resource disk "/iis_dev_dataset3" {pools ""}
                resource disk "/iis_dev_dataset4" {pools ""}
                resource disk "/iis_dev_dataset5" {pools ""}
                resource scratchdisk "/iis_scratch02" {pools ""}
                resource scratchdisk "/iis_scratch03" {pools ""}
                resource scratchdisk "/iis_scratch04" {pools ""}
                resource scratchdisk "/iis_scratch01" {pools ""}
 }
 node "Compute6"
 {
  fastname "SERNAME3.localhost"
  pools ""
                resource disk "/iis_dev_dataset7" {pools ""}
                resource disk "/iis_dev_dataset8" {pools ""}
                resource disk "/iis_dev_dataset1" {pools ""}
                resource disk "/iis_dev_dataset2" {pools ""}
                resource disk "/iis_dev_dataset3" {pools ""}
                resource disk "/iis_dev_dataset4" {pools ""}
                resource disk "/iis_dev_dataset5" {pools ""}
                resource disk "/iis_dev_dataset6" {pools ""}
                resource scratchdisk "/iis_scratch03" {pools ""}
                resource scratchdisk "/iis_scratch04" {pools ""}
                resource scratchdisk "/iis_scratch01" {pools ""}
                resource scratchdisk "/iis_scratch02" {pools ""}
 }
}
Last edited by ureddy on Thu Dec 25, 2014 11:02 am, edited 1 time in total.
Reddy
ureddy
Premium Member
Premium Member
Posts: 36
Joined: Wed May 14, 2008 1:59 pm

Post by ureddy »

correction to the above post...
12,3 ---> 12 tasks but restrict to run on only 3 nodes.
Reddy
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

:!: Best when you post stuff like this to remove any client-specific information. And don't post a correction, go back and edit the original post.
-craig

"You can never have too many knives" -- Logan Nine Fingers
Post Reply