DSXchange: DataStage and IBM Websphere Data Integration Forum
View next topic
View previous topic
Add To Favorites
Author Message
tine_bi
Participant



Joined: 24 Nov 2014
Posts: 18

Points: 304

Post Posted: Mon Feb 26, 2018 5:20 am Reply with quote    Back to top    

DataStage® Release: 11x
Job Type: Parallel
OS: Windows
Hi

We currently have a server setup with 2 cpu for our prod enviroment.
We now look at virtual server setup with
2 sockets and 32 virtual processors.
Drives: E, F

Our current node.apt
Code:
{
   node "node1"
   {
      fastname "DSsrv1"
      pools ""
      resource disk "E:/Datasets" {pools ""}
      resource scratchdisk "F:/Scratch" {pools "sort"}
      resource scratchdisk "E:/Scratch" {pools "sort"}
      resource scratchdisk "F:/Buffer" {pools "buffer"}
      resource scratchdisk "E:/Buffer" {pools "buffer"}
   }
   node "node2"
   {
      fastname "DSsrv1"
      pools ""
      resource disk "E:/Datasets" {pools ""}
      resource scratchdisk "E:/Scratch" {pools "sort"}
      resource scratchdisk "F:/Scratch" {pools "sort"}
      resource scratchdisk "E:/Buffer" {pools "buffer"}
      resource scratchdisk "F:/Buffer" {pools "buffer"}

   }
}

This works under current settings. But should we change this in a virtual setting, and if so to what?

Appreciate any input as this is unknown terrain for me.

BR
Dan

_________________
BR
Dan
PaulVL



Group memberships:
Premium Members

Joined: 17 Dec 2010
Posts: 1232

Points: 8099

Post Posted: Mon Feb 26, 2018 4:18 pm Reply with quote    Back to top    

Don't forget to add pool "" to your scratchdisk entries.

Your quantity of logical nodes does not need to match the quantity of physical CPUs (or virtual CPUs).

You could have your node.apt file with 8, 16, 32, etc... entries, not just 2.

Performance would take a hit the higher you go, but you COULD do it.

You have to balance degrees of parallelism (your 2 nodes) multiplied by the quantity of concurrent jobs, factoring in the complexity of the jobs, in order to get a sense of how much you will stress your itty bitty PROD box.

Remember that your virtual box sounds bigger than your physical, you may have to see how that affects your licensing.
Rate this response:  
Not yet rated
tine_bi
Participant



Joined: 24 Nov 2014
Posts: 18

Points: 304

Post Posted: Tue Feb 27, 2018 3:04 am Reply with quote    Back to top    

PaulVL wrote:
Don't forget to add pool "" to your scratchdisk entries.


So I should have it like this then

resource scratchdisk "E:/Scratch" {pools "" "sort"}

After reading several posts and articles I now will test having 4 entries not just 2.

_________________
BR
Dan
Rate this response:  
Not yet rated
PaulVL



Group memberships:
Premium Members

Joined: 17 Dec 2010
Posts: 1232

Points: 8099

Post Posted: Tue Feb 27, 2018 3:10 pm Reply with quote    Back to top    

Are you having issues with the duration of your job?
Rate this response:  
Not yet rated
tine_bi
Participant



Joined: 24 Nov 2014
Posts: 18

Points: 304

Post Posted: Wed Feb 28, 2018 5:03 am Reply with quote    Back to top    

Yes, we experience that we now need to expand the node.apt file to have 4 nodes, not just two.

And I now want to understand why Smile


Dan

_________________
BR
Dan
Rate this response:  
Not yet rated
ray.wurlod

Premium Poster
Participant

Group memberships:
Premium Members, Inner Circle, Australia Usergroup, Server to Parallel Transition Group

Joined: 23 Oct 2002
Posts: 54198
Location: Sydney, Australia
Points: 293951

Post Posted: Thu Mar 01, 2018 8:57 pm Reply with quote    Back to top    

I would argue that, in a Development environment, you only ever need a one-node and a two-node configuration file. The only exception is where you need to develop with node pools. If a job runs on t ...

_________________
RXP Services Ltd
Melbourne | Canberra | Sydney | Hong Kong | Hobart | Brisbane
currently hiring: Canberra, Sydney and Melbourne
Rate this response:  
Not yet rated
Display posts from previous:       

Add To Favorites
View next topic
View previous topic
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum



Powered by phpBB © 2001, 2002 phpBB Group
Theme & Graphics by Daz :: Portal by Smartor
All times are GMT - 6 Hours