DSXchange: DataStage and IBM Websphere Data Integration Forum
View next topic
View previous topic
Add To Favorites
Author Message
mouthou
Participant



Joined: 04 Jul 2004
Posts: 180

Points: 1756

Post Posted: Sun Jul 15, 2018 10:35 am Reply with quote    Back to top    

DataStage® Release: 8x
Job Type: Parallel
OS: Unix
Hi All,

When there are 4 nodes in a config file and when the job is run, is there a way to find out which node played as the conductor node out of those 4 nodes.

It may be possible if the node has defined the pool as 'conductor' in the config file. But if there is only default pool, what is the way of knowing the chosen node as conductor node name (like node1 or node2 etc). Can it found from the Director logs while the job runs? Appreciate your response.

Thanks
PaulVL



Group memberships:
Premium Members

Joined: 17 Dec 2010
Posts: 1249

Points: 8216

Post Posted: Mon Jul 16, 2018 11:10 am Reply with quote    Back to top    

Paste your APT config file, change the actual hostnames and paths if you wish.


What are you researching that makes you look for the "conductor" hostname? Are you wondering which host is reading your sequential files?
Rate this response:  
Not yet rated
mouthou
Participant



Joined: 04 Jul 2004
Posts: 180

Points: 1756

Post Posted: Wed Jul 18, 2018 6:18 am Reply with quote    Back to top    

I am not rresearching as such. It was a general query as to know which node was picked as the conductor node by the engine when the job runs. If there was a node pool defined with keyword "conductor", then we would know the node easily. But if there is only default pool, how we come to know the conductor node. Is there a place where we can see that node which acted as the conductor
Rate this response:  
Not yet rated
ray.wurlod

Premium Poster
Participant

Group memberships:
Premium Members, Inner Circle, Australia Usergroup, Server to Parallel Transition Group

Joined: 23 Oct 2002
Posts: 54283
Location: Sydney, Australia
Points: 294429

Post Posted: Wed Jul 18, 2018 7:42 pm Reply with quote    Back to top    

By default it's the first-named node in the default node pool.

_________________
RXP Services Ltd
Melbourne | Canberra | Sydney | Hong Kong | Hobart | Brisbane
currently hiring: Canberra, Sydney and Melbourne
Rate this response:  
mouthou
Participant



Joined: 04 Jul 2004
Posts: 180

Points: 1756

Post Posted: Thu Jul 19, 2018 10:33 am Reply with quote    Back to top    

Thanks Ray and Paul.

I am not quite clear on the response with the references of named node and default pool, both of which are mutually exclusive as far as my understanding goes. ie, if there is named node then it cannot be called as default pool. Please correct if it is not the case.

If there is named node I thought the conductor node can be identified easily. Rather my query is with the default definition of everything in the configuration file like [pools " "] etc under each node.

Based on some read materials on this , I think the chosen conductor node can be seen in the job run log. There is some log entry which gives the list of values assigned to env variables. May be the variable APT PM CONDUCTOR NAME gives some idea. I don't have access to Datastage being between jobs! Otherwise this can be verified.
Rate this response:  
Not yet rated
ray.wurlod

Premium Poster
Participant

Group memberships:
Premium Members, Inner Circle, Australia Usergroup, Server to Parallel Transition Group

Joined: 23 Oct 2002
Posts: 54283
Location: Sydney, Australia
Points: 294429

Post Posted: Sat Jul 21, 2018 4:42 am Reply with quote    Back to top    

All nodes must be named. The nodes that are in the default pool are those for which the node pool name "" occurs. This does not prevent the node from being named in other pools as well. Best p ...

_________________
RXP Services Ltd
Melbourne | Canberra | Sydney | Hong Kong | Hobart | Brisbane
currently hiring: Canberra, Sydney and Melbourne
Rate this response:  
Not yet rated
mouthou
Participant



Joined: 04 Jul 2004
Posts: 180

Points: 1756

Post Posted: Sun Jul 22, 2018 6:08 am Reply with quote    Back to top    

Thanks Ray. I thought to clarify the question with 2 samples of config files.

config1:

Code:
node1
       {
         fastname dsserver
         pools ""
         ..........
         ...........
       }
node2
       {
         fastname dsserver
         pools "conductor"
         ..........
         ...........
       }


config2:

Code:
node1
       {
         fastname dsserver
         pools ""
         ..........
         ...........
       }
node2
       {
         fastname dsserver
         pools ""
         ..........
         ...........
       }



As we know, in config1 we can easily say node2 is the conductor node. But in config2 sample, we cannot identify that easily. But based on your responses for config2 samples, can I assume that the first node 'node1' will be picked by the engine? And after running the job, where can we witness for such auto picking was my query.

[Note: inserted code blocks / formatting to make it easier to read - Andy]
Rate this response:  
Not yet rated
ray.wurlod

Premium Poster
Participant

Group memberships:
Premium Members, Inner Circle, Australia Usergroup, Server to Parallel Transition Group

Joined: 23 Oct 2002
Posts: 54283
Location: Sydney, Australia
Points: 294429

Post Posted: Mon Jul 23, 2018 9:55 pm Reply with quote    Back to top    

That's correct. Your nodes are named node1 and node2. The first-named node in the default node pool is node1. DataStage itself does not use node names; these are purely for human consumption. ...

_________________
RXP Services Ltd
Melbourne | Canberra | Sydney | Hong Kong | Hobart | Brisbane
currently hiring: Canberra, Sydney and Melbourne
Rate this response:  
Not yet rated
chulett

Premium Poster


since January 2006

Group memberships:
Premium Members, Inner Circle, Server to Parallel Transition Group

Joined: 12 Nov 2002
Posts: 42662
Location: Denver, CO
Points: 219709

Post Posted: Tue Jul 24, 2018 7:22 am Reply with quote    Back to top    

mouthou wrote:
As we know, in config1 we can easily say node2 is the conductor node.

Apparently, not so easily. Wink

Ray is saying in both cases that node1 is the conductor node.

_________________
-craig

And I'm hovering like a fly, waiting for the windshield on the freeway...
Rate this response:  
Not yet rated
ray.wurlod

Premium Poster
Participant

Group memberships:
Premium Members, Inner Circle, Australia Usergroup, Server to Parallel Transition Group

Joined: 23 Oct 2002
Posts: 54283
Location: Sydney, Australia
Points: 294429

Post Posted: Wed Jul 25, 2018 1:24 am Reply with quote    Back to top    

Actually I'm not. I was referring specifically to the configuration file that has ONLY a default node pool. Where there is a "conductor" node pool, that will take precedence. So: in confi ...

_________________
RXP Services Ltd
Melbourne | Canberra | Sydney | Hong Kong | Hobart | Brisbane
currently hiring: Canberra, Sydney and Melbourne
Rate this response:  
Not yet rated
chulett

Premium Poster


since January 2006

Group memberships:
Premium Members, Inner Circle, Server to Parallel Transition Group

Joined: 12 Nov 2002
Posts: 42662
Location: Denver, CO
Points: 219709

Post Posted: Wed Jul 25, 2018 5:52 am Reply with quote    Back to top    

Thanks for the clarification... for some reason (mostly since that point hadn't been explicitly mentioned yet) I didn't think it was as simple as calling a pool "conductor" and apparently it is. Been far too long. Embarassed

_________________
-craig

And I'm hovering like a fly, waiting for the windshield on the freeway...
Rate this response:  
Not yet rated
Display posts from previous:       

Add To Favorites
View next topic
View previous topic
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum



Powered by phpBB © 2001, 2002 phpBB Group
Theme & Graphics by Daz :: Portal by Smartor
All times are GMT - 6 Hours