WAS Connections for ISD and Operation Queue Monitoring

Dedicated to DataStage and DataStage TX editions featuring IBM<sup>®</sup> Service-Oriented Architectures.

Moderators: chulett, rschirm

Post Reply
pbttbis
Premium Member
Premium Member
Posts: 36
Joined: Thu Dec 11, 2014 3:30 am
Location: South Africa
Contact:

WAS Connections for ISD and Operation Queue Monitoring

Post by pbttbis »

Hi,

Anyone know if there is any limitation on the number of open connections that can be made to an ISD service via the WAS?

Also is there anyways to monitor the number of unserviced requests waiting to be served up to the ISD job instances? From what I understand is that these requests would sit in the operations queue, but I am unsure where to monitor the state of it.

Thanks,

Shaun
PBT TBIS Consultant
eostic
Premium Member
Premium Member
Posts: 3838
Joined: Mon Oct 17, 2005 9:34 am

Post by eostic »

no way to monitor requests active in the queues....there are several places where requests might queue up.......the operations queue and the pipeline buffer and all of the requests that are concurrently inside a single job instance.

better to understand why you are asking......what is the issue you are having?

Ernie
Ernie Ostic

blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
pbttbis
Premium Member
Premium Member
Posts: 36
Joined: Thu Dec 11, 2014 3:30 am
Location: South Africa
Contact:

Post by pbttbis »

We make use of RGGmanager on Red Hat Linux to control the fail over of our Data Stage tiers.

We had a situation where the

"IBM/InformationServer/HAScripts/InfoSvrServices status" called via RGManager returned a non 0 error code. Causing the DataStage services tier to fail over.

At the point the fail over was triggered we had processed roughly 6000 requests in 2 hours successfully that were served up to two ISD jobs each configured to have 5 instances running. The external services that we call from the ISD job via hierarchical stage was starting to respond slower which means we would most likely have being backing up requests "somewhere". The ISD jobs are all running on a single engine node configuration with the stages set to sequential. The ISD service is scheduled to handle 5 requests per instance (i believe this is the pipelining size).

So my thinking is that we may have possibly hit some sort of threshold when it comes to number of open connection (unserviced requests) which may have caused the non 0 error code from the status script. Possibly timeout or maybe some memory limit for keeping the requests in the queue.

I can see in information server console under session management -> Active Session there is a field called "Current Sessions", but I don't think that has anything to do with unserviced requests.

Looking at the WAS console if I go to Container Services -> Transaction Service I see various timeout fields. Namely:

1) Total transaction life time timeout
2) Async response timeout
3) Client inactivity timeout
4) Maximum transaction timeout

For 1, 3 and 4 these are very large 30min+.

Not sure if the async timeout is applicable here as we may be going over 30 seconds. Either way I would expect the request to be timed out cleanly with no error being thrown by InfoSvrServices status

The ISD jobs do generate 1 warning per request, which is another idea I am investigating. Not sure if this warning being logged indefinitely for each request on an "always on" ISD job may have caused it.

I am working the issue with IBM via a PMR but the person I am dealing with does not seems to know where/how I can monitor the operations queue.
PBT TBIS Consultant
qt_ky
Premium Member
Premium Member
Posts: 2895
Joined: Wed Aug 03, 2011 6:16 am
Location: USA

Post by qt_ky »

I once tried asking IBM Support similar ISD questions and they kept recommending to purchase Lab Services. I hope you have better luck.
Choose a job you love, and you will never have to work a day in your life. - Confucius
Post Reply