DSXchange

Hi All, I have a job reading a dataset, with many stages preceding a Modify stage going into a lookup stage, all of which have RCP enabled, except the modify stages before the lookup stage. There is a parallel flow which is funneled with the output of the lookup stage above. This stream also uses RC...

Hi, Our environment has one environment hosting the services and metadata tiers. We then have two separate environments hosting an engine tier each. When trying to install a third engine tier on a new environment a failure occurs. Completed the following the manual uninstall (https://www.ibm.com/sup...

Confirmed with IBM that there is hard limit of 100 ISD jobs that can run concurrently per engine.

To work around this we are implementing a multi engine solution and load balancing request across them.

yeah I set the workload management to start up only 3 jobs every 30 seconds otherwise things do not go well.

Well the jobs are very light weight. EE but running on a single node. The reason for creating so many instances is that even though the job is light weight there is a REST call made by a hierarchical stage in the job that takes anything from 5-12sec to respond. As we need to process a single request...

Well for example now we are testing a workload of ~360 request per minute which we are attempting to handle with three applications created in Information Services Director. The applications are configured with 45, 45 and 10 instances as the min and max. We may be able to reduce the 100 instances do...

Hi Ernie, The high water mark we estimated to be at 200-300. The console is a little confusing that it is under the tab "Queue Management", but the parameter there that I want to change is "Maximum Running Jobs". It is currently set to 100 and when trying to change it get the res...

Hi Paul,

Yeah that is for system wide, which I have set to 200.

What I need to be able to do is modify the max limit on ISD jobs under Queue Management.

Thanks,

Shaun

Hi, If you look at operations console -> Workload Management -> Queue Management There are "Maximum Running Jobs" for various queues. The parameter for the ISD queue is set at 100, but it is not configurable (at least in the console) like the others. Anyone know how to change the value? Th...

We make use of RGGmanager on Red Hat Linux to control the fail over of our Data Stage tiers. We had a situation where the "IBM/InformationServer/HAScripts/InfoSvrServices status" called via RGManager returned a non 0 error code. Causing the DataStage services tier to fail over. At the poin...

Hi, Anyone know if there is any limitation on the number of open connections that can be made to an ISD service via the WAS? Also is there anyways to monitor the number of unserviced requests waiting to be served up to the ISD job instances? From what I understand is that these requests would sit in...

Have being making use of CURL and a shell script to run my test harness. Have settled now on the number of instances that will be needed. I have being making use of operations console to monitor resources during my testing and notice that currently I am limited by memory and not CPU. This kinda make...

hahaha Ernie I believe I am the poster you are referring to w.r.t to the HTTP plumbing issue I had :) Currently in our DataStage test environment I have the ISD service to generate 3 instances of a job "ready and waiting" to process requests. The job is simply put ISD_Input --> Hierachial ...

Hi, We have a lightweight ISD job that run for 4-5 seconds. The majority of this time is spent waiting for a response from an application request sent via a REST step from the hierarchical stage. We are expected to handled +- 300 000 calls to this job a day. I have asked the question around peak wor...

Okay we have changed the plumbing a bit and the architecture now is: apache httpd -> Datastage -> HTTP Forwarder -> apache tomcat loadbalancer -> apache tomcat worker Seems in this configuration Apache is now handling the keep alives and re-opening of connections to tomcat. Do not understand it 100%...