Jobs Getting Failed Due to Lack of Sessions

Rubu · Post by **Rubu** » Wed Oct 03, 2007 7:49 pm

Hi All

We have developed around 400+ jobs and trying to schedule them using CONTROL-M. Almost every next job is using at least 1 Teradata EE stage and requesting for multiple FEXPORT or FLOAD session from Teradata.

Hundreds of jobs are getting triggered at the same time and failing due to non availability of enough Teradata session. [I guess there is limitation of maximum number of sessions (~15) from Teradata]

Now how should we approach to deal with this situation?

1. Can we apply some condition at DS Sever level to restrict the number of session requested ?
2. Or, it should be handled at the CONTROL M schedule level?

What is the best practice in the industry?

Note that jobs are being called through Sun Grid Engine.

Regards

chulett · Post by **chulett** » Wed Oct 03, 2007 8:15 pm

I'm guessing these aren't 400+ DataStage TX jobs.

1. No.
2. Yes.

In other words, your *scheduler* needs to understand the limitations of your system and not overwhelm it. If that means you need to 'teach' it or lead it by the nose so it doesn't run 'too many' jobs at the same time, then that's what you need to do.

Are you using Sequence jobs or some kind of internal job control or does Control-M run every job individually? Proper use of job control can change the answer to #1.

Lastly, I'm sure that Teradata 'limit' is configurable. Not that you'd want to up it too far or it sounds like you'll just bury it.

Rubu · Post by **Rubu** » Wed Oct 03, 2007 8:59 pm

1. No

These are PX jobs designed to load data.

Teradata does have a restriction of around 15 sessions at a time, but since the jobs are running through a Grid, expectation was, if requirement crosses 15 the jobs remain in queue-- does not fail.

There are no Sequence job and jobs are being configured through a Process Control System (Scripts+Metadata tables)

I guess its time to teach the scheduler to "lead it by nose" and hope our schedulers can achieve that by changing some predefined parameters in CONTROL-M.

Hope this is the best practice followed in the industry.

Thanks a ton for your reply

chulett · Post by **chulett** » Wed Oct 03, 2007 9:04 pm

Not that familiar with a true Grid implementation, so perhaps not everything I've said applies. Others, however, are - hopefully they'll pop in here with more grid-specific Words of Wisdom.

ray.wurlod · Post by **ray.wurlod** » Wed Oct 03, 2007 10:32 pm

Moderator: please move to parallel forum