Problem with logging into Director - Project locked

vivekgadwal · Post by **vivekgadwal** » Thu Sep 02, 2010 11:24 am

Hello all,

Today, we had a problem with logging into Production environment to check on some processes that we are running. The issue was that, when we try to log-in, it says that the Project is locked. Not just this project, we were not able to log-in to any project on Prod.
The irony is, one - and only one - person can log into the project. If everybody is off it, then I am able to log-in. It automatically disappeared after the process that we are running has finished.

I also noticed a number (more than 100) of job locks (LIST.READU) present. I am not sure how to check when that lock is opened, but they are there when this process was running. I am assuming this process opens it because it is one humongous sequence job which runs a lot of jobs which load into our Data mart. This process runs every month-end (is being done for the past 2 years) without this kind of an issue before.

Is this job locks related to the problem we faced (inability to log-on to a project)? I checked uvconfig and found that GLTABZ and RLTABZ is set at 150, if this helps. I am not completely aware of what those are, but I changed them once before in a different shop and seems like these are some lock related values and so I posted it here as an FYI.

Thanks in advance for any advice that you can provide.

chulett · Post by **chulett** » Thu Sep 02, 2010 3:38 pm

Out of curiousity, did you take your actual error message and do an exact search for it here? I know it has been discussed a number of times in the past, seems like those discussions would be of some help to you.

vivekgadwal · Post by **vivekgadwal** » Fri Sep 03, 2010 7:27 am

Craig,

I did search here and found some interesting topics, but I couldn't resolve with that knowledge. The issue was, one person at a time is able to log-in and others were not able to. Right now, it is not a problem (we are able to log-in to the Director), but it got me very curious as to the relationship between the job runs (and the numerous locks it opened) vs logging into Director/any other DataStage component.

I am concerned that this might happen again when the month-end run occurs and I want to be a little proactive in avoiding the same problem again.

I would really appreciate any help regarding this. As usual, thank you for the reply.

vivekgadwal · Post by **vivekgadwal** » Thu Sep 16, 2010 7:34 am

This problem re-surfaced today again. I was looking at the processes and I see old "dsapi slave" connections still lying around. I terminated the processes spawned by our generic log-in and then tried to log-in again and I was able to do that successfully.

This makes me suspect that DataStage is counting those processes open as active sessions and it probably is exceeding the license limit. However, I am under the impression that licenses are provided to the number of users to log-in, not to number of active sessions. This is really strange!!

Any suggestions

P.S: I enabled the dsdlockd process in Development, not in other areas. This problem is in a different area unfortunately.

chulett · Post by **chulett** » Thu Sep 16, 2010 8:10 am

Project Locked is completely different from a license exceeded issue, the latter will simply tell you that. The former is a resource issue with the lock table from disconnects and crashes (typically) and needs to be cleaned up, either manually or automagically, as you have been doing.

vivekgadwal · Post by **vivekgadwal** » Thu Sep 16, 2010 8:26 am

Ahh...thanks. I had a feeling that my theory about the licenses is wrong. The problem that needs to be fixed is, why is it leaving so many of those "dsapi slave" processes there. I am hoping that the "dsdlockd" will fix that. I cannot enable that right away, unfortunately.

Thanks for your help.

vivekgadwal · Post by **vivekgadwal** » Thu Sep 16, 2010 8:52 am

Question: Would it make a difference if we close the director/designer instances by clicking the red "X" mark on the window versus closing them using "Project>Exit"? Which will be a cleaner exit, if there is a difference?

I apologize if it sounds dumb...

kumar_s · Post by **kumar_s** » Thu Sep 16, 2010 9:26 am

Both would exit the Client and close the client connection.
So shouldnt really make any difference.
But did you find any?

vivekgadwal · Post by **vivekgadwal** » Thu Sep 16, 2010 9:41 am

I did not know there was a difference and I was assuming the same. But, my boss says that the latter is a graceful exit and hitting the red "X" is not. He was so confident in this that he sent out an e-mail notification to the whole team to do the "Project/Exit" way of exiting. That is why I am posting this question here to find out if there is a difference :D

chulett · Post by **chulett** » Thu Sep 16, 2010 12:26 pm

It won't make any difference, they are equivalent. The problem should only be happening if your connection "breaks", takes a less than graceful exit say via an End Task or a crash of the application. Otherwise it really shouldn't be happening on any kind of a regular basis. Well, unless you have connectivity issues on a regular basis.

vivekgadwal · Post by **vivekgadwal** » Thu Sep 16, 2010 12:32 pm

Connectivity issues...may be...I can think of a scenario (logging from home networks) why. Let me make sure about that with people.

This is great input Craig. Thanks...

kishore2456 · Post by **kishore2456** » Tue Sep 21, 2010 5:00 am

Hi Vivek,

I think the point you made in the first post was correct to some extent.

You can refer to
http://www-01.ibm.com/support/docview.w ... wg21318249

chulett · Post by **chulett** » Tue Sep 21, 2010 6:46 am

As that article notes: "These failures are due to the lock and open file limits in DSEngine being reached." and while they can get reached by the shear number of people accessing the system at the same time, more tyically these are left-over processes / locks from defunct connections. At least in my experience.

vivekgadwal · Post by **vivekgadwal** » Tue Sep 21, 2010 8:11 am

Thanks Kishore and Craig for your posts.

There are not a large number of users using DataStage here. We are a team of 8, out of which DataStage is very seldom used by 2 of them. So, mostly 6 people will be using DataStage. When this issue is happening, none of them were online. It is when we tried to log-in after some time (processing is still happening then) is the problem occurring. I would tend towards Craig's explanation a little, but we over here are unable to explain these possible network failures. We are monitoring these now.

However, our T30FILES limit is set to 200 only. MFILES is set to 100, GLTABSZ and RLTABSZ are set to 150 each. Would you still recommend increasing T30FILES limit to about 512 or even more?

chulett · Post by **chulett** » Tue Sep 21, 2010 8:58 am

No, not for that small of a number of connected developers I don't see the point. And if your T30FILES setting was too low, you would see specific error messages mentioning that limit.