Intermitent SFTP Pull Job Failure

[chanaka]

Hi DataStage Experts,

I have a SFTP parallel job which pulls source files connecting to a different server and placing them in the DataStage server. Job is triggered in multiple instances. Each instance will pick up a different source file based on the location folder name (with a different invocation id). Upon execution of 40-50 instances, it will fail randomly once or twice with "Fatal Error: A descriptor must be non-negative" error message. When we rerun the job, it will get executed without any issues. And the invocation, where it fails is totally random (i.e. one time it fails for file A, next time it will fail for file B). Tried to raise this issue with the DS administrator. However he is not able to raise a PMR for this as it is not possible to reproduce the issue consistently.

I am wondering whether any of you faced to the same issue before.

Any help would be very much appreciated.

Cheers!

Chanaka

[FranklinE]

That error message is frustrating, as I'm sure you already know. I can offer my experience, but the only way to be sure is to have someone actively monitoring to catch it happening. The other option, not popular, is to set the logging to verbose.

Our system has a single FTP portal, through which we get many files in separate sessions. The server is set to passive mode, which restricts the number of data ports available for data transfer. When a job attempts to run with no data ports available, it will abend on equally frustrating error messages such as "file doesn't exist".

Engage your shop's FTP owners. The key thing here is that you can rerun without further intervention, and that (to me) indicates an issue at the server level, not in DataStage.

Good luck.

[PaulVL]

you can alias SFTP within your user profile to be "alias sftp='sftp -v'" that should enable the verbose for you and not others.

[chanaka]

Hi Franklin, Paul,

Thanks a lot for the replies. Will try use the verbose mode and see whether we can get some traces.

Cheers!

Chanaka