Strange Behaviour in Loop

Post questions here relative to DataStage Server Edition for such areas as Server job design, DS Basic, Routines, Job Sequences, etc.

Moderators: chulett, rschirm, roy

Post Reply
palmeal
Participant
Posts: 122
Joined: Thu Oct 14, 2004 7:56 am
Location: Edinburgh, Scotland

Strange Behaviour in Loop

Post by palmeal »

I have a Job Sequence where I FTP files to another server. The FTP activity is within a StartLoop...EndLoop activity which is set for 1 to 3. i.e. if the first FTP attempt fails then the process will go round the loop and retry it 3 times. If the FTP fails on the 3rd attempt then it should follow the fail path as seen in the flow below. The trigger on the FTP activity is set to Failure/Otherwise.
What actually happens is that after the 1st FTP failure the code goes down the Fail path and does not go round the loop.

<------------------
S.L. -> FTP-----E.L. --> Fail after 3rd attempt
.............\
..............Continue on success

To try and ascertain why this was happening I stuck in a BASIC Routine (see flow below) to pull more information. I ran this with Ans = 0, Return (Ans) and no other code and the code started to go round the loop and not down the Fail path until after the 3rd failure.

<-----------------------
S.L. -> FTP---B.R.--E.L. --> Fail after 3rd attempt
.............\
..............Continue on success

Can anyone explain why this would be ? If I have to leave in a Basic Routine that does nothing to get this working then I will but there's just something wrong about this.
There are only 10 kinds of people in the world, those that understand binary and those that don't.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Seems pretty straight-forward to me. The failure trigger on the FTP stage fires when that job fails - the first time, the third time, anytime it fails. How is it supposed to know that you only really want it to pay attention after three trips around the block? :?

Change your logic around. Exit from the loop on the first success, only keep going 'round if the job fails. That way you know if it comes out the end of the loop it must have failed three times to get there.
-craig

"You can never have too many knives" -- Logan Nine Fingers
palmeal
Participant
Posts: 122
Joined: Thu Oct 14, 2004 7:56 am
Location: Edinburgh, Scotland

Post by palmeal »

Thanks for the answer but I do exit the loop on success which is usually on iteration 1 of 3. The FTP Activity only stays in the loop on Failure "otherwise" it goes down the "Continue on success" path.

What I am trying to say is that when the FTP does fail the loop is exited down the "Fail" path on the first iteration - it does not burl round 3 times but if I add in a dummy Basic Routine between the FTP and EndLoop Activities then it does go round the loop before exiting down the "Fail" path.
There are only 10 kinds of people in the world, those that understand binary and those that don't.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Ok. Must be something specific in the way the triggers are setup, then. Your loop is just a numeric loop, set to iterate From: 1 Step: 1 To: 3?

There is an example in the Designer's Guide, something similar to what you are trying to do. Check the Job Sequences chapter for the Examples of Using Loop Stages section. Perhaps something in there will shed some light on what might be going on here?
-craig

"You can never have too many knives" -- Logan Nine Fingers
palmeal
Participant
Posts: 122
Joined: Thu Oct 14, 2004 7:56 am
Location: Edinburgh, Scotland

Post by palmeal »

Yes my loop is as simple as you have stated.

The example in the Designers Guide is the exact scenario that I am looking at. Referring to this example, if I took the waitabit routine out of the job sequence and piped the failure trigger from server job "networkwrite" then the code would not go round the loop but would follow DSLink7 after the first failure.

I have other jobs that stretch loops to their limits that all work fine - the difference with them is that there is only one path to follow and no Failure Triggers.

I'm just going to go with the dummy Basic Routine in my code - it doesn't hurt I suppose but does not logically make sense that one method works and the other doesn't.

Thanks for your help. Maybe I should raise this as a case.
There are only 10 kinds of people in the world, those that understand binary and those that don't.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Couldn't hurt to open an issue with support, they should be able to either provide an explanation or deem it a problem. In either case, can you post back with whatever you find out?
-craig

"You can never have too many knives" -- Logan Nine Fingers
palmeal
Participant
Posts: 122
Joined: Thu Oct 14, 2004 7:56 am
Location: Edinburgh, Scotland

Post by palmeal »

I will post back the response that I get - maybe a few days before I can do this though as I am on a two day week this week. Thanks for your help.
There are only 10 kinds of people in the world, those that understand binary and those that don't.
palmeal
Participant
Posts: 122
Joined: Thu Oct 14, 2004 7:56 am
Location: Edinburgh, Scotland

Post by palmeal »

I tried to create a simple loop Job Sequence that I could send to Support that mimicked the behaviour described above rather than send the entire Job Sequence that I have(had). Each example that I created worked differently (and as expected) contrary to the problem I have described. I can only imagine that a combination of other activities within the Job Sequence have caused this issue. One thing worth noting is that I had a loop within a loop and it was the inner loop that was causing the problem. Again in my attempt to create a simple Job Sequence I did try a nested loop but this also worked.

I can't send the original Job Sequence to Support as I had to change it to get things working - I had to remove the inner loop as DataStage doesn't seem to support this use of loops in combination with multiple pathways with paths merging back into common Sequencer Activities. This may have caused a knock on effect that could explain the behaviour I was seeing.

So to cut a long story short - this to me is no longer an issue as I have worked around it by other means.
There are only 10 kinds of people in the world, those that understand binary and those that don't.
Post Reply