Job seqencer killing of other process.

A forum for discussing DataStage<sup>®</sup> basics. If you're not sure where your question goes, start here.

Moderators: chulett, rschirm, roy

Post Reply
Childrensmn
Premium Member
Premium Member
Posts: 32
Joined: Mon Apr 12, 2010 3:42 pm
Location: Minneapolis
Contact:

Job seqencer killing of other process.

Post by Childrensmn »

I have a job sequence that does several tables updates and then I use a sequencer to split off 2 task. Task 1 does a set of things (updates tables, updates some files ) and then checks to see if everything in task 1 is complete. At the same time I have Task 2 doing other task (more updates.....), just on a different set of tables and then checks to see if all things in task 2 are done. Both task link back to a single sequencer (sequence 447) that is defined as (all). Which means wait for both threads to be finished and then update the last table. What is happening, task 1 is finishing and goes sequence 5 and then kills task 2 because task 1 is finished.... I can see it in the log.... task one is finished and task 2 shows no errors but just goes away. Ill post the log as a attachment... if I can
UCDI
Premium Member
Premium Member
Posts: 383
Joined: Mon Mar 21, 2016 2:00 pm

Post by UCDI »

so you split to two, one of those goes into another seq, that one is set to any (?) and the outer one is set to all? This would mean that the any one can finish one thing, pass that to the all outer, and the all would be set to true for that side, so if the other side finishes as well, some of the 'anys' could still be going and the 'all' triggered.

Generally you have 2 things that could be going on here.
1- the sequencers are used to split data down a conditional path -- the data goes down one or the other, but not both
or
2 - the data can go down both paths.

if the data can go down both paths, you need an all at the end.
if it only does down 1 path, you want any at the end. (for your situation, this isnt a rule of thumb).

then the outer one would be all, for sure.

the deal is that if your data can't go down one of the paths, that path will never trip and 'all' will cause a problem. Conversely, if it can go down either path, an 'any' lets it trigger when one branch is done even if the other branch is still working. And sometimes it does not matter, but if you want everything piled up before writing, it does.
Childrensmn
Premium Member
Premium Member
Posts: 32
Joined: Mon Apr 12, 2010 3:42 pm
Location: Minneapolis
Contact:

Post by Childrensmn »

Thanks for the feedback. So data can go down 1 or both task. Example would be a list of 1 or 3 tables....... in the case of 1 table task one would get a table and task 2 would check to see if its null and then go to Sequence 447. In the case of 3 tables, task 1 would get a table, task 2 would get a table and then task 1 would pickup the 3rd table and task 2 would check for a null.

Ok, yes I have it set the final sequence set to all, but it will not wait for the other task to finish, it just kills it off.

Below is the log from a 1 table scenario. (yes I know I have not said much about looping...., but that is how I go get the next table)


Item #: 227
Event ID: 226
Timestamp: 2018-06-06 15:01:12
Type: Info
User Name: dsadm
Message Id: IIS-DSTAGE-RUN-I-0019
Message: ETL1_GE_COMMON_FULL.RR.JobControl (@Unixcmd_GetDateValue): Executed: cat /data/ETL1/GE_CHARGE_EXT_PARAMS_1.txt | wc -l | tr -d '\n';
Reply=0
Output from command ====>
0

Item #: 228
Event ID: 227
Timestamp: 2018-06-06 15:01:12
Type: Info
User Name: dsadm
Message Id: IIS-DSTAGE-RUN-I-0019
Message: ETL1_GE_COMMON_FULL.RR.JobControl (DSWaitForJob): Waiting for job GE_ETL1_Chk_SourceTable_GE.GE_CHARGE_THREAD2 to finish

Item #: 229
Event ID: 228
Timestamp: 2018-06-06 15:01:14
Type: Info
User Name: dsadm
Message Id: IIS-DSTAGE-RUN-I-0019
Message: ETL1_GE_COMMON_FULL.RR.JobControl (DSWaitForJob): Job GE_ETL1_Chk_SourceTable_GE.GE_CHARGE_THREAD2 has finished, status = 1 (Finished OK)

Item #: 230
Event ID: 229
Timestamp: 2018-06-06 15:01:14
Type: Info
User Name: dsadm
Message Id: IIS-DSTAGE-RUN-I-0019
Message: ETL1_GE_COMMON_FULL.RR.JobControl (@CmdT2): Executed: cut -d',' -f2 /data/ETL1/GE_CHARGE_TBLEXXTCHK_2.txt | tr -d '\n';
Reply=0
Output from command ====>
YES

Item #: 231
Event ID: 230
Timestamp: 2018-06-06 15:01:14
Type: Info
User Name: dsadm
Message Id: IIS-DSTAGE-RUN-I-0019
Message: ETL1_GE_COMMON_FULL.RR.JobControl (@Coordinator): Note: Sequencer 'Sequencer_447' was entered, but never exited

Item #: 232
Event ID: 231
Timestamp: 2018-06-06 15:01:14
Type: Info
User Name: dsadm
Message Id: IIS-DSTAGE-RUN-I-0019
Message: ETL1_GE_COMMON_FULL.RR.JobControl (@Coordinator): Note: Loop 'Loop1' was started, but did not complete all its iterations

Item #: 233
Event ID: 232
Timestamp: 2018-06-06 15:01:14
Type: Info
User Name: dsadm
Message Id: IIS-DSTAGE-RUN-I-0019
Message: ETL1_GE_COMMON_FULL.RR.JobControl (@Coordinator): Note: Loop 'Loop2' was started, but did not complete all its iterations

Item #: 234
Event ID: 233
Timestamp: 2018-06-06 15:01:14
Type: Info
User Name: dsadm
Message Id: IIS-DSTAGE-RUN-I-0019
Message: ETL1_GE_COMMON_FULL.RR.JobControl (@Coordinator): Summary of sequence run
15:00:33: Sequence started
15:00:33: ETL1_NZ_Datagov_Joblog_Insert (JOB ETL1_NZ_Datagov_Joblog_Insert.GE_CHARGE) started
15:00:38: ETL1_NZ_Datagov_Joblog_Insert (JOB ETL1_NZ_Datagov_Joblog_Insert.GE_CHARGE) finished, status=1 [Finished OK]
15:00:38: ETL1_NZ_SQLSRVR_PreCheck_Metadata1 (JOB ETL1_NZ_SQLSRVR_PreCheck_Metadata1.GE_CHARGE) started
15:00:51: ETL1_NZ_SQLSRVR_PreCheck_Metadata1 (JOB ETL1_NZ_SQLSRVR_PreCheck_Metadata1.GE_CHARGE) finished, status=1 [Finished OK]
15:00:51: ETL1_NZ_PreCheck_Metadata2 (JOB ETL1_NZ_PreCheck_Metadata2.GE_CHARGE) started
15:01:03: ETL1_NZ_PreCheck_Metadata2 (JOB ETL1_NZ_PreCheck_Metadata2.GE_CHARGE) finished, status=1 [Finished OK]
15:01:03: ETL1_NZ_Datagov_View1_Ext_Params (JOB ETL1_NZ_Datagov_View1_Ext_Params.GE_CHARGE) started
15:01:06: ETL1_NZ_Datagov_View1_Ext_Params (JOB ETL1_NZ_Datagov_View1_Ext_Params.GE_CHARGE) finished, status=1 [Finished OK]
15:01:06: MoveListToDB (JOB MoveTableList) started
15:01:08: MoveListToDB (JOB MoveTableList) finished, status=1 [Finished OK]
15:01:08: Loop2 loop iteration 1 started
15:01:08: Get1TblFromDbThread2 (JOB Get1TableUpdtList.Thread2) started
15:01:09: Loop1 loop iteration 1 started
15:01:09: Get1TblFromDbThread1 (JOB Get1TableUpdtList.Thread1) started
15:01:11: Get1TblFromDbThread2 (JOB Get1TableUpdtList.Thread2) finished, status=1 [Finished OK]
15:01:11: Unixcmd_GetDateValueT2 (COMMAND cat /data/ETL1/GE_CHARGE_EXT_PARAMS_2.txt | wc -l | tr -d '\n';) started
15:01:11: Unixcmd_GetDateValueT2 finished, reply=0
15:01:11: Pass_paramsT2 (COMMAND head -1 /data/ETL1/GE_CHARGE_EXT_PARAMS_2.txt | tail -1 | tr -d '\n';) started
15:01:11: Pass_paramsT2 finished, reply=0
15:01:11: GE_ETL1_Chk_SourceTableT2 (JOB GE_ETL1_Chk_SourceTable_GE.GE_CHARGE_THREAD2) started
15:01:12: Get1TblFromDbThread1 (JOB Get1TableUpdtList.Thread1) finished, status=1 [Finished OK]
15:01:12: Unixcmd_GetDateValue (COMMAND cat /data/ETL1/GE_CHARGE_EXT_PARAMS_1.txt | wc -l | tr -d '\n';) started
15:01:12: Unixcmd_GetDateValue finished, reply=0
15:01:14: GE_ETL1_Chk_SourceTableT2 (JOB GE_ETL1_Chk_SourceTable_GE.GE_CHARGE_THREAD2) finished, status=1 [Finished OK]
15:01:14: CmdT2 (COMMAND cut -d',' -f2 /data/ETL1/GE_CHARGE_TBLEXXTCHK_2.txt | tr -d '\n';) started
15:01:14: CmdT2 finished, reply=0
15:01:14: Sequence finished OK

Item #: 235
Event ID: 234
Timestamp: 2018-06-06 15:01:14
Type: Control
User Name: dsadm
Message Id: IIS-DSTAGE-RUN-I-0077
Message: Finished Job ETL1_GE_COMMON_FULL.RR.

End of report.
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

Unfortunately there is little to see in the log apart from the messages that "Note: Loop 'Loop1' was started, but did not complete all its iterations"

It is hard to visualize what your Job Looks like from the description without a picture, but perhaps UCDI has already addressed the issue. If you have a sequencer set to "all" you must ensure that each Input link to that sequencer has an Output. From your description
So data can go down 1 or both Task
it seems that this is not the case.
Post Reply