Page 1 of 1

ETL Job rowcount and other details

Posted: Tue Sep 12, 2017 2:43 pm
by Vrisha
I have a requirement that once the job got finished, it should populate the
JOB_NAME,
PROCESS_EXECUTION_ID,
PROCESS_START_DATE,
PROCESS_END_DATE,
PROCESS_DURATION,
ROWCOUNT_EXTRACTED,
ROWCOUNT_INSERTED,
ROWCOUNT_UPDATED,
ROWCOUNT_ERROR into a table called D_ETL_AUDIT_JOB_LOG.

Say for example, if I have 10 jobs after each job got finished , there should be an entry in D_ETL_AUDIT_JOB_LOG.

Do I need to create another parallel job to do this or should I use the sequencer to do this. Please let me know.

Thanks.

Posted: Tue Sep 12, 2017 4:21 pm
by Mike
Why reinvent a redundant wheel? Everything you need is already available in DSODB.

Mike

Posted: Wed Sep 13, 2017 6:33 am
by Vrisha
Thanks for your reply, Mike. As you said, I went through this website 'https://www-304.ibm.com/support/docview ... wg21509567' and I checked the file 'D:\IBM\InformationServer\Server\DSODB\logs\AppWatcher_Cmd.

AppWatcher_Cmd has the contents like below

2017/08/03 21:10:15: DSODB is currently turned off. AppWatcher will not run.
2017/09/07 15:47:45: AppWatcher start requested.
2017/09/07 15:47:45: DSODB is currently turned off. AppWatcher will not run.
2017/09/07 21:20:10: AppWatcher start requested.
2017/09/07 21:20:10: DSODB is currently turned off. AppWatcher will not run.

Last it ran on 2017/09/07. Am I looking at the correct place? Please let me know. Thanks.

Posted: Wed Sep 13, 2017 6:42 am
by chulett
I haven't been through all of the documentation (thanks Paul) but perhaps something here might help.

Posted: Wed Sep 13, 2017 6:45 am
by Vrisha
Thanks, Craig. I will look into it.

Posted: Wed Sep 13, 2017 8:50 am
by PaulVL
Work with your DataStage admin team to see if the DSODB database is configured for your environment. It's not a mandatory item. But it is super nice to have.

Posted: Wed Sep 13, 2017 8:59 am
by Vrisha
Hi Craig / Mike/ Paul,

I went through the documentation, it was very useful. But in our environment they didn't build/configured the DSODB database and not ready to do so.

Is there any other way to get the information without DSODB.

Thanks.

Posted: Wed Sep 13, 2017 11:36 am
by chulett
Sure but it's messy and painful... which is the whole reason that the DSODB was added to the product.

You can leverage the API for this, either from BASIC routines or from the command line depending on where your skillset lies, equivalent functions exist in both. For BASIC routines, search the documentation for the DSGet* functions. From the command line, leverage dsjob and the equivalent functions there. I would suggest the output be simply a flat file which, once fully populated, could be easily loaded into your table. This rather than each iteration trying to push records directly there. My two cents.

Posted: Wed Sep 13, 2017 2:16 pm
by Vrisha
Thanks, Craig. I will try the methods suggested by you and keep you posted.