DSXchange: DataStage and IBM Websphere Data Integration Forum
View next topic
View previous topic
Add To Favorites
This topic is not resolved, but there is a WORKAROUND.
Author Message
JRodriguez



Group memberships:
Premium Members

Joined: 19 Nov 2005
Posts: 413
Location: New York City
Points: 4531

Post Posted: Wed Mar 28, 2018 1:05 pm Reply with quote    Back to top    

DataStage® Release: 11x
Job Type: Parallel
OS: Unix
Additional info: Kicking off BPM processes
Hi All,
All Information Server events are published as kafka events in the "InfosphereEvents" topic but DataStage. The events could be consumed with any kafka/Jason client or a command line consumer well documented in the Big Blue website

Now we are trying to add the DataStage events (abort) to the existing and default topic for Information Server to kick off BPM processes depending on the source of the events. I know that I could create a new Kafka topic, a new producer and just do my own stuff, I know also that I could feed the Subscription Manager with the exceptions from the DataStage process and do whatever is needed from there... but think about it for a moment, adding the DS events to the existing kafka topic makes a bunch of stuff much more easier and straight forward to maintain...a single point of configuration and a single source for the Events.

have anyone out there tried something similar? Would you mind sharing your ideas/solution?

_________________
Julio Rodriguez
ETL Developer by choice

"Sure we have lots of reasons for being rude - But no excuses
JRodriguez



Group memberships:
Premium Members

Joined: 19 Nov 2005
Posts: 413
Location: New York City
Points: 4531

Post Posted: Mon Apr 16, 2018 7:53 am Reply with quote    Back to top    

Ended up submitting an RFE for to see if the feature could be added in the future. If you feel like is a useful feature please vote for this request, the link is listed below

https://www.ibm.com/developerworks/rfe/execute?use_case=viewRfe&CR_ID=118932

In the meantime if someone is looking to get going, shell scripting with redirection could be used as below

./bin/dsjob -run ETLProject Job_1 >Event_log 2>&1;cat Event_log|/opt/IBM/InformationServer/shared-open-source/kafka/install/bin/kafka-console-producer.sh --broker-list `hostname`:59092 --topic InfosphereEvents

_________________
Julio Rodriguez
ETL Developer by choice

"Sure we have lots of reasons for being rude - But no excuses
Rate this response:  
Not yet rated
Display posts from previous:       

Add To Favorites
View next topic
View previous topic
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum



Powered by phpBB © 2001, 2002 phpBB Group
Theme & Graphics by Daz :: Portal by Smartor
All times are GMT - 6 Hours