HTTP Post from DataStage

Dedicated to DataStage and DataStage TX editions featuring IBM<sup>®</sup> Service-Oriented Architectures.

Moderators: chulett, rschirm

Post Reply
ag_ram
Premium Member
Premium Member
Posts: 524
Joined: Wed Feb 28, 2007 3:51 am

HTTP Post from DataStage

Post by ag_ram »

We are in process of designing a new ETL process here. Our requirement is to extract messages from MQ and post it to destination URL by making an HTTP post. The extract job will be scheduled and the job should pick up all the messages in the queue in one go and then do the data transformation and do the data posting. In this regard I have the below queries

1. Can we do an HTTP post from Data stage?

2. Consider the scenario where we have 10 messages to be posted using HTTP and the HTTP post job fails after sending 4th message.
a. What will happen to the remaining messages pending to be sent out?
b. What will happen to the ones that are already sent out?

3. Consider the scenario where the 4th message failed in transformation job. Can we continue with the rest of the messages discarding the one that is failed?
eostic
Premium Member
Premium Member
Posts: 3838
Joined: Mon Oct 17, 2005 9:34 am

Post by eostic »

There is nothing built-in for HTTP post.... JavaPack is the most immediate method I would think of using to do this, if only becuase it's fairly easy to implement a java class with DS using JavaPack, and it's fairly straightforward to do HTTP work inside of a java class....

...as for error handling, nothing is assured with HTTP, so that logic would have to exist in your java class, or with your integration with DataStage, probably dependent upon the relationship between "rows" in DataStage and the "messages" you are reading from MQ and/or posting to HTTP.

You have lots of options, but all are just creative ways of using DataStage..... create audit files, browse the MQ queue instead of destructively reading it, let your java class save messages somewhere on its own, etc.

Ernie
Naren12345babu
Premium Member
Premium Member
Posts: 76
Joined: Wed Jan 06, 2016 9:40 am
Location: Bangalore

Post by Naren12345babu »

what is the Java pack you mentioned in your reply ? Kindly help me to understand this.
Thanks
Naren Babu Ch
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Here is a link to the (easily searched for) IBM InfoSphere DataStage Java Pack documentation.
-craig

"You can never have too many knives" -- Logan Nine Fingers
Post Reply