DSXchange: DataStage and IBM Websphere Data Integration Forum
View next topic
View previous topic
Add To Favorites
Author Message
ag_ram



Group memberships:
Premium Members

Joined: 28 Feb 2007
Posts: 524

Points: 4344

Post Posted: Sun Oct 07, 2007 11:28 pm Reply with quote    Back to top    

DataStage® Release: 7x
Job Type: Parallel
OS: Unix
We are in process of designing a new ETL process here. Our requirement is to extract messages from MQ and post it to destination URL by making an HTTP post. The extract job will be scheduled and the job should pick up all the messages in the queue in one go and then do the data transformation and do the data posting. In this regard I have the below queries

1. Can we do an HTTP post from Data stage?

2. Consider the scenario where we have 10 messages to be posted using HTTP and the HTTP post job fails after sending 4th message.
a. What will happen to the remaining messages pending to be sent out?
b. What will happen to the ones that are already sent out?

3. Consider the scenario where the 4th message failed in transformation job. Can we continue with the rest of the messages discarding the one that is failed?
eostic

Premium Poster



Group memberships:
Premium Members

Joined: 17 Oct 2005
Posts: 3771

Points: 30287

Post Posted: Thu Oct 11, 2007 3:47 am Reply with quote    Back to top    

There is nothing built-in for HTTP post.... JavaPack is the most immediate method I would think of using to do this, if only becuase it's fairly easy to implement a java class with DS using JavaPack, and it's fairly straightforward to do HTTP work inside of a java class....

...as for error handling, nothing is assured with HTTP, so that logic would have to exist in your java class, or with your integration with DataStage, probably dependent upon the relationship between "rows" in DataStage and the "messages" you are reading from MQ and/or posting to HTTP.

You have lots of options, but all are just creative ways of using DataStage..... create audit files, browse the MQ queue instead of destructively reading it, let your java class save messages somewhere on its own, etc.

Ernie
Rate this response:  
Not yet rated
Naren12345babu



Group memberships:
Premium Members, Heartland Usergroup

Joined: 06 Jan 2016
Posts: 68
Location: Bangalore
Points: 841

Post Posted: Fri Jan 08, 2016 11:27 am Reply with quote    Back to top    

what is the Java pack you mentioned in your reply ? Kindly help me to understand this.

_________________
Thanks
Naren Babu Ch
Rate this response:  
Not yet rated
chulett

Premium Poster


since January 2006

Group memberships:
Premium Members, Inner Circle, Server to Parallel Transition Group

Joined: 12 Nov 2002
Posts: 42615
Location: Denver, CO
Points: 219398

Post Posted: Fri Jan 08, 2016 12:09 pm Reply with quote    Back to top    

Here is a link to the (easily searched for) IBM InfoSphere DataStage Java Pack documentation.

_________________
-craig

And I'm hovering like a fly, waiting for the windshield on the freeway...
Rate this response:  
Not yet rated
Display posts from previous:       

Add To Favorites
View next topic
View previous topic
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum



Powered by phpBB © 2001, 2002 phpBB Group
Theme & Graphics by Daz :: Portal by Smartor
All times are GMT - 6 Hours